Quantitative analysis of cell mitosis, the process by which cells regenerate, is important in cell biology. Automatic cell mitosis detection can greatly facilitate the investigation of cell life cycle. However, cell-type diversity, cell non-rigid deformation and high cell density pose difficulties on handcrafting visual features for traditional approaches. Aided by massively captured microscopy image sequences, deep neural networks have recently become available for automatic cell mitosis detection. This paper proposes an end-to-end framework named as F3D-CNN for mitosis detection, and F3D-CNN is directly trained from data without requiring designing domain dependent features. Well-trained F3D-CNN first filters out potential mitosis events based on the static information in each individual image, and further discriminates candidates by incorporating the spatiotemporal information from image sequences. The state-of-the-art performance of F3D-CNN was confirmed in experiments on two public datasets (multipotent C3H10T1/2 mesenchymal stem cells and C2C12 myoblastic stem cells).