The initiation and propagation of shear bands is an important mode of localized inhomogeneous deformation that occurs in a wide range of materials. In metallic glasses, shear band development is considered to center on a structural heterogeneity, a shear transformation zone that evolves into a rapidly propagating shear band under a shear stress above a threshold. Deformation by shear bands is a nucleation-controlled process, but the initiation process is unclear. Here we use nanoindentation to probe shear band nucleation during loading by measuring the first pop-in event in the load–depth curve which is demonstrated to be associated with shear band formation. We analyze a large number of independent measurements on four different bulk metallic glasses (BMGs) alloys and reveal the operation of a bimodal distribution of the first pop-in loads that are associated with different shear band nucleation sites that operate at different stress levels below the glass transition temperature, Tg. The nucleation kinetics, the nucleation barriers, and the density for each site type have been determined. The discovery of multiple shear band nucleation sites challenges the current view of nucleation at a single type of site and offers opportunities for controlling the ductility of BMG alloys.