In millimeter wave (mmWave) systems, it is challenging to ensure reliable communication links due to the high sensitivity to the presence of blockages. In order to improve the robustness of mmWave systems in the presence of random blockages, we consider the deployment of multiple reconfigurable intelligent surfaces (RISs) to enhance the spatial diversity gain, and the design of robust beamforming schemes based on stochastic optimization methods that minimize the maximum outage probability among multiple users so as to ensure fairness. Under the stochastic optimization framework, we adopt the stochastic majorization–minimization (SMM) method and the stochastic successive convex approximation (SSCA) method to construct deterministic surrogate problems at each iteration, and to obtain closed-form solutions of the precoding matrix at the base station (BS) and the beamforming vectors at the RISs. Both stochastic optimization methods are proved to converge to the set of stationary points of the original stochastic problems. Simulation results show that the proposed robust beamforming for RIS-aided systems can effectively compensate for the performance loss caused by the presence of random blockages, especially when the blockage probability is high.