In this study we investigate the ability of several different machine learning models to provide probabilistic predictions as to whether interplanetary shocks observed upstream of the Earth at L1 will lead to immediate (Sudden Commencements, SCs) or longer lasting magnetospheric activity (Storm Sudden Commencements, SSCs). Four models are tested including linear (Logistic Regression), nonlinear (Naive Bayes and Gaussian Process), and ensemble (Random Forest) models and are shown to provide skillful and reliable forecasts of SCs with Brier Skill Scores (BSSs) of ∼0.3 and ROC scores >0.8. The most powerful predictive parameter is found to be the range in the interplanetary magnetic field. The models also produce skillful forecasts of SSCs, though with less reliability than was found for SCs. The BSSs and ROC scores returned are ∼0.21 and 0.82, respectively. The most important parameter for these predictions was found to be the minimum observed BZ. The simple parameterization of the shock was tested by including additional features related to magnetospheric indices and their changes during shock impact, resulting in moderate increases in reliability. Several parameters, such as velocity and density, may be able to be more accurately predicted at a longer lead time, for example, from heliospheric imagery. When the input was limited to the velocity and density the models were found to perform well at forecasting SSCs, though with lower reliability than previously (BSSs ∼ 0.16, ROC Scores ∼ 0.8), Finally, the models were tested with hypothetical extreme data beyond current observations, showing dramatically different extrapolations.