With the widespread integration of renewable energy sources (RES) in the power grid, it has become more crucial to predict future energy generation multiple steps ahead to support grid reliability and operational planning. However, renewable generation data often contains uncertainty and high variability due to weather changes, posing a major challenge to the benchmark forecasting methods. To deal with this, we propose a Bayesian probabilistic approach incorporated with bidirectional long-short term memory (BiLSTM) neural networks for multi-step ahead (MSA) renewable generation forecasting. The proposed method is examined on highly granular solar generation data from Ausgrid using probabilistic evaluation metrics such as Pinball loss and Winkler score. Moreover, a comparative analysis between MSA and the single-step ahead (SSA) forecasting is provided to test the effectiveness of Bayesian methods on different forecasting horizons. The numerical results clearly demonstrate that the proposed Bayesian BiLSTM outperforms standard BiLSTM and other benchmark methods for MSA forecasting in terms of error performance.