In this work, we propose a mixed radio frequency (RF) and multiple-input-multiple-output (MIMO) free-space optical (FSO) system based on a variable-gain dual-hop relay transmission scheme. The RF channel is modeled by Rayleigh distribution and Gamma–Gamma turbulence distribution is adopted for the MIMO FSO link, which accounts for the equal gain combining diversity technique. Moreover, new closed-form mathematical formulas are obtained including the cumulative distribution function, probability density function, moment generating function, and moments of equivalent signal-to-noise ratio of the dual-hop relay system based on Meijer’s G function. As such, we derive the novel analytical expressions of the outage probability, the higher-order fading, and the average bit error rate for a range of modulations in terms of Meijer’s G function. Furthermore, the exact closed-form formula of the ergodic capacity is derived based on the bivariate Meijer’s G function. The evaluation and simulation are provided for system performance, and the effect of spatial diversity technique is discussed as well.