Electromagnetic ion cyclotron (EMIC) waves play an important role in relativistic electron losses in the radiation belts through diffusion via resonant wave‐particle interactions. We present a new approach for calculating bounce and drift‐averaged EMIC electron diffusion coefficients. We calculate bounce‐averaged diffusion coefficients, using quasi‐linear theory, for each individual Combined Release and Radiation Effects Satellite (CRRES) EMIC wave observation using fitted wave properties, the plasma density and the background magnetic field. These calculations are then combined into bounce‐averaged diffusion coefficients. The resulting coefficients therefore capture the combined effects of individual spectra and plasma properties as opposed to previous approaches that use average spectral and plasma properties, resulting in diffusion over a wider range of energies and pitch angles. These calculations, and their role in radiation belt simulations, are then compared against existing diffusion models. The new diffusion coefficients are found to significantly improve the agreement between the calculated decay of relativistic electrons and Van Allen Probes data.