Image-to-Class (I2C) distance firstly proposed in the naive Bayes nearest neighbour (NBNN) classifier has shown its effectiveness in image classification. However, due to the large number of nearest-neighbour search, I2C-based methods are extremely time-consuming, especially with highdimensional local features. In this paper, with the aim to improve and speed up I2C-based methods, we propose a novel discriminative embedding method based on I2C for local feature dimensionality reduction. Our method 1) greatly reduces the computational burden and improves the performance of I2C-based methods after reduction; 2) can well preserve the discriminative ability of local features, thanks to the use of I2C distances; and 3) provides an efficient closed-form solution by formulating the objective function as an eigenvector decomposition problem. We apply the proposed method to action recognition showing that it can significantly improve I2C-based classifiers.