In this paper, we propose a low-latency virtual reality (VR) delivery system where an unmanned aerial vehicle (UAV) base station (U-BS) is deployed to deliver VR content from a cloud server to multiple ground VR users. Each VR input data requested by the VR users can be either projected at the U-BS before transmission or processed locally at each user. Popular VR input data is cached at the U-BS to further reduce backhaul latency from the cloud server. For this system, we design a low-complexity iterative algorithm to minimize the maximum communications and computing latency among all VR users subject to the computing, caching and transmit power constraints, which is guaranteed to converge. Numerical results indicate that our proposed algorithm can achieve a lower latency compared to other benchmark schemes. Moreover, we observe that the maximum latency mainly comes from communication latency when the bandwidth resource is limited, while it is dominated by computing latency when computing capacity is low. In addition, we find that caching is helpful to reduce latency.