In this paper, we investigate multiple unmanned aerial vehicles (UAVs) enabled data collection system in Internet of Things (IoT) networks with time windows, where multiple rotary-wing UAVs are dispatched to collect data from time constrained terrestrial IoT devices. We aim to jointly minimize the number and the total operation time of UAVs by optimizing the UAV trajectory and hovering location. To this end, an optimization problem is formulated considering the energy budget and cache capacity of UAVs as well as the data transmission constraint of IoT devices. To tackle this mix-integer non-convex problem, we decompose the problem into two subproblems: UAV trajectory and hovering location optimization problems. To solve the first subproblem, an modified ant colony optimization (MACO) algorithm is proposed. For the second subproblem, the successive convex approximation (SCA) technique is applied. Then, an overall algorithm, termed MACO-based algorithm, is given by leveraging MACO algorithm and SCA technique. Simulation results demonstrate the superiority of the proposed algorithm.