We describe the design of the HARMONI adaptive optics control system. HARMONI is the first light visible and nearIR integral field spectrograph for the ELT. It covers a large spectral range from 450nm to 2450nm, and can operate in two Adaptive Optics modes-SCAO (including a High Contrast capability) and LTAO-or without AO. The project is preparing for Final Design Reviews. Control systems for adaptive optics in the ELT era will be complex hardware and software systems. In order fully to realise the scientific potential of such large telescopes, the AO control system must meet demanding requirements in terms of latency, throughput, computational performance, and data distribution. In addition, the expected longevity of such systems require them to be sustainable and serviceable for a number of decades into the future. We describe the high-level design of the HARMONI AO control system in terms of its decomposition into functional modules, and their deployment to hardware. The system is partitioned into a "hard"real-time domain, in which the wavefront reconstruction pipeline runs at the frame-rate of the AO wavefront sensors; and a "soft"real-time domain in which computationally-demanding tasks such as tomographic reconstructor calculation are performed, at slower update rates. The vital infrastructure by which real-time telemetry data is distributed forms the third leg"of the design. The mapping of the design onto the framework provided by ESO's Real Time Control Toolkit (RTC-TK) is illustrated The system will be deployed on COTS hardware based on conventional CPU-based computer servers, with GPU acceleration in the "soft"real-time domain where necessary to meet performance requirements. Finally, the performance benchmarking carried out to validate the design is presented.