Gait abnormalities are typically derived from neurological conditions or orthopaedic problems and can cause severe consequences such as limited mobility and falls. Gait analysis plays a crucial role in monitoring gait abnormalities and discovering underlying deficits can help develop rehabilitation programs. Contemporary gait analysis requires a multi-modal gait analysis approach where spatio-temporal, kinematic and muscle activation gait characteristics are investigated. Additionally, protocols for gait analysis are going beyond labs/clinics to provide more habitual insights, uncovering underlying reasons for limited mobility and falls during daily activities. Wearables are the most prominent technology that are reliable and allow multi-modal gait analysis beyond the labs/clinics for extended periods. There are established wearable-based algorithms for extracting informative gait characteristics and interpretation. This paper proposes a multi-layer fusion framework with sensor, data and gait characteristics. The wearable sensors consist of four units (inertial and electromyography, EMG) attached to both legs (shanks and thighs) and surface electrodes placed on four muscle groups. Inertial and EMG data are interpreted by numerous validated algorithms to extract gait characteristics in different environments. This paper also includes a pilot study to test the proposed fusion approach in a small cohort of stroke survivors. Experimental results in various terrains show healthy participants experienced the highest pace and variability along with slightly increased knee flexion angles (≈1°) and decreased overall muscle activation level during outdoor walking compared to indoor, incline walking activities. Stroke survivors experienced slightly increased pace, asymmetry, and knee flexion angles (≈4°) during outdoor walking compared to indoor. A multi-modal approach through a sensor, data and gait characteristic fusion presents a more holistic gait assessment process to identify changes in different testing environments. The utilisation of the fusion approach presented here warrants further investigation in those with neurological conditions, which could significantly contribute to the current understanding of impaired gait.