Stability of a linear autonomous non-conservative system in the presence of potential, gyroscopic, dissipative, and non-conservative positional forces is studied. The cases when the non-conservative system is close to a gyroscopic system or to a circulatory one are examined. It is known that marginal stability of gyroscopic and circulatory systems can be destroyed or improved up to asymptotic stability due to action of small non-conservative positional and velocity-dependent forces. The present paper shows that in both cases the boundary of the asymptotic stability domain of the perturbed system possesses singularities such as “Dihedral angle” and “Whitney umbrella” that govern stabilization and destabilization. In case of two degrees of freedom, approximations of the stability boundary near the singularities are found in terms of the invariants of matrices of the system. As an example, the asymptotic stability domain of the modified Maxwell–Bloch equations is investigated with an application to the stability problems of gyroscopic systems with stationary and rotating damping.