As a result of progressive use of BIM in the AEC sector, the amount of diverse project information is increasing rapidly, thus necessitating interoperability of tools, compatibility of data, effective collaboration and sophisticated data management. Media-rich VR and AR environments have been proven to help users better understand design solutions, however, they have not been quite advanced in supporting interoperability and collaboration. Relying on capabilities of openBIM and IFC schema, this study posits that this shortcoming of VR and AR environment could be addressed by use of BIM server concept allowing for concurrent multiuser and low-latency communication between applications. Successful implementation of this concept can ultimately mitigate the need for advanced technical skills for participation in design processes and facilitate the generation of more useful design solutions by early involvement of stakeholders and end-users in decision making. This paper exemplifies a method for integration of BIM data into immersive VR and AR environments, in order to streamline the design process and provide a pared-down agnostic openBIM system with low latency and synchronised concurrent user accessibility that gives the ``right information to the right people at the right time''. These concepts have been further demonstrated through development of a prototype for openBIM-Tango integrated virtual showroom for offsite manufactured production of self-build housing. The prototype directly includes BIM models and data from IFC format and interactively presents them to users on both VR immersive and AR environments, including Google Tango enabled devices. This paper contributes by offering innovative and practical solutions for integration of openBIM and VR/AR interfaces, which can address interoperability issues of the AEC industry.