Transforming the way the world works.

Mixed Reality for the AEC Industry

Extending Trimble’s Product Capabilities with Microsoft HoloLens

Mixed Reality technology blends real world objects with digital content, interactively, and in real time. It helps users efficiently interpret physical and digital information, and the spatial relations between them. This article describes the benefits of integrating Microsoft HoloLens’ Mixed Reality technology with Trimble products, during key stages of the Design-Build-Operate (DBO) workflow.

Microsoft HoloLens

Microsoft HoloLens is a wearable, self-contained holographic computer. The device features a see-through, holographic display and advanced sensors that map the physical environment. HoloLens enables users to interact with 3D holograms blended into the real world. In this Mixed Reality environment, users can pin holograms to physical objects and interact with data using GGV (gesture, gaze and voice) commands.

Between Reality and Virtuality

Mixed Reality (MR) spans the purely virtual and purely real environments. In the context of the building industry, this is the phase in which digital and real content co-exist, where architectural design collides with reality, and where construction teams transform digital content into physical objects.

Reality - Virtuality Continuum, Paul Milgram (1)
Reality - Virtuality Continuum, Paul Milgram (1)

In today’s practice, the interpretation of digital content and its translation to real world objects, heavily depend on the user’s spatial understanding. This is an error-prone process, demanding a highly skilled workforce. Interpretation errors are common during the design and construction stages, and often result in poor quality, cost overruns and schedule delays.

Visualizing digital content as holograms in the context of the physical world bridges the gap between virtual and real, and eliminates the current workflow’s inefficiencies. In addition, while our physical world is finite, Mixed Reality presents the opportunity for an infinite environment in which additional data such as schedule, specs, and simulation can be overlaid onto the world, creating a hyper-reality environment.

Real 3D Visualization

The AEC industry is spatial by definition. The last few years’ transition from 2D documents to 3D models was a natural evolution, which improved team communication and coordination. 3D models are common today, but still, interacting with volumetric data behind a 2D screen is extremely limited. Mixed Reality and holographic technology bring the models out of the screen and provide users the ability to engage and interact with design data more intuitively. Also, unleashing the 3D model democratizes the data by offering a natural way to experience and understand the design. While years of education and practice might train architects to visualize their designs in 3D, other stakeholders have a hard time deciphering them. Using Mixed Reality, stakeholders can walk around and explore the design in real 3D without the need for an expert to guide them and dictate their point of view.

The freedom to move along the Reality - Virtuality Continuum is additional advantage of Mixed Reality, with a major impact on the efficiency along the design-build-operate workflow. For some decisions or specific tasks, a real “unmodeled” world is required as visual input. For other tasks, a mixed reality – a “world partially modeled” – or even a completely immersive Virtual Reality mode is ideal. With Mixed Reality the user controls the “mix” and adjusts it according to their task requirements.

Moving along the Reality - Virtuality Continuum: in this proof of concept, a Trimble SketchUp model is displayed using Microsoft HoloLens. To best explore and experience the design, stakeholders can switch between a tabletop holographic view and an immersive view mode.

Mixed Reality and holographic technology also improve remote collaboration. The ability to share 3D holograms with remote stakeholders, including participants’ position and gaze, improves communication and collaboration efficiency.

Remote collaboration: in this proof of concept, Trimble Connect data is shared with remote users. Using Microsoft HoloLens’ holographic technology, the remote teams experience the ‘spatial presence’ of the design model, share views, position, and gaze information, and communicate over real 3D holograms.

Seamless Translation

In every construction project there is a decisive process, in which design is transformed into reality, abstract ideas are translated into physical objects. As almost every project is unique, site-specific and labor-based, intensive and accurate “translation” of the design information is required. The current translation process of turning design documents into reality is a constant struggle. It is a complex cognitive process in which wrong interpretation of data often leads to costly errors, quality issues, and rework. Digital 3D models reduce the level of abstraction, and help clarify the design intent, but still, at the execution stage, there is a clear boundary between documentation and the real world.

Mixed Reality technology blurs this line. With MR, the information is superimposed on the physical environment. This minimizes the need for translation and reduces subjective interpretation. The user visualizes the design in context, and can better understand, interact with and execute the required actions in real time. The immediate visual feedback also supports a more effective production control process. Overlaying design on as-build structures reveals any deviation and provides immediate feedback.

Seamless Translation: a holographic 3D BIM model from Trimble Connect is visualized on-site using Microsoft HoloLens.

Extending the Digital Thread

Integration of business workflows and improved communication are clear goals in today’s AEC industry. The silo structure and the need to re-create data, when moving from one stage to the other, are still sources for major inefficiencies. Project delivery methods like IPD and technology changes such as BIM support this trend and have proven to positively impact project quality, cost and schedule.

Mixed Reality technology supports integration and collaboration along three main axes:

The Project Stage Axis: Design to Build (and Operate):

Moving along this axis, MR brings 3D data to live and puts information in the user’s hands without the need to change or adjust the data format. A sign-off BIM model can be projected in context to guide construction teams on-site, and later, by facility management teams when evaluating required changes or maintenance work.

The Digital to Physical Axis:

Holographic display is not limited to 3D models. In fact, using Microsoft HoloLens, multi-layered datasets can be overlaid as holograms on the real world. A partial list includes 2D documents, energy analysis, light simulation, acoustics, layout data, and equipment metadata. The abilities to integrate digital and physical content and present data in context improve communication and leverage confidence in decision--making.

Office to Field Axis: Design to Build, Build to Design

Sharing up-to-date design data with on-site teams is crucial in the dynamic construction environment. Using Mixed Reality technology, construction, teams benefit from access to geo-located, context-based data, which can be projected and anchored to the physical environment around them. The ability to map the as-build physical environment supports communication from the field to the office and completes the Build to Design – Design to Build loop.


Mixed Reality will have a significant impact on the AEC industry over the next few years. The technology addresses some of the industry’s major inefficiencies during the design, construction and operation stages. Integrated with Trimble products and with BIM models as a main data sources, Mixed Reality improves communication, tightens workflow integration, and enables real time collaboration with remote teams.

While it is possible to describe the benefits of Mixed Reality in words, one must experience with it first hand to appreciate its power and realize how it can transform AEC industry.


  1. Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum" (pdf).