Mixed reality is often wrongly mistaken for being synonymous with augmented reality, but the truth is that mixed reality is somewhere in between augmented and virtual reality; combining the real world with virtual objects that are designed to appear as if they belong in the real world. One could say that mixed reality is a fusion of the best aspects of both virtual and augmented reality, allowing physical and virtual objects to coexist and work together in real time. When you consider that the virtual reality industry in the UK alone is expected to be worth £354m in 2020, you can understand that mixed reality could be equally big business.
Mixed reality: from the operating theatre to the warehouse
The idea behind mixed reality is to go beyond visual displays in terms of virtual reality and incorporate environments, spatial sounds, and locations that fuse human perception and conventional reality with digital interaction. This has all been aided by the development of technologies that have enabled platforms to overlay virtual content on the real world that reacts in sync together for the benefit of our lives, not just our entertainment. Already mixed reality is being used by experienced surgeons to discuss and consult on difficult operations, beamed into an operating theatre in The Royal London Hospital as digital 3D avatars. It’s a tech development that could revolutionize the healthcare industry, allowing experts to assist from across the world, making the world a significantly smaller place.
So, what are the other common uses of mixed reality devices today and where is the technology going in the coming years? Microsoft Windows is very much at the forefront of the mixed reality industry. Windows’ mixed reality differs to its virtual reality devices as it is created to encourage users to participate in a particular event or action, rather than peer in on the rails of a virtual world as a complete spectator. As we’ve already discussed, surgeons are already utilising the technology to the benefit of patients and Microsoft’s mixed reality technology is designed to work seamlessly within the Windows 10 operating system, which is now active on at least 400 million global devices, with over 197 billion hours of usage, to bring digital content to real-world environments and replace the real world with a wholly immersive digital experience.
Microsoft’s two leading mixed reality devices are the HoloLens and the Acer Windows Mixed Reality Headset. The former is regarded as a fully holographic mixed reality device and the developers’ kits were sold in 2016 for $3,000 to introduce the mixed reality world. Its see-through display enables users to picture the real-world environment through the digital lenses. The lenses overlay virtual objects and content in the real world as if it were really there, and the headset works with complete freedom and rotation. In February 2017, the HoloLens was used to create an app to help visually impaired people negotiate their way around the home and beyond. Developer Javier Davalos demonstrates his latest mixed reality app which spatially maps out a room or space for a user to look in any direction and ascertain the proximity of objects through the medium of sound.
Sound is not the only technology used to take mixed reality to another level. The sheer power of mixed reality is its ability to recognize objects. For any user of mixed reality devices, the ability to acknowledge many different objects simultaneously is not too much of a challenge but it certainly is for machines and devices themselves. Instead, they rely on Optical Character Recognition (OCR) technology to recognize and read objects and both handwritten and printed characters which, in turn, can be converted into readable data and a digital output.
Mixed reality overlays and OCR technology
There are many simpler examples of where OCR technology is being adopted, such as automation and manufacturing. Warehouse workers wearing smart glasses can work with both of their hands while the glasses overlay virtual information of codes printed on objects and materials just by looking at them through the lenses. It could also radically change the way homeowners input meter readings to their energy suppliers, with smart glasses and OCR technology identifying the values displayed on the meter and translated into digital data immediately for utilities firms to calculate bills. In the entertainment sector, iGaming firms are already utilizing OCR technology to quickly recognize and read cards and numbers in live casino card games such as casino hold’em and blackjack, displaying them swiftly with on-screen overlays to enhance the live casino table game experience. OCR technology is also used to scan physical documents and books and convert them into fully-editable digital documents. The British Library is utilizing the technology as part of its bulk digitalization of historical documentation.
Meanwhile, the Acer Windows Mixed Reality Headsets, available for around $299, are beginning to transform the way gamers play their favorite virtual reality titles on Steam, the popular online gaming community. Valve has announced that the Mixed Reality Headsets are now compatible with VR games on Steam, with the software to play them being offered entirely free as part of an “early access” preview. This suggests that some of the 30+ SteamVR games are still ridden with bugs and those fortunate enough to own a Windows Mixed Reality Headset can help pinpoint and eradicate such issues prior to the official launch.
The future: more data-driven experiences and environments
Microsoft is certainly pushing hard the concept of mixed reality, far more than they ever did with virtual and augmented reality. This could be partly for commercial reasons of course, as the company seeks to bring the technology into both the consumer and enterprise sectors simultaneously. There’s no doubt that the clock is ticking for digital content creators to avail themselves of dedicated mixed reality studio environments that will enable them to provide new dimensions of visualization and simulation, creating more engaging and stimulating mixed reality experiences underpinned by more data, video, audio and 3D holographs than ever before.