Mixed reality is increasingly becoming a part of our lives, and those of us working in software and hardware development should keep a keen eye on these technologies. We are all likely to be affected, and the demands of these technologies will impact the work we do.
The term “Mixed Reality” was coined in 1994 by Paul Milgram and Fumio Kishino to describe the continuum that exists between our perceived reality and a reality that has been generated or modified artificially. That continuum stretches between the real environment, at one extreme end of the spectrum, to an augmented reality, an augmented virtuality, and to a completely virtual environment on the extreme opposite end of the spectrum.1
Some additional definitions may be helpful here:
- Augmented Reality (AR) is a term used to describe the technology of using digital components to enhance our experience in the real world. Information not ordinarily available to our natural senses is provided by AR tools to improve our interactions with the world around us.
- The Mixed Reality term Augmented Virtuality is used to describe a system consisting mostly of an artificially generated environment that also has some real elements. This includes using real-world images within a virtual space, for example. The common lexicon used to describe these technologies usually defines Augmented Virtualty (AV) as either AR or Virtual Reality (VR).
- Virtual Reality is getting a great deal of attention these days, and it refers to a completely digital recreation of a real setting. It is not uncommon for a 360-video experience to be labeled as VR, but that is not always technically correct. As VR technology improves and audiences become more familiar with the technology, they will increasingly expect to be more than a passive observer of a VR experience. They will expect full interactions with the artificial environment.
Virtual Reality’s wide-ranging applications
Not surprisingly, the entertainment industries are pushing the development of Virtual Reality (VR), and this is particularly true of the gaming industry. In a recent game developer’s conference held in San Francisco, attendees were surveyed, and 16% of those polled indicated they were actively involved in virtual reality game development. To put that in perspective, that was more than the game development of the two current Nintendo game systems combined.2 This year’s Sundance film festival held in January featured no less than 30 VR-related experiences.3 Movies increasingly are becoming more interactive and games are becoming increasingly visual. The merging of these industries through VR seems inevitable.
The entertainment and gaming industries seem to be driving VR development at the consumer level, but VR is prevalent in other areas as well. The military has used simulators to train personnel for many decades, and the use of VR has only enhanced training efforts. VR provides a no-risk means of exposing soldiers to dangerous situations to train for proper responses.4 This type of training is also being used by police departments for similar reasons.5 Not just on the streets, VR is being investigated as a way to give jurors a more detailed view of a crime scene and events that occurred. Researchers in Zurich have found it makes it easier for jurors to determine if someone is guilty or not.6
VR is also being employed in manufacturing. For example, Ford is using VR as a form of prototyping to inspect a vehicle design for flaws and usability.7 Ford is not alone. According to one international VR company, Peugeot, Renault, BMW, and Jaguar all have VR centers, and Lockheed Martin, Boeing, Airbus, Miele, and BAE also use VR to prototype their designs.8 Toyota has used VR to help educate teens on the dangers of distracted driving and they believe VR will be used extensively in education to make learning more efficient.9 Several companies dedicated to VR education are creating packages of content and instructor materials for K – 12 schools.10
Other examples of VR use include museums using it to give visitors to their website a chance to tour their museum remotely, real estate companies providing potential buyers the ability to explore a new listing, and in medicine, VR is helping to train surgeons in operating rooms. It is also being used to help soldiers and others dealing with PTSD. The list of applications of VR just goes on and on. The same is true for AR.
Augmented Reality may prove to be even more important
As much attention as VR is getting these days, Augmented Reality (AR) is poised to be many times bigger. According to a recent article on ExtremeTech.com, “Analyst Digi-Capital predicts an AR market of $90 billion annually by 2020, compared to $30 billion for VR.” 11 The difference is due to the fact that AR does not replace the user environment, but instead adds information to the user’s normal environment. This technology is poised to become ubiquitous in our daily lives.
A common example of AR use today in our lives is the use of a QR code to obtain information about something encountered in our environment. Snapping a picture with a smart phone of the QR code in an application that can read the code will result in additional information about an object or event presented at the point of interest. This technology is commonplace at retail shelves, but to give an idea of just how far this may go; one company is making use of QR codes for gravestones. They see a future in which people will snap the QR code on a headstone to see details of the deceased.12 It may seem strange today, but in a world where such information augmentation is everywhere, it will feel entirely normal.
Similar to QR code use, AR is also used to assist in visualizing how real elements will look with some sort of addition or modification by taking a picture and adding in artificial elements to see how they look together. Design software makes use of this sort of thing, making it possible, for example, to see how furniture or wall paint colors will look within an existing room.13 Retailers use this technology to see how a product or modification will look on store display shelves with existing products. This same technology has been employed in construction to visualize how a structure will fit and work in an existing environment.14
Some other uses of AR include an application in development that allows encrypted text to be viewed through AR glasses in unencrypted form. Also, real-time translation of foreign languages is increasingly being used in International communications and travel. Automotive windshield dashboard display, military use of heads-up displays, location-based notifications and alerts, parts identification based on a picture of the object, and in medicine, the aggregation of multiple displays into one combined display for a surgeon, are all examples of AR technology being used today.
We are the forefront of new technologies in VR and AR that will increasingly be commonplace in our lives. From entertainment to medicine and industrial use, these advancements will necessitate those of us in the software development lifecycle to stay informed and seek to find ways these technologies can be incorporated into our work.
We’d love to hear what you think of the VR/AR evolving landscape; share your thoughts with us on our Tweeter feed – https://twitter.com/quardev, @quardev
Resources:
- https://en.wikipedia.org/wiki/Mixed_reality
- https://www.businessinsider.com/game-developers-focusing-heavily-on-vr-content-2016-3
- https://www.theverge.com/2016/1/29/10871384/sundance-film-festival-2016-best-vr
- https://www.wareable.com/vr/how-vr-is-training-the-perfect-soldier-1757
- https://www.wired.com/2015/03/virtra/
- https://www.newscientist.com/article/dn26764-forensic-holodeck-to-transport-jury-to-the-crime-scene
- https://media.ford.com/content/fordmedia-mobile/fna/us/en/news/2013/12/12/new-virtual-lab-improves-ford-global-vehicle-quality–engineers-.html
- https://www.middlevr.com/resources/vr-and-engineering/
- https://www.theguardian.com/technology/2015/jan/15/toyota-oculus-rift-virtual-reality-distracted-driving
- https://techcrunch.com/2016/01/23/when-virtual-reality-meets-education/
- https://www.extremetech.com/extreme/224618-2016-is-turning-out-to-be-an-amazing-year-for-augmented-reality
- https://www.theatlantic.com/technology/archive/2014/05/qr-codes-for-the-dead/370901/
- https://www.fastcodesign.com/3024113/this-crazy-app-makes-a-magazine-an-interior-design-tool
- https://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6705607&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6705607
