Augmented Reality | From the Archives, March 2015

From the Archives is a series that pulls from a collection of research papers written for a high school STEM program between the years 2013-2015. These papers pushed students to investigate a subject, formulate questions and further developments upon the matter, and ultimately expand knowledge through some form of experimentation and execution. While much of the information and personal opinions presented have changed since the time of writing, I wanted to publicly document my thinking and explorations from that time.


Background

All of the technologies that have been introduced and utilized by a mass market of consumers fulfill different purposes and are meant to be interacted with in widely different manners. Despite these disparities, all of these products are similar in that they introduce a new form of consumption, creativity, and level of interactivity. Take for example the television: since its inception, the TV has primarily been a device meant for content consumption. Users turn on the device, and can choose between various channels, all of which enable viewers to see different types of content. TVs are not solely consumption devices, as someone must create the news report or show that is watched, but the amount of creators in the television business is only as high as the number of channels and time slots available - according to the World FactBook, there are only four major television broadcast networks, all of which have a total of roughly 1000 channels [24]. By the same token, a TV is meant to be a family experience, a source that groups of people can gather around and enjoy together. While this inclusive nature is a positive aspect that contrasts the hinderance on creating content, it also limits the amount of privacy given to a viewer. There is no feasible way for a person to watch an R-rated film on the TV while in a room with children, as the content on such a large device can be seen by everyone.

The introduction of the smartphone was accompanied by an increase in consumption, creation, and privacy levels. Because smartphones are highly portable, people can access and view news or videos wherever they are, whenever they want to. A smartphone’s ability to run games, show new stories, and even let users shop for clothes, among other actions, gave developers the opportunity to create more content for consumption; as of June 2014, Apple’s App Store had 1.2 million applications available for download, all of which are in some way unique yet still can fulfill the same goal, whether it is being more productive or being better at racing on a phone [19]. Each of those 1.2 million apps has a team of hundreds of creative developers behind, who are still creating and refining content for smartphones, clearly demonstrating the leap in content creation and prospective consumption incurred by the smartphone era. Content has become more private, too, with these devices: the individual above that wanted to watch an R-rated movie in a room full of kids now can do so safely with a smartphone. Simply open up the movie on the phone’s movies app, plug headphones into the standard 3.5 millimeter headphone jack, and enjoy. The displays are now large enough for the movie to be adeptly experienced, and can also be held close a person’s face so only they are viewing the content. 

The Next Generation

Eight years after smartphones were introduced to the mass market, a new form of technology will be released that will attempt to not only balance the creativity and consumption of a user, but also alter the very manner a person interacts with the content: Microsoft HoloLens [16]. To understand the purpose of HoloLens and how it can change lifestyles, interaction, creativity, consumption, and privacy must all be analyzed. At first glance, this device may seem to be a simple transparent lens with a heads-up display; however, HoloLens provides and is interacted with through three-dimensional holograms [17]. These holograms are manipulated with by a user’s hands. Mounted on the headset is a depth camera that “has a field of vision that spans 120 by 120 degrees…so it can sense what your hands are doing even when they are nearly outstretched” [8]. Such a process may sound like a Kinect on steroids; both the Kinect and HoloLens were manufactured by Microsoft, so it is only natural that both devices share some technologies and operational methods. Other sensors provide the computer within with terabytes of data every second, all of which is processed very quickly to properly render digital holograms in the real environment a user is in. The data needed to process and create these holograms is interesting, but an even larger question is how we are able to see these holograms. The answer lies in simply tricking the eyes and brain. Within the device’s “light engine” are thousands of light particles, all of which bounce around before entering the lenses and reflecting into the user’s eye: this process must be done so precisely so that the light enters the eye at the proper angle to create a holographic illusion [11]. Interacting with HoloLens is a physical experience performed with hand motions, and HoloLens itself is an intensely private and singular event because the images are projected into a user’s eyes, meaning that no one else can see what the user sees. 

Now that interaction and privacy have been deciphered, how does HoloLens revolutionize the way content is created and consumed? The consumption aspect is easy enough to figure out: holograms allow people to see new objects that they may have only come into contact with via Google Images in a 3D setting. For example, a person may be interested in purchasing a new car, and is mainly interested in the aesthetics of various models. Rather than taking the time to drive to a dealer just to see a car, HoloLens can create a holographic representation of the vehicle’s exterior and present it to the user. From here, the user can decide whether to go to the dealer and test drive the car or to forgo the vehicle entirely. But what of content creation, and its limits, or lack thereof?

Conceiving and building new content and ideas must have been a key part of HoloLens’ abilities since its creation, as Microsoft has created an application that allows users to bring their ideas into the real world. This application is called Holo Studio, and according to The Verge’s Dieter Bohn, Holo Studio could potentially “unleash a wave of creators who would be able to dream up 3D objects with little to no training” [2]. When Holo Studio is initiated, creators are greeted by a holographic toolbox, hovering at eye-level and moving to accommodate posture or hand interactions [20]. This toolbox contains anything a creator would require: different shapes of wooden blocks, spray paint, and other nuts and bolts [22]. The most integral feature, though, of the toolbox and Holo Studio is the support for voice commands - when a person’s hands are figuratively full, they can use their voice to copy and paste objects and expand the toolbox. When a person is done constructing, they can choose to send their design to a 3D Printer for a size-based fee [7]. 

Personal Connection

Because it is a prospective product, there is not a definitive answer as to how HoloLens as a whole could apply to my life. However, based on what I know about the product currently and Holo Studio specifically, there is an application of this technology that would prove very useful in my life. Industrial design is a field that has interested me for many years now, and involves the conceptual visualization of new technology. Traditionally, such designs are done either in a 2D or 3D format using programs on a computer; however, these programs present two faults, both of which are addressed in an adequate manner by Holo Studio. The first issue is that while modeling is undoubtedly possible on existing softwares, such as Keynote and Blender, it is a very inefficient process. 2D modeling, as on Keynote, is very simplistic and only works well on very basic representations of an object from only a single view. When moving on to Blender or other 3D alternatives, the software is cluttered by complex functions and difficult controls, making it hard to successfully create anything in a proper amount of time. Holo Lens seems to solve this issue by making 3D designing akin to building with your hands, thus eliminating the complexity associated with manipulating a trackpad or understanding a multitude of keystrokes. Furthermore, Holo Studio is based around a toolbox that provides any necessary resources for building as opposed to the bombardment of functions presented by orthodox modeling programs. Purchasing a HoloLens and using Holo Studio to do my industrial design concepts rather than Keynote would allow me to move on from very basic designs that demonstrate little range and skill and instead express my ideas in a fully realized manner.

Further Development

The second problem that would be solved by Holo Studio is the fine tuning enabled to these designs. When designing a device, there are many elements to consider: form factor, size, and shape are among these factors. Ultimately, though, designing a device becomes an emotional experience, and the relationship a person has with their device is one of emotion, too. Channeling such emotion helps create the best product possible, as the investment one has in the aesthetics of a product can spur better decision-making. However, traditional modeling softwares do not enable a designer to achieve a proper emotional connection with their concept for one sole reason: they cannot interact with it, and thereby only regard the concept as a piece of paper or a picture on a cold machine rather than an object that should be played with and used properly. As mentioned before, the building process of Holo Lens is a hands-on one, allowing the creator to not only build the concept device, but also hold the device. Is it too large? Can be bezels be shortened in any way? Can the edges of the screen be comfortably reached? These are important design questions that can now be answered using Holo Studio because it allows designers to interact with their creations on a new level and make that emotional connection to their product. On top of this, designers can 3D print their creations directly from the Holo Studio software, allowing them to share their models with theory people on a design team and receive feedback.

A Broader Audience

While the above examples apply mainly to me, Holo Studio’s benefits can expand to my peers, specifically sophomore L2K students. An upcoming project that requires all students to design and build products is the R&D project at the end of the year. Many students may opt to use materials or even exteriors that already exist in order to construct their own creation. However, other groups, including mine, may want to build their product from scratch and only borrow minimal materials. To do this, a 3D modeling software is needed. While groups may opt to use Blender or Google SketchUp, this decision may only be made out of desperation as there is no other alternative. Even after designing the product, it must be 3D printed before it can be constructed and presented at the end of the assignment. After a while, such a process can become expensive and time consuming, especially if the first design is not adequate. With Holo Studio, students can quickly create their designs without worrying about the complexity of Blender or SketchUp, and also make it the best product possible by interacting with the hologram and further refining it. After doing this, the design must be 3D printed, but this process could be made easier if Microsoft release a 3D printing center where Holo Lens users can send their designs and have them printed for a small fee. This would be less expensive than purchasing a new 3D printer, and also much easier to use since trained professionals would know how to operate and maintain the printers. 

Smoke and Mirrors

The term “hologram” has been used heavily in association with the way HoloLens operates and presents content. Despite this terminology, many have pointed out that HoloLens is more akin to a virtual reality headset rather than a true hologram, which has been traditionally depicted as a 3D object that can be viewed and interacted with sans headset. In order to understand this distinction, I decided to research the technology behind true holograms and compare it to the way HoloLens displays 3D images. 

Simply put, a hologram is a photographic technique that can produce 3D images. To begin this technique, a laser is used to create light interference patterns. A laser is imperative to this process because it is coherent, meaning that “all of the light waves emitted are at exactly the same wavelength” [3]. This laser beam is then split into two separate beams using a mirror and enlarged using a lens. The first beam, or object beam, is pointed at the object that will become a hologram, while the second, reference, beam is directed towards the area where the hologram will be displayed [25]. The hologram is created at the point where the two beams intersect. 

In contrast, and as mentioned previously, the “holograms” of HoloLens are created by light particles moving rapidly and then reflecting into the viewers eyes. Clearly, this is not a traditional hologram in any sense. However, after learning about the technology needed to produce a hologram and how holograms are displayed, I have concluded that the HoloLens is superior in privacy and efficiency. As far a privacy goes, a traditional hologram can be seen by anyone in the room the hologram is projected in, while a HoloLens image is tailored specifically towards the viewer. Thus, HoloLens users that are designing a new top-secret product need not worry about prying eyes observing these designs, as only the creator is able to see these images. Furthermore, HoloLens is not bound by the same constraints as a hologram, the main constraint being that the lasers in a hologram must first scan an object before projecting it. This makes the assumption that an object must already tangibly exist before being turned into a hologram. On the opposite side, HoloLens opens a new world to the user by allowing any object, real or not, to be created from scratch and cast into an environment. This also takes up less space in a room, as space doesn’t need to be designated for a laser emitter, mirrors, the object, and then a landing pad for the hologram. 

Extension

To gain a more visual understanding of the above techniques, I decided to create graphical representations of the creation of a hologram. By doing so, I not only understood the complexity associated with the first method, but also fully realized how much room was required to properly create a hologram as compared to the compact nature of HoloLens’ projection methodAs part of this application, I also wanted to create a visual of the HoloLens’ light particles bouncing and entering a user’s eyes, but was unable to accurately depict this because the exact manner of this process has not been described officially by Microsoft, and therefore creating a model would present false information.

HoloLens is a very forward thinking product that has the potential to alter the way that people see and interact with the world around them. However, even though all of this new technology is cool and has said potential, its success is not at all guaranteed; until the product is released, its level of success cannot be known. But the success of other similar products can provide a reasonable projection of how the public will respond when HoloLens is released.

A Different Story

One such product is Google Glass. Glass was developed for many years by Google’s Project X division before being officially unveiled in April 2012 [23]. At that date, not much was known about Google Glass: all that was clear was that the device “would be wirelessly connected to the Internet, powered by Android and enable the user to interact using voice commands and eye gestures by tracking eye movements.” Another feature similar to HoloLens is that the attached camera could provide an augmented data overlay in an environment. In June of 2012, developers were allowed to pre-order Google Glass for $1,500 and begin testing the product [12]. This date marked the first time that Glass was available for people outside of Google’s offices to use; unfortunately, it would be a long time before anyone outside of the developer community was able to interact with it. Despite this slow expansion time, Glass was seemingly slated for success, with TIME Magazine declaring it the greatest invention of 2012 [21]. 

In February 2013, Google asked the general public via Twitter or Google+ what they would do if they had Google Glass: contest winners would have the opportunity to order a Glass unit, albeit still with the $1,500 fee [9]. Contest winners were notified the following month whether they were eligible to order, and received their orders three month after that. It may seem that Glass’ introduction into the world was going smoothly, but doubts were raised because the general public and even some developers were only receiving their units more than a year after the product’s announcement. Later in 2013, Google attempted to gain more positive publicity by featuring Glass on the fashion magazine Vogue, but other factors, including some device malfunctioning generated negative publicity as well [13].

Ultimately, Google Glass did not take off into mass markets as expected in 2014 [15]. Software and hardware were still clunky, and privacy and safety concerns were raised by the presence of a camera that could take pictures without people being aware. On January 15, 2015, Google announced that it would end explorer sales of its current version of Glass [5]. If the story of Glass were to end here and used as an analogous situation, it would spell failure for HoloLens. However, the story continues and may indicate that if HoloLens does get off to a rocky start, it can still succeed given the right circumstances. The Glass project isn’t dead, Google has announced that it has simply graduated from being a Project X item to being its own official part of Google. Furthermore, iPod and Nest creator/designer Tony Fadell has taken over as head the Glass project, and may give the product the appeal needed for consumers to accept it [18]. Based on this, it can be predicted that if HoloLens is unable to make a good first impression with consumers, Microsoft should try to rework the design with a proven talent rather than scrap the project entirely.

With the emergence of smartwatches and virtual reality headsets, and now HoloLens, technology is no longer limited to a device that is used occasionally and then stowed in a pocket. Instead, technology is now becoming as much a fashion product and accessory as it is an electronic marvel. Companies are only now going through such epiphanies, leading me to wonder: how many technology companies are seeking advice from fashion gurus and test audiences while developing these wearable products, and is Microsoft doing the same with HoloLens? 

To answer the first part of the above question, it must first be understood what companies are currently developing wearable devices, such as smartwatches. Unsurprisingly, many companies are racing to get their product onto consumers’ wrists, including Apple, Motorola, Samsung, and Sony [14]. However, the surprising aspect of that statement is that only one of those manufacturers has consulted with an established fashion company and shown the device during development to someone outside of the in-house software and hardware team: Apple. In October 2013, a year before the Apple Watch was announced, it was reported that Apple had hired Angela Ahrendts, former chief of British luxury fashion icon Burberry [4]. At the time, Ahrendts was mainly tasked with redesigning Apple’s retail stores to create a more unified, simplistic, and user-friendly shopping experience; however, newer rumors have cited Ahrendts as also working on the design of Apple Watch to not only being one of the few smartwatches to cater to females and be viewed foremost as an article of clothing, not an electronic [6]. In the same vein, Ahrendts is not the only name in fashion that Apple has hired. Paul Deneve, CEO of Yves Saint Laurent, also joined Apple in 2013 as “vice president of special projects” and reports directly to Tim Cook [1]. YSL is a company that caters mostly to women but also has products for men, including suits, colognes, and handbags/purses. By joining Apple, Deneve is able to bring his sense of quality when choosing materials to be used in something very intricate like Apple Watch bands, and even the specific tone of stainless steel and gold used in Apple Watch’s body. 

Clearly, Apple is taking an extensive amount of feedback and design input from renowned fashion figures in their quest to make an impactful fashion and technology product. Is Microsoft doing the same, or are they following in the steps of Google and Google Glass pre-Fadell era? If HoloLens were to be judged based on aesthetics alone, it can be reasonably assumed that the device has seen very little design input from fashion workers: as seen in Figure 1.5, the product looks very thick, heavy, and uncomfortable - very reminiscent of badly designed ski masks. Any amount of research does little to support or refute this claim, as Microsoft is remaining very tight-lipped about this product and revealing no details about the manner in which it was designed. Only time will tell whether people perceive HoloLens to be a fashion accessory or not, and if it is a worthy one at that. However, one thing can be learned from the success of the iPod or iPhone over other precursors, or even the possible success of Google Glass now that Tony Fadell is heading the project: no matter grim the prospects of current products are in a certain category, there is always an hope for rejuvenation if those with an eye for current style and fashion lead the initiative.


References

[1] "Apple - Press Info - Apple Leadership - Paul Deneve." Apple - Press Info - Apple Leadership - Paul Deneve. Apple, Inc., n.d. Web. 21 Feb. 2015.

[2] Bohn, Dieter. "We Just Tried HoloLens, Microsoft's Most Intriguing Product in Years." The Verge. Vox Media, 21 Jan. 2015. Web. 20 Feb. 2015.

[3] Campbell, Todd. "Answer Geek: How Holograms Work." ABC News. ABC News Network, n.d. Web. 21 Feb. 2015.

[4] Chen, Brian X., and Mark Scott. "Apple Hires Burberry Chief to Polish Image of Online Stores." The New York Times. The New York Times, 15 Oct. 2013. Web. 21 Feb. 2015.

[5] "Google Calls End to Glass Experiment." BBC News. BBC, 15 Jan. 2015. Web. 19 Feb. 2015.

[6] Gurman, Mark. "Apple Retail SVP Angela Ahrendts: Apple Watch Launching in "Spring," after Chinese New Year." 9to5Mac. WordPress, 02 Nov. 2014. Web. 21 Feb. 2015.

[7] Hackman, Mark. "Hands-on with Microsoft's HoloLens: The 3D Augmented Reality Future Is Now." PCWorld. IDG Consumer & SMB, 22 Jan. 2015. Web. 21 Feb. 2015.

[8] Hempel, Jessi. "Project HoloLens: Our Exclusive Hands-On With Microsoft's Holographic Goggles | WIRED." Wired.com. Conde Nast Digital, 21 Jan. 2015. Web. 22 Feb. 2015.

[9] "The History of Google Glass." Glass Almanac. Glass Almanac, n.d. Web. 19 Feb. 2015.

[10] HoloLens Model. Digital image. Forbes. Forbes Magazine, n.d. Web. 21 Feb. 2015.

[11] Kovach, Steve. "Microsoft's New HoloLens Headset Is Very Similar To A Secret Product Google Has Invested In." Business Insider. Business Insider, Inc, 21 Jan. 2015. Web. 20 Feb. 2015.

[12] Mack, Eric. "Brin: Google Glass Lands for Consumers in 2014 - CNET." CNET. CBS Interactive Inc., 28 June 2012. Web. 21 Feb. 2015.

[13] Madrigal, Alexis C. "Huh, Vogue Publishes 12-Page Google Glass Spread in September Issue." The Atlantic. Atlantic Media Company, 16 Aug. 2013. Web. 21 Feb. 2015.

[14] Mamiit, Aaron. "Best Upcoming Smartwatches in 2015: Apple Watch, Moto 360 2, LG G Watch R2, Kairos MSW, Withings Activité and More." Tech Times RSS. TechTimes.com, 24 Jan. 2015. Web. 21 Feb. 2015.

[15] McGee, Matt. "Sergey Brin "Not Sure" If Google Glass Launch Will Happen This Year - Glass Almanac." Glass Almanac. Glass Almanac, 29 May 2014. Web. 21 Feb. 2015.

[16] Microsoft HoloLens - Transform Your World with Holograms. YouTube. Google, Inc., 21 Jan. 2015. Web. 20 Feb. 2015.

[17] "Microsoft HoloLens." Microsoft HoloLens. Microsoft, 21 Jan. 2015. Web. 20 Feb. 2015.

[18] Patel, Nilay. "Google Puts Nest's Tony Fadell in Charge of Google Glass." The Verge. Vox Media, 15 Jan. 2015. Web. 21 Feb. 2015.

[19] Perez, Sarah. "ITunes App Store Now Has 1.2 Million Apps, Has Seen 75 Billion Downloads To Date." TechCrunch. AOL Inc., 2 June 2014. Web. 19 Feb. 2015.

[20] Signum News - Microsoft Holo Studio Demo. YouTube. Google, Inc., 21 Jan. 2015. Web. 21 Feb. 2015.

[21] Staff, TIME. "Google Glass." Time. Time, 31 Oct. 2012. Web. 21 Feb. 2015.

[22] Stein, Scott. "Microsoft HoloLens: Not Holograms, Exactly, but Strike One in AR Turf War - CNET." CNET. CBS Interactive Inc., 21 Jan. 2015. Web. 20 Feb. 2015.

[23] Team, Trefis. "Google Glasses Sound As Crazy As Smartphones And Tablets Once Did." Forbes. Forbes Magazine, 5 Apr. 2012. Web. 21 Feb. 2015.

[24] "United States: Communications." Central Intelligence Agency World Factbook. Central Intelligence Agency, n.d. Web. 19 Feb. 2015.

[20] Workman, Robert. "What Is a Hologram?" LiveScience. TechMedia Network, 23 May 2013. Web. 21 Feb. 2015.