CS 491 - Project 2 // Every Day is Halloween

Working on (Oculus) Go?

Oculus Go Headset

Oculus Go Homescreen

For decades now, the American work culture has been obsessed with being productive around the clock. As technology continues to evolve and solutions to problems become more and more efficient, I imagine this trend will continue to rise. Currently, this drive for being productive can be limited on occasions when we feel we have the time to get things done but not the capabilities. Take the everyday Chicago commuter as an example. According to the Bureau of Transportation Statistics, 9.1% of the commuter population in Illinois uses public transportation and until vehicles are fully autonomous, we will have to exclude the 81.9% who travel by car, truck or van (carpooled and alone), or other methods. Moovit, a fact and statistic page dedicated to commuters who use public transportation, says the average commute time of someone who uses public transportation in Chicago is 86 minutes with 87% of those riders spending more than two hours on public transportation every day. Through my own anecdotal evidence, most of these commuters appear to be mindlessly scrolling through various social media feeds, catching up on their favorite show or getting ahead on their work for the entire duration of their ride. An IT Specialist by the name of Malte Brodersen is one of users in the “getting ahead on their work” category. Having grown tired by being limited to one screen on his long, frequent commutes, he decided to experiment with augmented reality to boost his productivity. The purpose of Brodersen’s experiment is “see if a head-mounted display (HMD) can be implemented as a (useful) extension for professional needs.” At the most basic level, he wanted an application to project multiple computer screens he would be able to control and work with. Currently there are not many HMDs that have the computing power combined with being portable and quick to set up, so Brodersen settled with using Oculus Go for his software/application testing as soon as the hardware came out.

He starts by using Bigscreen, which is one of the most popular applications out right now. The app functions differently on Oculus Go versus across other devices such as the Vive and Rift. On all devices, you receive a full virtual reality experience and not the augmented reality he was hoping for. Also, during the time of the experiment (9/24/2018), Bigscreen was in its Beta for Go. The procedure in order to run the application on Go entails first having to launch the application on a PC, creating a ‘room’, then connecting to that room from Go using the room’s unique identifier. Brodersen quickly realizes that the idea for a full fledged application is there, but the current capabilities are still lacking. A user is unable to move the mouse or control the screen (e.g. resize or reposition it), just simply work on it. He writes the following about his general observations: “Hard” setup each time, strong PC and connection needed, bad performance, and the PC cannot be controlled. He discards Bigscreen in its current form but hopes to return to it as new features are implemented. Next, after spending some time enabling the developer mode and installing APKs on his Oculus Go, Brodersen was able to side load other applications not natively meant for virtual or augmented reality but that would still be able to run on the device. He notes that this preprocess has to only be done once before being able to side load apps. The first app he side loaded was called Riftcat. This app, also in its beta version during the time of testing, allows a user to turn a smart phone or in this case, an Oculus Go headset, into a HTC Vive Headset. After doing so, he loads up the Bigscreen application once again to see if there is any variation. The setup is the same as the last and after completing it, at a first glance, the application seems promising. The main difference this time was that Malte was able to use settings he was previously unable to like the screen sizing and positioning, but that ending up being more or less it. A user still needs a strong PC and reliable internet connection to be able to use the app, the “hard” setup has to be done as a preprocess every time and the quality and performance appear to have diminished considerably as well.

Bigscreen on Oculus Go

Bigscreen streamed with RiftCat

After having tested two applications that did not come close to satisfying his needs, Brodersen turned his view to a new commercial company, Microsoft. The last application Brodersen tested was Microsoft RDP, or Microsoft Remote Desktop Protocol. Brodersen had to side load the application similar to Riftcat, but the process was straightforward this time having done it as a preprocess once already. However when using Microsoft RDP, one more step was required: for any device (i.e. just not the Oculus Go), you need to create a host which is essentially the screen that will be shared. Personally, I have never done this but the author claimed it was straightforward. In the same fashion as the Riftcat application, the preprocess for Microsoft RDP has to be done only once. The first impression Brodersen had upon loading the application the first time was that the scale of the screen was off. Even after messing with all of the video settings, he was unable to fix the quality. Furthermore only one screen was possible to be worked on at a time even if the RDP was connected to a host that has multiple screens. On the plus side, the control felt natural and was easy to maneuver. Also, there was no need for a local computer. The advantages still, unfortunately, did not outweigh the disadvantages for the application to be deemed a success in Brodersen’s eyes.

Oculus TV with MS RDP

Generally, an app’s reviews are a solid indicator of its performance in typical use cases in the real world. Having tested the apps that are readily available and had relatively solid reviews, Brodersen had not found one that satisfied his original description: a readily available, easy to set up (both software and hardware) and genuinely virtual working environment. In the first scenario with Bigscreen standalone, the set up had to be done each time and it was quite time consuming. Furthermore, you need a strong PC and reliable internet connection local to where you will be working, the PC could not be controlled but only used to display the screen, and the performance was poor overall. Using Bigscreen with Riftcat, he found only improvements with the controlling and customizations of the screen, the other problems remained with the addition of reduced screen quality. Lastly was Microsoft’s RDP. This had been the closest application to his description, having no need for a local computer and control of the screen pretty doable. However, he was unable to change the settings for the screen and it had seemed “off” the entire time. Also, only one screen was possible, so the critical point of the experiment had been compromised, as he could easily just pulled out a laptop and gotten to work. So, the cons were still too prevalent to have the application considered a success and to be used regularly by many people. All of the applications considered in this experiment fell short on a few key points. First and foremost, if there is not an option to display multiple screens at once, the app should be thrown in the trash because a user could more easily pull out a laptop and work on that. Next, being able to reposition and resize various screens is difficult. I wonder what sort of protocols are involved with being able to resize the display of a screen in AR/VR compared to a regular PC. Lastly, being able to maintain a high frame rate is crucial. Like in any other game or video, consistent lag will turn a user off quicker than any other feature. Being able to maintain every key stroke and correspondingly update the text on a screen in real time would be a big upgrade. Overall, it seems that as there becomes more of an audience for these sorts of applications, other tech giants will become more and more involved with both the hardware and the software side of things, eventually creating the ideal app Brodersen has been dreaming of.

Regarding this app Brodersen is dreaming about, I imagine it would contain (but not limited to) the following features:

1.) View multiple screens at once,
2.) Connect to host computer easily,
3.) Control the screens while in AR/VR,
4.) High frame rate, and 5.) 480p minimum quality

All of these constraints assume that the hardware being used is also highly portable and lightweight, enabling users to wear the headset for the average commute time, which again is about 86 minutes. The first constraint seems rather obvious. If a user isn’t able to view multiple screens at once, it seems extremely likely that the user would choose using their laptop over the application every time. The ideal app would give the user an option to either fix the screens to a certain position so they can look away from the displays or have the screens track in front of your eyes at all times, allowing a user high accessibility without repositioning constantly. If we again imagine the scenario where the user is on a train, and the train is traversing hilly terrain, it would be annoying to constantly remap the screens. On the other hand, it might be nice to look away from the screen for a minute for whatever reason, so both options are reasonable. Next is being able to connect to the host computer easily. If it takes a considerable amount of time to setup the host computer and then connecting to it, it seems to me that the results will not be worth effort. Especially in this age, humans are creatures of instant satisfaction and waiting for things to load will quickly lose the interest of a typical user. In an ideal app, it would be nice to only do setting up when connecting to a new computer for the first time. From there, it would be in a list of “favorites” that the user could connect to with the click of a button assuming it is turned on and ready to pair. Having a sense of control of the screens while in the AR/VR would be a nice feature. Being able to reposition or rearrange screens along with resizing them would provide that sense of control. Lastly comes the quality and the frame rate of the screens while they are being used. If the text or images on any screen are illegible, it is unlikely that a user will continue to use the application. Surely if you are projecting a computer screen that is 12” across and you are trying to scale it up, it makes sense that the screen will be blurry. However, if a 1 to 1 mapping of a screen is blurry, that is where the issue lies. It is important though, that this choice is given to the user in order to give that sense of control. Another option could be to zoom into a certain portion of a screen rather than scale the whole thing up.

Resolution — Bigscreen

Resolution — RiftCat

Resulution — RDP

Ultimately, the ideal app is not on the market… yet. A couple of main things are still holding up this process. One of which is portability of an AR/VR headset that is powerful and inexpensive enough to get out to the general public. Another component is that setting up with a host computer without the intervention of a third party application is still unfeasible. These two constraints are keeping the market for a multi-screen remote desktop productivity app low and until they are resolved, that app will continue to not exist. As time goes by and technology continues to improve in both the hardware and software aspects, it is inevitable that the ideal app discussed throughout this essay will be mastered and released.

See the pdf of this text here.
See the article that inspired this text here.