For decades now, the American work culture has been obsessed with being productive around the clock.
As technology continues to evolve and solutions to problems become more and more efficient, I imagine
this trend will continue to rise. Currently, this drive for being productive can be limited on occasions
when we feel we have the time to get things done but not the capabilities. Take the everyday Chicago
commuter as an example. According to the Bureau of Transportation Statistics, 9.1% of the commuter
population in Illinois uses public transportation and until vehicles are fully autonomous, we will have to
exclude the 81.9% who travel by car, truck or van (carpooled and alone), or other methods. Moovit, a fact
and statistic page dedicated to commuters who use public transportation, says the average commute time of
someone who uses public transportation in Chicago is 86 minutes with 87% of those riders spending more than
two hours on public transportation every day. Through my own anecdotal evidence, most of these commuters
appear to be mindlessly scrolling through various social media feeds, catching up on their favorite show or
getting ahead on their work for the entire duration of their ride. An IT Specialist by the name of Malte
Brodersen is one of users in the “getting ahead on their work” category. Having grown tired by being limited
to one screen on his long, frequent commutes, he decided to experiment with augmented reality to boost his
productivity. The purpose of Brodersen’s experiment is “see if a head-mounted display (HMD) can be implemented
as a (useful) extension for professional needs.” At the most basic level, he wanted an application to project
multiple computer screens he would be able to control and work with. Currently there are not many HMDs that have
the computing power combined with being portable and quick to set up, so Brodersen settled with using Oculus Go
for his software/application testing as soon as the hardware came out.
He starts by using Bigscreen, which is one of the most popular applications out right now. The app functions
differently on Oculus Go versus across other devices such as the Vive and Rift. On all devices, you receive a
full virtual reality experience and not the augmented reality he was hoping for. Also, during the time of the
experiment (9/24/2018), Bigscreen was in its Beta for Go. The procedure in order to run the application on Go
entails first having to launch the application on a PC, creating a ‘room’, then connecting to that room from Go
using the room’s unique identifier. Brodersen quickly realizes that the idea for a full fledged application is
there, but the current capabilities are still lacking. A user is unable to move the mouse or control the screen
(e.g. resize or reposition it), just simply work on it. He writes the following about his general observations:
“Hard” setup each time, strong PC and connection needed, bad performance, and the PC cannot be controlled. He
discards Bigscreen in its current form but hopes to return to it as new features are implemented.
Next, after spending some time enabling the developer mode and installing APKs on his Oculus Go, Brodersen was
able to side load other applications not natively meant for virtual or augmented reality but that would still be
able to run on the device. He notes that this preprocess has to only be done once before being able to side load
apps. The first app he side loaded was called Riftcat. This app, also in its beta version during the time of
testing, allows a user to turn a smart phone or in this case, an Oculus Go headset, into a HTC Vive Headset.
After doing so, he loads up the Bigscreen application once again to see if there is any variation. The setup is
the same as the last and after completing it, at a first glance, the application seems promising. The main
difference this time was that Malte was able to use settings he was previously unable to like the screen sizing
and positioning, but that ending up being more or less it. A user still needs a strong PC and reliable internet
connection to be able to use the app, the “hard” setup has to be done as a preprocess every time and the quality
and performance appear to have diminished considerably as well.
Bigscreen on Oculus Go
Bigscreen streamed with RiftCat
After having tested two applications that did not come close to satisfying his needs, Brodersen turned his view
to a new commercial company, Microsoft. The last application Brodersen tested was Microsoft RDP, or Microsoft
Remote Desktop Protocol. Brodersen had to side load the application similar to Riftcat, but the process was
straightforward this time having done it as a preprocess once already. However when using Microsoft RDP, one
more step was required: for any device (i.e. just not the Oculus Go), you need to create a host which is
essentially the screen that will be shared. Personally, I have never done this but the author claimed it was
straightforward. In the same fashion as the Riftcat application, the preprocess for Microsoft RDP has to be done
only once. The first impression Brodersen had upon loading the application the first time was that the scale of
the screen was off. Even after messing with all of the video settings, he was unable to fix the quality.
Furthermore only one screen was possible to be worked on at a time even if the RDP was connected to a host that
has multiple screens. On the plus side, the control felt natural and was easy to maneuver. Also, there was no
need for a local computer. The advantages still, unfortunately, did not outweigh the disadvantages for the
application to be deemed a success in Brodersen’s eyes.
Oculus TV with MS RDP
Generally, an app’s reviews are a solid indicator of its performance in typical use cases in the real world.
Having tested the apps that are readily available and had relatively solid reviews, Brodersen had not found one
that satisfied his original description: a readily available, easy to set up (both software and hardware) and
genuinely virtual working environment. In the first scenario with Bigscreen standalone, the set up had to be
done each time and it was quite time consuming. Furthermore, you need a strong PC and reliable internet connection
local to where you will be working, the PC could not be controlled but only used to display the screen, and the
performance was poor overall. Using Bigscreen with Riftcat, he found only improvements with the controlling and
customizations of the screen, the other problems remained with the addition of reduced screen quality. Lastly was
Microsoft’s RDP. This had been the closest application to his description, having no need for a local computer and
control of the screen pretty doable. However, he was unable to change the settings for the screen and it had seemed
“off” the entire time. Also, only one screen was possible, so the critical point of the experiment had been
compromised, as he could easily just pulled out a laptop and gotten to work. So, the cons were still too prevalent
to have the application considered a success and to be used regularly by many people. All of the applications
considered in this experiment fell short on a few key points. First and foremost, if there is not an option to
display multiple screens at once, the app should be thrown in the trash because a user could more easily pull out
a laptop and work on that. Next, being able to reposition and resize various screens is difficult. I wonder what
sort of protocols are involved with being able to resize the display of a screen in AR/VR compared to a regular PC.
Lastly, being able to maintain a high frame rate is crucial. Like in any other game or video, consistent lag will
turn a user off quicker than any other feature. Being able to maintain every key stroke and correspondingly update
the text on a screen in real time would be a big upgrade. Overall, it seems that as there becomes more of an
audience for these sorts of applications, other tech giants will become more and more involved with both the hardware
and the software side of things, eventually creating the ideal app Brodersen has been dreaming of.
Regarding this app Brodersen is dreaming about, I imagine it would contain (but not limited to) the following features:
1.) View multiple screens at once,
2.) Connect to host computer easily,
3.) Control the screens while in AR/VR,
4.) High frame rate, and
5.) 480p minimum quality
All of these constraints assume that the hardware being used is also highly portable and lightweight, enabling users to
wear the headset for the average commute time, which again is about 86 minutes. The first constraint seems rather obvious.
If a user isn’t able to view multiple screens at once, it seems extremely likely that the user would choose using their
laptop over the application every time. The ideal app would give the user an option to either fix the screens to a certain
position so they can look away from the displays or have the screens track in front of your eyes at all times, allowing a
user high accessibility without repositioning constantly. If we again imagine the scenario where the user is on a train,
and the train is traversing hilly terrain, it would be annoying to constantly remap the screens. On the other hand, it might
be nice to look away from the screen for a minute for whatever reason, so both options are reasonable. Next is being able to
connect to the host computer easily. If it takes a considerable amount of time to setup the host computer and then connecting
to it, it seems to me that the results will not be worth effort. Especially in this age, humans are creatures of instant
satisfaction and waiting for things to load will quickly lose the interest of a typical user. In an ideal app, it would be
nice to only do setting up when connecting to a new computer for the first time. From there, it would be in a list of “favorites”
that the user could connect to with the click of a button assuming it is turned on and ready to pair. Having a sense of control
of the screens while in the AR/VR would be a nice feature. Being able to reposition or rearrange screens along with resizing
them would provide that sense of control. Lastly comes the quality and the frame rate of the screens while they are being used.
If the text or images on any screen are illegible, it is unlikely that a user will continue to use the application. Surely if
you are projecting a computer screen that is 12” across and you are trying to scale it up, it makes sense that the screen will
be blurry. However, if a 1 to 1 mapping of a screen is blurry, that is where the issue lies. It is important though, that this
choice is given to the user in order to give that sense of control. Another option could be to zoom into a certain portion of a
screen rather than scale the whole thing up.
Resolution — Bigscreen
Resolution — RiftCat
Resulution — RDP
Ultimately, the ideal app is not on the market… yet. A couple of main things are still holding up this process. One of which is
portability of an AR/VR headset that is powerful and inexpensive enough to get out to the general public. Another component is
that setting up with a host computer without the intervention of a third party application is still unfeasible. These two
constraints are keeping the market for a multi-screen remote desktop productivity app low and until they are resolved, that app
will continue to not exist. As time goes by and technology continues to improve in both the hardware and software aspects, it
is inevitable that the ideal app discussed throughout this essay will be mastered and released.
See the pdf of this text here.
See the article that inspired this text here.