Doc Stash
A Mixed Reality System that Augments Academic Reading on PC
Active reading activities, especially academic reading, is very tiring and time-consuming for researchers and students. The limitation of laptop display and the huge amount of information needed to read lead to problems like cognitive overload in navigation and inefficient multi-doc comparisons.
Thus, I designed mixed-reality workspaces based on Hololens to help users utilize the spatial environment for convenient document preview, searching, and management, while preserving the advantages of reading on the laptop.
TYPE |
Individual Project
DURATION |
2020.4 - 2020.6 (2 months)
Problem
Think about how many times you get lost in a huge number of academic papers during literature investigation, or jump from one tab to another in the browser to search for relevant information. Despite that digital reading on the personal computer facilitates academic reading with enabling functionality, e.g., the ability to search for additional content, organize digital documents, etc, visual feedback was limited by the dimension of screen display.
The display limits give rise to problems like navigation lost and high cognitive burden throughout the academic reading experience.
Also, while monitors to some extent extend the display of the personal computer, they are not portable.
OVERVIEW
Solution
I explored the idea of a mixed-reality system that enables users to leverage the 3D virtual workspaces to augment their academic reading experience on PC.
By wearing a Heads-up Display (I use Hololens here), users can make full use of different workspaces surrounding their personal computer to facilitate document-intensive tasks, e.g., searching, cross-referencing, and ad-hoc storage.
.png)
Process
RESEARCH
Literature Review
I looked through literature related to people's needs and behaviors in active reading. And I analyzed the limitations and strengths of reading digital information on the personal computer to find design opportunities. For example, it's flexible to pile or fan out physical documents, or put physical documents around for cross-referencing, but it can be very cumbersome to do so with digital documents.
User Research
I conducted observations and interviews on 6 master students and Phd students who had rich experience in academic reading. Through the user journey map, I dove deeper to understand their pain-points, needs, and design opportunities.
Pain Points
Through card-sorting, three key pain points in PC-based academic reading emerged.
Cognitive Overload in Navigation
The single-screen limitation leads to frequent window switching during academic reading. Researchers need to locate the page they just read and the page to be viewed, which makes it hard to maintain constant attention and adds cognitive burden.
Tedious Searching Experience
When searching for related work, researchers have to google each of the paper they are interested in the references part, scroll and open the document, and close it when the result is unwanted. Such searching tasks are repetitive and tedious.
Inefficient Multi-doc Comparison
During cross-referencing, researchers cannot refer to relevant documents quickly or compare multiple documents at the same time. In comparison, physical paper enables them to quickly review information in context and utilize the spatial layout to view multiple docs.
Insights
Also, I summarized three major insights I gained based on user pain points and literature on the strengths&weakness of digital reading on personal computers.
Extended Workspaces
Allow users to utilize separate extended workspaces for different tasks to facilitate information navigation.
Smart In-time Searching
Automatically provide related information that the user might need in the peripheral vision for easy and quick reference.
Spatial Layout for Multi-doc View
Support a more flexible spatial layout to help the user skim and compare between multiple documents for sensemaking.
IDEATION
Brainstorming
I brainstormed possible layouts of the extended virtual spaces.
DESIGN & DEVELOPMENT
I tried to balance between exploration and exploitation in the ideation phase. Thus, I would first draw the sketch, built minimum viable prototypes using Unity to test idea in Hololens, and then quickly iterated the idea.
1st Round: Multi-Doc View
In my first design, the user can view different documents by scrolling to quickly see the overview of different documents. After they select the document using a pointer in Hololens, they could scroll to view the documents in the virtual space and flexibly arrange the position/angle of the documents.
Quick Testing
I asked 4 Phd students to experience this demo and asked them how they would utilize the scroll view, whether it would help them with document location or skimming.
Finding 1: Improve flexibility of spatial layout to increase productivity.
The idea of a virtual display is good and might increase productivity, but the scroll view lacks flexibility in arranging the layout of docs, e.g., scaling, folding, repositioning, etc.
Finding 2: A broader display and more friendly interaction to increase navigation efficiency.
It's inconvenient to locate the desired document by scrolling. Also, the cover of the doc is too small and cannot provide enough information.
Finding 3: Cross-platform information transferring can increase productivity.
It would be helpful if the documents on the personal computer can be switched into the virtual docs for temporary storage and later reference. Also, it would be nice if the virtual docs could be transferred to the computer view for tasks like note-taking.
2nd Round: System Overview
In the second round, I designed a system of 4 workspaces, including the core workspace, the laptop screen, where users can focus on content operation tasks like note-taking.
This time, I designed the overview space into a grid view to create a broader display as well as to increase navigation efficiency. This space is meant for users to quickly open documents in their current workspace for reference.
Smart Search
Smart Search was designed to help readers quickly find related information of the document selected (either on the laptop or in the virtual space).
It would automatically search for associated academic information in the peripheral vision, including the associated video, contained figures, and related work. This would help reduce readers' cognitive burden in collecting related materials and sensemaking.
Whiteboard
Whiteboard is designed to archive information that might be useful for quick reference some time. Users can easily add any pieces of information to the whiteboard for ad-hoc storage e.g., docs/notes/videos, etc. For example, if a reference paper found using smart search is interesting, the user can archive it first to the whiteboard for future use.
Lo-Fi Prototype: System Overview + Smart Search
I created lo-fi prototypes using Mixed Reality Kit in Unity and tested it in Hololens.
Multi-doc Preview
Cross-platform Interaction: AR Drop
Sharing information between the laptop and the virtual space can increase efficiency and seamlessly connect all the workspaces together. Similar to Air Drop, I designed the feature, AR Drop, to enable users to easily share and receive documents, notes, videos, etc, between virtual and reality. If websites can also be shared, it would dramatically increase efficiency.
Lo-Fi Prototype: Virtual Screenshot
TESTING
User Testing
I ran a quick testing with 3 phd students using low-fi prototypes, and asked them to follow specific steps to experience the features of the system. Then I conducted semi-structured interviews on their opinions of this system.
Results
Still working in progress... Coming Soon!