SEU Interactive Museum
Extending reality into virtual
Brief Introduction
The exhibition hall of Southeast University presents models of masterpieces or students works. However, due to limited space and improper way of preservating physical exhibits, the museum is becoming increasingly unpopular.
So we decided to design an interactive digital museum for SEU, in order to use more modern methods to preserve exhibits, at the same time bring more interesting experience to visitors.
Generative Research
We did research on the current usage of our site through our observation and survey.
The exhibition hall is located in an important area of my department. Its functions include displaying exhibits, and holding special lectures. But at that moment, the utilization rate of the exhibition hall is relatively low. Among all the activities, multimedia plays and lectures are most attractive. Students and teachers hope that the museum will have a more modern style and connect with our everyday life. They also hope they can easily view popular and newly added works.
What are people talking about...
" When I am alone, I seldom stay at this area unless there are interesting multimedia plays.
"
" Space is limited. The hall is relatively small to hold more and more student work. We hope to archive all the excellent works.
"
" Sometimes I just want to see the most popular exhibits or newly-added ones instead of go through all the works.
"
" The exhibits are good but we cannot pick it up for a look from another angle because they are so fragile.
"
Wrap up
Wrap up
I summarized the pain points and opportunities.
Pain points
Opportunities
Pain points
Opportunities
Pain points
Opportunities
Limited space &
limited exhibits
There's no proper way to display and preserve all the exhibits.
Cannot touch the models so students cannot learn the works better.
Digital exhibits can prevent the problem. But we should make them look 'real'
Make digital exhibits able to be picked up, scaled, rotated...
Make the digital space looks like an 'extension' of real world. So people may feel the room larger.
I summarized the pain points and opportunities.
Ideation
Make the digital space looks like an 'extension' of real world. So people may feel the room larger.
Concept Brainstorming
With the support of the Interactive Architucture Lab, our team brainstormed 10+ solutions and synthesized them into four basic types
- VR, AR+QR code, Somatic, Holographic Projection. All thiese four solutions can cover the three big pain points listed above. For the reason that we have to bring the design to life in this project, we decided to select the most 'efficient-to-use' and 'easy-to-realize' solution with a 2*2 metric. For the reason that the concept of AR is similar to another team in the Lab, so we took Somatic as our solution.
VR QR+AR Somatic Holographic Projection
Easy-to-use
Easy-to-realize
VR
Holo Projection
AR
Somatic
I summarized the pain points and opportunities.
We decided to apply somatic interactive technology to our new virtual museum. The advanced ideas are:
1.The screen is an extension of the real world which visually expands the space. We enhance the feeling of
realism by changing viewpoint through head-tracking.
2.Breaks the traditional mode of human moving and becomes objects moving. Objects are transmitted after user's motion is detected.
Scenario Sketching
Design
Potential Features
We digged into the pain points and scenario above and thought of potential features
Extend real world
Unify interior design in real space and
digital space
Space of infinite depth
Holographic
Perspective
Way of 'preserve' exhibits
Show lists in categorized menu like a library
Make each window a 'PORTAL'
Put exhibits on 'conveyer track'.
Interact with exhibit
Scale, Drag, Rotate with gesture.
'Enter' the model into real perspective
Structural decomposition
Check detailed blueprint of exhibits
Designer Introduction
Like and comment
Other user requirements
Filter by Category
See new exhibits
Filter by likes or comments
Event / News Billboard
Privacy. Login with student id
Unity is a cross platform 2D / 3D game engine developed by unity technologies
User Flow
I streamlined the storyline I got from user interview and ideation, and created our user flow.
Interior Design & Interface Design
We create three set of Interior + Interface design of different style and asked our users to choose.
Style A: Realism
Develop
Interfaces
Welcome
Browsing
Interaction
Like
Style B: Future
Interfaces
Interaction
Like
Welcome
Browsing
Style C: Abstract
Interfaces
Browsing
Welcome
Interaction
Like
All three styles are accepted and appreciated by students and staff, while we came into a problem that our current engine could not perform well in real-time rendering if we choose the future style and the full-of-refraction-calculation abstract style. At last, we decided to realize style A and build it in real site.
Technical Solution
Unity is a cross platform 2D / 3D game engine developed by unity technologies.
Kinect uses the human skeleton tracking algorithm based on the depth image, which can directly recognize and capture the player's movements, and record the three position information of the player's joints at the same time.
Data
Reflected light
on objects
Scenario
Coding
Lattice light map
for measurement
We create three set of Interior + Interface design of different style and asked our users to choose.
Realization
Due to the time limit, we realized the scene of style A with Unity3D and Kinect SDK (c#).
The key points of the somatosensory interaction can be summarized as follow : joint recognition, head tracking, gesture recognition (scale/move/rotate), item switching, scene switching, data collection.
Joint & Head Tracking
Gesture Recognition
Item Switching & Scene Switching
Demo Video
(video created by Ningyue)
MY ROLE
-
Performed research to gather pain points and user requirements of the museum via user interviews.
-
Modeled our concept into architecture and designed interfaces for screens.
-
Realized the head tracking and gesture recognition of our system.
TOOLS / TEAM
-
Adobe Creative Suite, Sketch, Unity3D, Kinect
-
Feb. 2019 - May. 2019
-
Ningyue Shang, Bo Ye, Yuetian Sun
