top of page

SEU Interactive Museum

Extending reality into virtual

Brief Introduction

The exhibition hall of Southeast University presents models of masterpieces or students works. However, due to limited space and improper way of preservating physical exhibits, the museum is becoming increasingly unpopular.

So we decided to design an interactive digital museum for SEU, in order to use more modern methods to preserve exhibits, at the same time bring more interesting experience to visitors.


Generative Research

We did research on the current usage of our site through our observation and survey.

The exhibition hall is located in an important area of my department. Its functions include displaying exhibits, and holding special lectures. But at that moment, the utilization rate of the exhibition hall is relatively low. Among all the activities, multimedia plays and lectures are most attractive. Students and teachers hope that the museum will have a more modern style and connect with our everyday life. They also hope they can easily view popular and newly added works.


What are people talking about...

资源 4.png

" When I am alone, I seldom stay at this area unless there are interesting multimedia plays. 


资源 3.png

" Space is limited. The hall is relatively small to hold more and more student work. We hope to archive all the excellent works. 


" Sometimes I just want to see the most popular exhibits or newly-added ones instead of go  through all the works.  


资源 5.png
资源 1.png
资源 2.png

" The exhibits are good but we cannot pick it up for a look from another angle because they are so fragile. 


Wrap up

Wrap up

I summarized the pain points and opportunities.

Pain points


Pain points


Pain points


Limited space &

limited exhibits

There's no proper way to display and preserve all the exhibits.

Cannot touch the models so students cannot learn the works better.

Digital exhibits can prevent the problem. But we should make them look 'real'

Make digital exhibits able to be picked up, scaled, rotated...

Make the digital space looks like an 'extension' of real world. So people may feel the room larger.

I summarized the pain points and opportunities.


Make the digital space looks like an 'extension' of real world. So people may feel the room larger.

Concept Brainstorming

With the support of the Interactive Architucture Lab, our team brainstormed 10+ solutions and synthesized them into four basic types

- VR, AR+QR code, Somatic, Holographic Projection. All thiese four solutions can cover the three big pain points listed above. For the reason that we have to bring the design to life in this project, we decided to select the most 'efficient-to-use' and 'easy-to-realize' solution with a 2*2 metric. For the reason that the concept of AR is similar to another team in the Lab, so we took Somatic as our solution.

资源 7.png
资源 10.png
资源 11.png

VR                    QR+AR                Somatic            Holographic Projection




Holo Projection



I summarized the pain points and opportunities.

We decided to apply somatic interactive technology to our new virtual museum. The advanced ideas are:

1.The screen is an extension of the real world which visually expands the space. We enhance the feeling of

 realism by changing viewpoint through head-tracking.


2.Breaks the traditional mode of human moving and becomes objects moving. Objects are transmitted after user's motion is detected.


Scenario Sketching


Potential Features

We digged into the pain points and scenario above and thought of potential features

Extend real world

Unify interior design in real space and

digital space 

Space of infinite depth



Way of 'preserve' exhibits

Show lists in categorized menu like a library

Make each window a 'PORTAL'

Put exhibits on 'conveyer track'.

Interact with exhibit

Scale, Drag, Rotate with gesture.

'Enter' the model into real perspective

Structural decomposition

Check detailed blueprint of exhibits

Designer Introduction

Like and comment

Other user requirements

Filter by Category

See new exhibits

Filter by likes or comments

Event / News Billboard

Privacy. Login with student id

Unity is a cross platform 2D / 3D game engine developed by unity technologies

User Flow

I streamlined the storyline I got from user interview and ideation, and created our user flow.

Interior Design & Interface Design

We create three set of Interior + Interface design of different style and asked our users to choose.

Style A: Realism




å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å??





Style B: Future



å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å??





Style C: Abstract

å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å??






All three styles are accepted and appreciated by students and staff, while we came into a problem that our current engine could not perform well in real-time rendering if we choose the future style and the full-of-refraction-calculation abstract style. At last, we decided to realize style A and build it in real site.

Technical Solution


Unity is a cross platform 2D / 3D game engine developed by unity technologies.

Kinect uses the human skeleton tracking algorithm based on the depth image, which can directly recognize and capture the player's movements, and record the three position information of the player's joints at the same time.



Reflected light

on objects



Lattice light map

for measurement

We create three set of Interior + Interface design of different style and asked our users to choose.


Due to the time limit, we realized the scene of style A with Unity3D and Kinect SDK (c#).

The key points of the somatosensory interaction can be summarized as follow : joint recognition, head tracking, gesture recognition (scale/move/rotate), item switching, scene switching, data collection.

Joint & Head Tracking
2å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?
3å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?
Gesture Recognition
5å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?
6å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?
Item Switching & Scene Switching
7å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?
8å±??å¹??å¿«ç??§ 2019-09-19 ä¸??å?

Demo Video

​(video created by Ningyue)


  • Performed research to gather pain points and user requirements of the museum via user interviews.

  • Modeled our concept into architecture and designed interfaces for screens.

  • Realized the head tracking and gesture recognition of our system.


  • Adobe Creative Suite, Sketch, Unity3D, Kinect

  • Feb. 2019 - May. 2019

  • Ningyue Shang, Bo Ye, Yuetian Sun

gesture flow.png
bottom of page