The Hammer Without A Nail

Oscillations is a New Art Form Making Meaningful Impact

Imagine a classroom of elementary-aged students. They appear to be sitting cross-legged on the floor or walking around the room, but in reality, they’re dancing. Dancers flash across the screens of the virtual reality headsets the students wear. The performers jump and turn just inches from the young audience members’ eyes. The children can see the subtle flexibility of the dancers, connecting the shifts of movement to the music’s tempo. Looking up, down and side to side, the students see a world that’s unlike their reality and a source of endless curiosity.

Oscillations provides accessibility to a new art experience for young audiences. The dancers don’t have permanent studios filled with a stretching prima ballerina or clapping artistic directors. The performances don’t happen in a big theatres to the sound of an orchestra. It’s dance, but in an elsewhere place: the mind. The responsive virtual reality experience is accessible place anywhere at anytime. Oscillations is a responsive VR immersion that challenges sensual perception. As audiences watch the performance, sensors in the VR headset detect the attentive state of the brain and alter visual and auditory effects to capture the mind’s maximum focus.

Each quarter the Knight Lab Studio class builds Oscillations’ tool. Previous cohorts focused on the VR experience, however no one was talked about experiencing the product for the first time. Finding the initial audience for Oscillations started with the question of accessibility. Which user needs accessible performance art and is likely to engage with an experience like that of Oscillations?

For this research, accessibility is defined as the ability to experience arts education and interpret meaning from the piece. On October 9th, we spoke with Oscillations’ Co-Founders Ilya Fomin and Danielle Perszyk. We raised the question of how this virtual reality performance can be accessed to bring meaningful impact to a specific audience, but the creators struggled to identify its target consumers.

We met with Northwestern University Performance Studies Librarian Greg MacAyeal, who lead us toward resources for background research. The following consisted of a deep background check into accessibility of art. Who were the people lacking accessibility? Why was accessibility even important?

An article by the National Endowment of Art from 2012 notes that engagement in a performance program improved social competence, especially to children from lower socioeconomic status families. Young people with intensive arts experiences also demonstrate higher levels of volunteering and civic engagement, from their experiences of being in shared spaces while performing same activities. The NEA argues that art education is not only important but crucial to developing a sense of community for elementary school students between 4th and 6th grades.

By focusing on arts education, we were able to establish potential audience segments for Oscillations to target. The three groups, “The Art Enthusiast,” “The Elementary Student,” and “The Tech Fan” were potential segments we created based on information of who attends performances. We ultimately decided to focus on the elementary student because of the proven potential of children to learn through art.

Young people with intensive arts experiences demonstrate higher levels of volunteering and civic engagement. According to a 2012 NEA study, young people of low socioeconomic status who engaged in the arts had better grades in school and had higher rates of college enrollment. Students who engaged in dance programs were related to improvements in social competence (pro-social behavior and cooperation) especially for children from lower-SES families.

Access to early arts education is often limited by (1) distance to venues, (2) affordability of performance and (3) availability of shows. Oscillations provides an experience that allow users to interact with the performance happening before their eyes. This tool can overcome obstacles limiting accessibility of performing arts in classrooms. An early age virtual reality experience will expose young people to a new form of technology and art, provoking a similar response to receiving arts education in a traditional classroom setting.

In user-centered design, “affordances” refer to the benefits that extend beyond the core deliverable of a product. A 2011 study found establishes the psychodynamic approach, a framing tool for cognitive growth. The framework means the brain expands its creativity because “tension between conscious reality and subconscious drives.” Oscillations is both a conscious and subconscious experience, testing the expectations of the user with their awareness. As a student’s mind explores their reality, a new form of understanding is formed, “giving tolerance for ambiguity…and mental freedom to find problems,” according to the study. Oscillations is not only entertaining, but also engaging for the mind.

“I can’t imagine that buying a headset would be less expensive to send the student to a show,” said Northwestern University Computer Science Professor Eleanor O’Rourke. “You have a hammer without a nail.”

O’Rourke added that competitive options exist in the market for giving elementary students access to art, and should be a major consideration for Oscillations. In-classroom movies, in-theatre performances, student theatre performances, children’s education mobile apps and museum visits are alternatives with specific ramifications.

The Oscillations difference is what the brain looks like on art. A 2017 study by the Association of Art Historians highlights the idea of neural plasticity, or how the brain’s neurons are impacted by experiences;

“The study of no other subject can introduce a pupil so directly to the problems of his sensory relation to his environment.” Oscillations literally shapes the minds of young audiences members by giving the brain’s sensory cortex function a new set of experiences. Our audience recommendation are consistent with how the brain’s plasticity decreases as the mind ages. Elementary students represent the population with the most plasticity, and are therefore able to benefit the most from a perceptual VR experience.

We recommend that Oscillations approach elementary students from the “4P” approach: product, price, place and promotion. The neuro-social impact of Oscillations is likely to create  positive reinforcement in students and teachers – the more students learn, the more frequently consumers will use the tool, providing feedback and contributing a more dynamic experience. As Oscillations grows more stable, the 4P approach may be adapted for the tertiary and secondary segments, “Art Enthusiasts” and “Tech Fans.” A consistent audience presence will create long-term growth for Oscillations.

What sets Oscillations apart from other virtual reality performances is the meaningful impact it can bring to public education. Not only will it reach the usual “Tech Fans,” but by exploring its use as a tool for teaching and learning, elementary students will receive art experiences that were previously inaccessible. By shifting focus to creating interactive performances appropriate for all audiences, Oscillations has the full potential to become a leading virtual reality experience used for social impact.

About the project

OscillationsImmersive Virtual Experiences in the Performing Arts

Advancements in neuroscience and immersive technologies offer mechanisms for engineering an entirely new mode of performance art one that engages audiences to unprecedented degree. Using the latest VR production techniques, students used motion capture and machine learning to teach a computer to improvise a performance, creating an engaging VR experience.

About the authors

Nicole Fallert

Janet Lee

More results from Oscillations

  • Oscillations Audience Engagement Research Findings

    During the Winter 2018 quarter, the Oscillations Knight Lab team was tasked in exploring the question: what constitutes an engaging live movement arts performance for audiences? Oscillations’ Chief Technology Officer, Ilya Fomin, told the team at quarter’s start that the startup aims to create performing arts experiences that are “better than reality.” In response, our team spent the quarter seeking to understand what is reality with qualitative research. Three members of the team interviewed more...

    Continue Reading

  • Comparing Motion Capture Techniques for Movement Art

    With Oscillations’ connection to the movement arts, it made sense to experiment with existing motion capture technology to find accurate, consistent, and scalable ways to obtain three-dimensional motion data for purposes such as animation or machine learning to augment performances in virtual reality. An additional motivation to learn more about motion capture was connected to our early experiments with spatial audio (read more about them in our spatial audio blog post). Apart from using ambisonic...

    Continue Reading

  • Prototyping Spatial Audio for Movement Art

    One of Oscillations’ technical goals for this quarter’s Knight Lab Studio class was an exploration of spatial audio. Spatial audio is sound that exists in three dimensions. It is a perfect complement to 360 video, because sound sources can be localized to certain parts of the video. Oscillations is especially interested in using spatial audio to enhance the neuroscientific principles of audiovisual synchrony that they aim to emphasize in their productions. Existing work in spatial...

    Continue Reading

  • Audience Engagement Research Findings

    During the Winter 2018 quarter, the Oscillations Knight Lab team was tasked in exploring the question: what constitutes an engaging live movement arts performance for audiences? Oscillations’ Chief Technology Officer, Ilya Fomin, told the team at quarter’s start that the startup aims to create performing arts experiences that are “better than reality.” In response, our team spent the quarter seeking to understand what is reality with qualitative research. Three members of the team interviewed more...

    Continue Reading

  • Giving Audiences Agency Over Art Our Work with Oscillations Dance Studio

    What will art look like in a more technologically advanced world? Ubiquitous computing makes art more readily accessible to anyone--I can look at any a photo of any painting in history in mere seconds using google, and you need only walk outside to see that nearly everyone walks around with headphones playing music from all over the world. When art becomes accessible on a personal level, all people develop specific tastes for the pieces they...

    Continue Reading