Synopsis

POLLOCK is an audiovisual, virtual reality experience aiming to explore the cross-linking of sensory modalities associated with the psychological condition known as chromesthesia. Chromesthesia is a neurological condition in which heard sounds automatically and involuntarily trigger the perception of color. Users will throw virtual paint as a means of triggering sonic events in an immersive 3-D environment. Through the use of different sensors as well as a cutting-edge, virtual reality head-mounted display (the Oculus Rift), users will be able to move freely without restriction, seeing and hearing a new virtual world where color and sound are one in the same.

Project Description
Within the last couple years, the possibility of an immersive, believable virtual reality experience has become even more possible. New technologies are continually closing the gap between the physical and digital worlds at an exponential rate. For example, within the last year, a headmount display called the Oculus Rift has been released to software developers around the world, sparking what is now seen as the beginning of a new age in simulated 3-D experience. What the Oculus Rift does is provide the users eyes with its own separate high-definition screen so that each eye is seeing the same image from a slightly different perspective, creating stereoscopic depth. In other words, it recreates the same physiological phenomenon we humans use everyday to see three-dimensionally. Furthermore, the Oculus Rift uses state-of-the-art motion tracking so that the users field of vision is a complete 360-degree sphere, allowing for unparalleled visual freedom. One can look up, down, left, right, forwards, backwards and never see anything but the virtual world theyve been placed in. The Oculus Rift, coupled with other revolutionary sensor and audio technologies, are what make this project both unprecedented and experimental, yet also very, very possible. Moreover, these technologies are no longer beyond the financial means for an artist like myself, which is why I have been able to purchase my own Oculus Rift as well as other necessary hardware/software out of my own pocket. However, there is still more that needs to be done and the interdisciplinary grant would profoundly lighten this financial load. What I find to be so exciting about POLLOCK is how it embodies so much of what the Music Technology department stands for, as it is not bounded by one artistic medium. It, is, ultimately, an interdisciplinary work, utilizing many different artistic methods, skills, and practices to achieve one, all-encompassing experience. Its not just graphic design. Its not just sound design. Its not just art and technology. Its not just music technology. In some way, shape, or form, its employing the artistry central to the curriculum of every department here at CalArts. This is why I will continue to reach out and collaborate with various members of the CalArts community as I develop this project, as its scope falls into the domain of many other mtiers.

In the artist statement I wrote for my CalArts admission application, I talked about how an experience out in the Brazilian countryside forever changed my lifes direction. Essentially, I was on a bus, and I was looking at the nights sky from my window seat, listening to some music. This experience of gazing at the stars left me with a sensation that Ive described as a cognitive iridescence. I was so overwhelmed with this sublime, sensory overload that I felt I was somehow existing on some other, transcendental plane. I feel that, at the heart of all my artistic endeavors, I am trying to recreate that experience for myself as well as other people. POLLOCK is my most wholehearted attempt at realizing my own artistic ethos. POLLOCK is why Im here at CalArts and this grant will allow me to continue my search for artistic identity.

Artists Statement
Having played music for the majority of my life, my initial artistic output was circumscribed by notes, beats, scales, and chords. But, music, in the classical sense, is only a subset of a larger sound spectrum, and it was my hope to further explore this aural phenomenon at CalArts, designing lush, captivating soundscapes that evoke worlds that transcend our own, worlds that people lose themselves in. POLLOCK is my most ambitious attempt at realizing this dream. I am currently fascinated with the neurological condition known as chromesthesia, where distinct sensory pathways are interconnected so that the visual experience of color and sound are intertwined. Though we perceive them differently, sound and color are, essentially, the product of waves moving in space at different frequencies. In the real world, the sounds and images we perceive are quite dissonant, in the sense that the colors we see are, more likely than not, inharmonically related to the sound we are hearing. In a virtual world, one could theoretically eliminate this dissonance and actually create a semblance of harmony between sound and color. I also have a great interest in surround sound systems. This stems from what I and other audiophiles perceive as a profound difference in quality between an actual, real-life sound event and one that is synthesized through analog or digital mixing consoles. Though the setup of these sound reproduction systems seems pretty straightforward, there is still an ongoing debate as to how to recreate sound in these artificial sonic environments. The current standard surround format only reproduces sounds on a horizontal plane. Ambisonics is a surround sound format that aims to encode audio with both horizontal and vertical information, which, in practice, yields a more accurate representation of how sound would propagate in an actual acoustic space. Since immersion is key to POLLOCK, I am implementing an Ambisonics-based surround system into the audio engine so that users are further enmeshed in the virtual world. Furthermore, Ive recently been thinking about what this project says about being human and why we feel the need to escape from our existence. The concept of virtual reality is not a new one and most, if not all, forms of entertainment are offering some form of distraction from our lives. It seems logical to assume that there is some inherent lacking in our existence that mankind has always dealt with since its inception. The uniqueness of our intellect, our most advantageous evolutionary adaptation, is also the source of our inescapable discontent. This isnt to say that we are never happy; its just that our existence is so ephemeral that all of our emotional states are volatile and fleeting. We continue to build and create as an antidote to our crippling awareness. I hope that POLLOCK will help reveal more insight as to why we humans work so hard to escape from ourselves. Its these and other concepts that are influencing the direction of POLLOCK. I have a lot of work ahead of me, but I welcome the challenge. With the help of my faculty mentor, Owen Vallis, and my team of collaborators, this project is becoming more of a reality everyday. Thank you for your consideration.

Budget
x-IMU Inertial Motion Sensor StrechSense Wireless Flex Sensor Kit 2 x StretchSense soft strain sensors 1 x StretchSense 5-channel sensing circuit 1 x Battery 1 x Charger 4 x Panasonic off-set weight motors EasyAcc Portable 4 x USB External Charger Venom 20C 2S LiPO Battery with Universal Plug Shipping and Handling Grand Total $492.36 $850.00

$63.96 $41.41 $42.99 $9.28 $1,500.00

Budget Notes: Every item except the off-set weight motors are parts that will be installed into the data glove I am constructing. This glove is essential to tracking a users hand orientation, direction of motion, and acceleration. Without these sensors and their respective power units, the data as well as the accuracy of the data necessary to continue this project will be lost. In short, these are essential items for this project. Adjustments can be made, but utilizing other sensors will drastically hamper my ability to bring POLLOCK to completion. The off-set weight motors are used to impose phase modulation on the user ID system so that tracking is possible with multiple Kinect cameras. If the items outlined in this budget are not deemed appropriate by the committee, any and all funds can be redirected towards other necessary purchases, as my projects total budget exceeds the $1,500 limit.

x-IMU Inertial Motion Sensor

StrechSense Wireless Flex Sensor Kit

Rough Draft of POLLOCK Virtual World

Picture of me with Oculus Rift and prototype data glove