SOUND SCENE 2025 PROPOSAL
Shao Wen Chew & Amogh Dwivedi
the title of our interactive sound art exhibition is
Help Me Help You
"Besties often empower and uplift each other. Imagine a dynamic gamified musical performance where one player’s success makes the other player’s task easier, and both need to collaborate in order to move forward."
We often rely on close friends for emotional support in order to move forward, and confiding in your bestie and navigating your emotions with them can make your life easier. It is tempting for some to believe that you can do it all alone, but what works more successfully is to embrace dependence and support from another, and allow them to help you. It is important to realize that your journey need not navigated alone, and that the presence of a healthy friendship can add value to both the supporter and the supported’s life.
People are never quite the same, and even contrasting personalities can complement each other. In this arcade-inspired gamified musical performance, two players perform contrasting tasks in order to pass through levels. P1 is the supporter, who through their unbound assurance and willingness to help, makes the task of P2 more manageable. The game does not progress unless and until both players put in the work.
warming up at Ambient Night Spring 2024
Our proposed installation rewards teamwork in many ways. For instance, the music sounds unpleasant if neither player performs well, and on the other end, the music sounds clear if both put in the work. The “score” in the game, triggered by P1’s success, only accumulates if P2 performs well, and P2’s ease of performing the task is defined by P1’s success. Other forms of reward would include the music becoming more layered as the players pass levels, and they are ultimately rewarded with a pleasant music coda which attests to their successful teamwork!
-
The game is modeled after the popular whack-a-mole video game, and requires two players. They respond to visual cues in order to play the game, and their performance impacts the audio fidelity of the music. The resulting original music from the game can be experienced by surrounding people too (much like an e-sports game). We are, in effect, sonifying the quality of teamwork between the two players.​ Additionally, we are also downsampling/upsampling the video fidelity of the camera based on the players’ performance.
-
P1's task would be controlled using 4 LED buttons- the buttons light up when they need to be stuck, and turn off when they are pressed. As the buttons light up, haptic sensors also indicate which button needs to be pressed. The score for P1 only accumulates if P2 is performing their task successfully. In our prototype, we are simulating P1’s performance with a QWERTY keyboard.
-
P2’s task would be controlled using an ultrasonic sensor. Their hand’s placement controls the on-screen cursor, which they must keep within a constantly modulating range. This modulating range widens/narrows based on P1’s own performance- P2’s range widens when P1 is performing well, and narrows when they aren’t. In our prototype, we are simulating P2’s performance with a mouse/touchpad.
-
The players’ performance impacts the music being played. If the players do not perform well, the music sounds convoluted and haphazard. If the players perform well, the music sounds clear and accurate. There is a spectrum of musical fidelity, i.e. 95% performance sounds more musical than 70%.
-
The player’s performance impacts the visual fidelity of the camera input. P1 impacts the X dimensions, while P2 impacts the Y dimensions. Additionally, every success is associated with the “flashes” of green, while failure to move on to the next round results in a flash of red.
-
We have implemented some sound effects in order to give the user feedback on their performance. In our final version, we will be happy to replace them with more compositionally appropriate sound objects in order to make the experience more musical. This also demonstrates our ability to provide people some kind of feedback on their tasks.
-
The game is set up in 3 or more levels. In order to progress to the next level, both players need to accumulate enough success by the end of a musical phrase. The game loops on a musical phrase till all levels are completed. When all levels are completed, a musical “coda” is played, congratulating the winner on their performance. The player is presented with the time it took them to complete the game, as well as the number of points scored.
-
We will aim to feature several musical soundtracks, including more ambient and calmer musical pieces, in our system. The user will be able to choose their track of choice, and the system will perform identically to the prototype.
-
The game presents itself through multiple sensory channels: visual, aural, and tactile. It can also be adjusted to multiple intellectual levels, by adjusting the game's difficulty level. Alternatively, we can remove the difficulty options altogether in favor of an easy mode, in order to establish a common experience for everyone. (Guideline I-A)
-
In our presentation of our exhibition, we will include the experiences of people with disabilities, such as in our posters, visual imagery, blurbs, etc. (Guideline I-B)
-
The game is accommodated for people who are deaf. For people hard of hearing, the game will work purely on a visual level. Both players’ performance is indicated by various colors on screen, such as red visual indicators for subpar performance, and a vibrant green for successful performances. The progress of each level is also intuitively tracked using a visual indicator, which traces the musical phrase. (Guideline II-B)
-
It is also possible to play the game in a non-visual context, much more simplified than the original, so that it gives audio descriptions of the instructions (press “left”/press “right”) for P1, or alternatively we can make use of the haptic sensors. For P2, we can also consider using the stereo width of our setup so that the player is prompted to go to the correct direction, i.e. warning sound comes out of the speaker on the player's left-hand side so that the player moves left (Guideline II-B, IV-C).
-
For deaf and hard-of-hearing people, we provide on-screen text pop-ups which prompt them on their performances within the game. This is demonstrated in our prototype. (Guideline IV-B)
-
All color schemes within the Max patch are high-contrast, and well-readable. We can refine this further as we move along (Guideline VII-D).
-
The game includes visual flashes due to its arcade theme, but for anyone sensitive to epilepsy, an option to turn them off is incorporated in the game. (Guideline VIII-A)
An audiovisual installation in Unreal Engine
A suite of 3 generative compositions in Max/MSP, featuring contrasting styles and sources of inspiration.
Wen is an Audio Artist, Film Producer and Game Developer from Malaysia. Her current few projects are all based in games, either in collaboration with USC, art studios, game jam teams formed online or personal game developments.
Recently she has been been very interested in haptics, audio and user experiences. Her upcoming project will be based in a virtual installation, exploring audio implementation made with Unreal, Metasound and RNBO.
Amogh Dwivedi is a musician. Described as an "incredibly thoughtful, curious, serious, open-minded, imaginative, and a hard-working student", Amogh is keen to realize his musical thoughts in a variety of different ways, be it procedurally generated music in Max/MSP, music written out by hand, or by improvising at the piano. ​