A project in cooperation with the masters course „Interactive Installation and Performing Arts Technology“ led by Prof. Chien-Wen Cheng at Taipei Tech.
Sounds of Taipei is an experimental and interactive archive that captures our experiences during a semester abroad at Taipei Tech (NTUT). This project combines visuals, sound, and interaction to create an immersive exploration of the city.
The website features photographs of various locations in Taipei, each enriched with dynamic and interactive elements. For example, selecting an image of the MRT allows users to manipulate it using a slider button. As the slider moves, the image glitches, and the corresponding sound—such as the iconic MRT voice announcements—is distorted, creating a unique audiovisual transformation.
Building A Simple But Durable Contact Microphone
For Sounds of Taipei, we built our own contact microphones to record authentic city sounds, following a guide by Zach Poff. The process began with a piezoelectric disc, shielded audio cable, and a compatible audio plug. We stripped the cable to expose the signal and ground wires, then soldered the ground wire to the outer ring of the piezo disc and the signal wire to its ceramic element. To protect these connections, we used heat-shrink tubing for insulation and strain relief. Finally, we attached the audio plug by soldering the signal wire to its tip and the ground wire to its ground terminal. This simple setup allowed us to capture subtle vibrations and enrich the auditory layer of our interactive archive.
Recording Mechanical Sounds of Taipei
Sound Alterations using cycling74's MAX Programming Software
We used Max/MSP, a visual programming language, as the core tool to synchronize and manipulate sound and visuals for Sounds of Taipei. Max/MSP is ideal for real-time audio-visual processing, enabling a seamless connection between user interactions and media alterations. In our setup, Max/MSP handled the sound input, processing, and output. When a user interacted with the slider on the website, the slider’s data was sent to Max/MSP, which processed the sound files associated with each image. For example, moving the slider altered parameters like pitch, speed, or added effects such as distortion or granular synthesis, making the audio respond dynamically to user input.
To integrate images and sounds, Max/MSP communicated with the website using a WebSocket or similar real-time data exchange method. The slider’s position not only triggered sound alterations in Max/MSP but also controlled the visual distortions created with p5.js on the front end. These visual distortions were integrated into a website layout built with HTML and CSS, while animations and interactive features were powered by p5.js, a creative coding library.
This dynamic interaction ensured that changes in sound were perfectly synchronized with the glitches and transitions applied to the images, creating an immersive experience where sound and visuals were deeply connected. Max/MSP’s capabilities allowed us to amplify the experimental and interactive nature of Sounds of Taipei, seamlessly blending technology and storytelling.
Example for sound manipulation:tapin~ and tapout~ work together to create a delay line for audio signals, then cycle to distort the sound
Connection tool used to connect max and the website
Mechanics x Tech
The website layout was crafted using HTML and CSS, providing a clean and intuitive structure to house the interactive elements. HTML defined the core framework, organizing the images and slider controls, while CSS was used to style the interface, ensuring a visually appealing and user-friendly design.
The animations and image alterations were implemented with p5.js, a powerful creative coding library. When users interacted with the slider, p5.js dynamically applied effects like glitches and distortions to the images. These effects were programmed to respond in real-time, making the visuals feel alive and directly tied to user input. By combining the structural reliability of HTML and CSS with the flexibility of p5.js, we created an engaging platform where users could explore the interplay of sound, image, and interaction.
✧✧✧
Sounds of Taipei serves as both a personal archive and an experimental exploration of technology and storytelling. By inviting users to engage with these interactive elements, we aim to evoke a sense of discovery and reflection, showcasing the spirit of Taipei through a playful yet meaningful digital experience.
Link to Video here.
