This project was an assignment for the "Open Source Cinema" course at ITP in Spring 2020.
The assignment was to draw on more of the body to immerse user in creativity.
I created a face-controlled DJ web application called Face DJ for a more immersive musical experience.
The app uses the Tensorflow Facemesh library to accurately detect facial movements and use these data to control music.
I created this app to explore how face-detection technology can be used for controlling/producing music and to provide a more immersive musical experience.
Here's the Live Demo on Glitch.
You need a webcam, headphones, and a microphone to fully use this app. Please try the app in the latest version of Chrome or Firefox. (Safari is currently not supported)
Moving your head by X, Y position changes the song pattern.
Moving your head by Z position changes the volume.
Tilting the head based on the Y-axis controls the panning.
Tilting the head based on the X-axis controls the filter cutoff.
Opening/closing your mouth controls the filter resonance.
Saying something to the microphone plays a vocoder.
For more information, please visit https://github.com/cuinjune/face-dj
Software Engineer | UX Designer | Creative Technologist