Vipul Gupta
Vipul is a software engineer at balena and a documentarian running his docs initiative called Mixster. He advocates strongly for open-source, cheesecakes and party parrots.
StacksFX
A music player that takes an out of the box approach to how you find that perfect song by detecting moods
Emotion-Based??
- Emotions are strong feelings that grow from different circumstances and situations.
- StacksFX harnesses those emotions and detects your mood in real-time.
- And from that, it recommends music to you. Getting better each time, by learning from you.
How it Works?
- StacksFX uses the camera installed in the laptops to click over 16 frames of the user in one go.
- These images get classified and help in detecting mood.
- On the basis of the mood, songs get recommended.
Browser (Client Side)
Server API
if classified as happy
then StacksFX recommends a song tagged "happy" using Spotify's API
If classified as sad:
then StacksFX
recommends a song tagged "sad" using Spotify's API
Cascade Classifier
How it Works?
- With every recommendation, there is a feedback loop; that learns preferences song by song, constantly building a custom profile of the user.
- Recommends better songs to the user, inching closer to that perfect song.
My emotion is my emotion, none of your emotion !
Feedback System
Under the hood
- StacksFX uses WebRTC to click images
- Cascade classifier of OpenCV for emotion detection
- Spotify API for song recommendations and parsing song metadata
- Django for the backend
- CSS + Bootstrap + Js for frontend
More Features to be added
- Wider range of emotions detected like anger, surprise.
- Improving the feedback system.
- More accessibility in the future - like chrome extensions, packages, web.
- Refactoring code - making it reusable for others.
Thank you !!
[IIIT hackathon winner] StacksFX
By Vipul Gupta
[IIIT hackathon winner] StacksFX
A revolutionary music player that recommends songs by detecting your emotions through the camera and constantly learning from your actions through machine learning with a constant feedback loop.
