Vipul Gupta
Vipul is a software engineer at balena and a documentarian running his docs initiative called Mixster. He advocates strongly for open-source, cheesecakes and party parrots.
Wats'on the roof
Managing large scale rescue operations, reducing response times, saving lives
HackMIT 2018
Roof? Wats'on?
- Emotions are strong feelings that grow from different circumstances and situations.
- StacksFX harnesses those emotions and detects your mood in real-time.
- And from that, it recommends music to you. Getting better each time, by learning from you.
How it Works?
- StacksFX uses the camera installed in the laptops to click over 16 frames of the user in one go.
- These images get classified and help in detecting mood.
- On the basis of the mood, songs get recommended.
Browser (Client Side)
Server API
if classified as happy
then StacksFX recommends a song tagged "happy" using Spotify's API
If classified as sad:
then StacksFX
recommends a song tagged "sad" using Spotify's API
Cascade Classifier
How it Works?
- With every recommendation, there is a feedback loop; that learns preferences song by song, constantly building a custom profile of the user.
- Recommends better songs to the user, inching closer to that perfect song.
My emotion is my emotion, none of your emotion !
Feedback System
Under the hood
- StacksFX uses WebRTC to click images
- Cascade classifier of OpenCV for emotion detection
- Spotify API for song recommendations and parsing song metadata
- Django for the backend
- CSS + Bootstrap + Js for frontend
More Features to be added
- Wider range of emotions detected like anger, surprise.
- Improving the feedback system.
- More accessibility in the future - like chrome extensions, packages, web.
- Refactoring code - making it reusable for others.
Thank you !!
[HackMIT 2018] Wats'on the Roof
By Vipul Gupta
[HackMIT 2018] Wats'on the Roof
Disaster can't be prevented, but it can be mitigated. When disaster struck, firefighters, police force and volunteers all come together to help people and save lives. These efforts could be more coordinated and deployed faster if the people could have access to data that tells them where to go, where the most damage is, where help is needed. Our solution, Wats'on the Roof aims to do just that.
