Building an intuitive web-based musical instrument
Ashish Dubey
WebAudio Conference '19
About me
- Software engineer at Grofers
- DevOps/Release Engineering
- Web Dev
- Music tech in spare time
Background
Music from visual artifacts like a painting
Music Mouse: An intelligent music instrument
Pastelloops
Playing music by sketching
Structural approach to building an instrument
- Canvas with basic drawing controls - draw/erase/clear
- Scale selection
- Texture selection
- Volume and tempo controls
- Built with Tone.js, P5.js, react, ant.design
Best explained with a demo!
User interface
Mappings
- y-axis maps to note in the selected scale
- timbre can be changed by changing brush texture
Sounds
- Easy to explore different sounds
- Morphing smoothly as the texture is changed
- Google Magenta's GANSynth used for smooth interpolation between acoustic instruments
Future possibilities
- A time based playback instead of XY
- Canvas stacking for long sequences
- Realtime collaborative jamming
- Melody (and sketch) generation using MusicVAE
Conclusion
Sunday evening hip-hop jam along
Try out live -
Collaborate -
Thank you
Questions?
Twitter @dash1291
Building an intuitive web-based musical instrument
By Ashish Dubey
