My son and I are working to prototype a gamified “practice watcher” built on Azure’s AI Custom Vision, Stream Analytics, and other services. We envision encompassing all musical instruments but are starting with violin. Critical to this is being able to detect that a violin really is being played. The frame of my son above was analyzed by the first iteration of the vision detection model.

Try the prototype vision detection ...

Solution In-Progress ...
• Azure Custom Vision service.
• Custom Vision prediction model, trained to detect a violin being played by a human.
• Azure Stream Analytics (for audio processing).
• Visual Studio 2022
• .NET MAUI app (written in C#), built for Android and iOS.
• App records A/V, sending video frames and audio stream to Azure, which detects whether a violin is being played or not, sends confirmation (or not) back to app for input into game scoring.
• (Allow selection of game modes or types ... leaderboards, badges, etc?)