What it does

Sustain is a mobile app that uses AI models to detect when a musician is practicing, and provides feedback on their practice sessions. The app is designed to help musicians of all levels improve their practice habits and achieve their musical goals.

Features

• Watches for instrument presence (AI object detection).
• Listens for instrument sound (AI sound classification).
• Rewards practice points while the instrument is seen and heard.
• Displays encouragements while you practice.
• Logs your practice sessions.

Future Features

• Rewards for streaks (consecutive days practiced) and cumulative time practiced.
• Additional spotter faces.
• Leaderboards or leagues.
• Practice quality feedback (intonation, tone quality).

Usage

1. Download the app from the App Store or Google Play.
2. Open the app and grant camera and microphone permissions.
3. Position your device to see your instrument and hands (a tripod is recommended).
4. Select your instrument and practice duration in the Settings menu.
5. Click the start button to begin your practice session (progress is measured only while you actively practice).

Technical Details

1. Over 600 images of various instruments and playing techniques.
2. Ultralytics Platform YOLO26 annotating and training instance segmentation.
3. YAMNet audio classification model (https://www.kaggle.com/models/google/yamnet).
4. Select your instrument and practice duration in the Settings menu.
5. Flutter / Android Studio / Apple XCode / VSCode / GitHub