This page simply serves as a window to my hobby exploration of Microsoft Azure's AI/ML services. Here are a few party tricks ... --Nate

• News2Pic
• PupperSpotter
• PandaPicturePusher
• POIProfilePusher
• [WIP] 500thsayer (time-series forecasting of the S&P500 index)
• [WIP] Violink (gamified violin practice watcher/listener)
• [WIP] Generalator (prototype of world conquerer type game)

News2Pic

Have you ever wondered how the top news headlines would look as a single image? No? No, neither have I, but here's a way to do it (and automate it to run daily).

Azure Resources
• Key vault
• Databricks service, with key vault-backed secret scope set.
• Databricks compute cluster with requisite Python libraries installed.
• Databricks notebook created and attached to cluster.
• Storage account (Datalake blob), set to the static website capability and Databricks access via secret scope and personal access token.
• Bing Search service, Databricks access to endpoint via secret scope key stored in key vault.
• Language service, Databricks access to endpoint via secret scope key stored in key vault.
• OpenAI service, Databricks access to endpoint via secret scope key stored in key vault.
• Data Factory, a pipeline with a single activity linked to the Databricks notebook and set to trigger daily.

Databricks Notebook (Python)
1. Get news headlines via Bing Search API.
2. Extract key phrases from headlines via Azure AI Language API. (What's with the puppy? See PupperSpotter below ...)
3. Generate image via OpenAI Dall-E, using key phrases as the text prompt.
4. Add meta details to JSON log.
5. Copy generated image and JSON log to web app folder (of this website). (This page uses jQuery to retrieve the latest image and JSON meta data.)

Datestamp
Headlines
Key phrases
Error message

PupperSpotter

Have you ever wanted to spot the face of a West Highland Terrier puppy among those visual news headlines? Still no? Westies are cute, so it's worth spotting them.

Azure Resources
• (Same key vault, Databricks service and cluster, storage account, and Data Factory as News2Pic.)
• Databricks notebook created and attached to cluster.
• Custom Vision service, Databricks access to endpoint via secret scope key stored in key vault.
• Custom Vision prediction model, trained over several iterations to maximize recall and precision to detect westie heads (model was found to degrade as new poses of westies were generated by the News2Pic routine, so model retraining was needed).
• Databricks notebook run added as second activity of the News2Pic pipeline (see above).

Databricks Notebook (Python)
1. Open News2Pic image and detect objects (look for westie faces) with Custom Vision model via API endpoint.
2. Extract bounding box coordinates (left, top, width, height) from model detection results.
3. Draw bounding box on News2Pic image and save image.
4. Add meta details to JSON log.
5. Copy annotated image and JSON log to web app folder (of this website). (This page uses jQuery to retrieve the latest image and JSON meta data.)

Bounding box (fraction of image)
Probability (box actually bounding a westie face)
Error message