Notes for before and after getting AI-900, AI-100, AI-102 certified, as we automate AI workflows in the Azure PaaS cloud
Overview
- AI = Bing = Cortana = Cognitive Services?
- Azure OpenAI
- Competitive futures
- Kinds of Cognitive Services
- Prequisites to this document
- Learning Sequence: basics, simplest first
- Automation necessary for PaaS
- Principled AI vetting
- Azure AI certifications
- Content Moderation
- NLP (Natural Language Processing) services
- LUIS (Language Understanding Intelligent Service)
- LUIS Regional websites
- Create LUIS resource
- Create a LUIS app in the LUIS portal
- Create a LUIS app with code
- Create entities in the LUIS portal
- Create a prediction key
- Install and run LUIS containers
- DEMO JSON responses
- LUIS CLI
- Luis.ai
- Text Analytics API Programming
- Retrieving Azure Cognitive Services API Credentials
- Speech Services
- Vision services
- Conversational AI
- Knowledge mining = Search?
- Search = Bing ?
- Anomaly Detector
- Data Science VM
- Resources
- Notes to be inserted
- Microsoft’s History with AI
- Additionally…
- More
This article presents my notes toward a guided tour to introduce use of Microsoft’s Artificial Intelligence offerings running on the Azure cloud, which Microsoft calls “Cognitive services”.
TL;DR look for “PROTIP:” in this article highlight the author’s hard-won experience.You likely won’t find such information anywhere else. My contribution to the world (to you) is a less overwhelming learning sequence, one that starts with the least complex of technologies used, then more complex ones.
AI = Bing = Cortana = Cognitive Services?
Microsoft has published different lists for what services constitute its “Cognitive Services” brand name to achieve AI-enhanced solutions which mimic human intelligence.
“Cortana” was the brand-name for Microsoft’s AI. Cortana is the name of the fictional artificially intelligent character in the Halo video game series. Cortana was going to be Microsoft’s answer to Alexa, Siri, Hey Google, and other AI-powered personal assistants which respond to voice commands controlling skills that turn lights on and off, etc. However, since 2019, Cortana is considered a “skill” (app) that Amazon’s Alexa and Google Assistant can call, working across multiple platforms.
Microsoft has three service “Providers”:
| Asset type | Resource provider namespace/Entity | Abbre- viation |
|---|---|---|
| Azure Cognitive Services | Microsoft.CognitiveServices/accounts | cog- |
| Azure Machine Learning workspace | Microsoft.MachineLearningServices/workspaces | mlw- |
| Azure Cognitive Search | Microsoft.Search/searchServices | srch- |
-
Search (the “Bing” brand, before OpenAI) was separated out from the “Cognitive Services” to its own at https://docs.microsoft.com/en-us/azure/search, although it’s used in “Conversational AI” using an “agent” (Azure Bot Service) to participate in (natural) conversations. BTW: in 2019 Cortana decoupled from Windows 10 search.
Since October 31st, 2020, Bing Search APIs transitioned from Azure Cognitive Services Platform to Azure Marketplace. The Bing Search v7 API subscription covers several Bing Search services (Bing Image Search, Bing News Search, Bing Video Search, Bing Visual Search, and Bing Web Search),
Azure IoT (Edge) Services are separate.
- https://www.computerworld.com/article/3252218/cortana-explained-why-microsofts-virtual-assistant-is-wired-for-business.html
Azure OpenAI
References:
- https://azure.microsoft.com/en-us/services/openai/
- https://learn.microsoft.com/en-us/training/paths/get-started-with-artificial-intelligence-on-azure/
- https://wilsonmar.github.io/genai for definition of terms and tutorials about Generative AI, which produce new content based on what is described in the input. The OpenAI models are a collection of generative AI models that can produce language, code, and images.
- https://azure.microsoft.com/pricing/details/cognitive-services/openai-service
In 2019 Microsoft invested $1 billion in the company as time on Azure cloud and to develop a large-scale AI supercomputer built exclusively for OpenAI’s research in Azure. Azure (with RBAC capabilities) power all of OpenAI’s workloads. OpenAI is a San Francisco-based artificial intelligence research laboratory. OpenAI was founded by Elon Musk, Sam Altman, Greg Brockman, and Ilya Sutskever in December 2015 (to compete with Google’s DeepMind acquisition). For that, in 2023, OpenAI exclusively licensed (closed-source) GPT-3 to Microsoft for their products and services. OpenAI’s mission is to artificial general intelligence (AGI). In 2022, OpenAI made available their GPT-3.5 foundation model for free trial, offering several categories of capabilities
- ChatGPT: Generating natural language: summarizing complex text for different reading levels, suggesting alternative wording for sentences, etc.
- Generating code: translating code from one programming language into another, identifying and troubleshooting bugs in code, etc.
- Generating images from text prompts, based on OpenAI’s DALL-E model.
NOTE:
Microsoft leverages OpenAI in ALL products and services (including Microsoft Teams):
Click picture for full-page view.
In the Microsoft AI Portfolio diagram above, Azure’s AI services are categorized into three groups:
- Applied AI Services
- Cognitive Services (the pillars of services: vision, speech, language, decision, plus the Azure OpenAI Service)
- Azure’s Machine Learning (ML) platform
OpenAI is a “separate” service from Azure Cognitive Services because now “traditional” Azure Cognitive Services focuses on making predictions based textual and discrete data where OpenAI added to ML “attention” algorithms working on binary data (voice, images, and video) to enable it to perform “Generative AI” which produces new content based on what is described in the input.
- Assuming that you have a Azure account and subscription at the Azure portal https://portal.azure.com
-
Consider pricing at https://azure.microsoft.com/pricing/details/cognitive-services/openai-service
OpenAI put a paywall behind its more advanced GPT-4 models,
Billing is based on 1,000 “tokens” increments, with the first 100,000 tokens per month free. Beyond that:
“Standard” use of the older/more limited gpt-3.5-turbo model is then $0.002 per 1,000 tokens.
Charges for the GPT-4 model has two dimensions: process stage and the size of the foundational model used :
- The “8K context” has 13 billion parameters.
-
The “32K context” has 175 billion parameters.
- Prompt
- Completions
Images generated using DALL-E are $2 per 100 images.
available for free for the first 100,000 tokens per month, then $0.004 per 1,000 tokens. Doing the math,
-
Apply for access to Asure OpenAI for your Region and Currency:
https://aka.ms/oaiapply
Competitive futures
Microsoft competes for talent with Google, Amazon, IBM, China’s Tencent.
BTW, by contrast, Bernard Marr identified four types of AI evolving:
-
“reactive” machines (such as Spam filters and the Netflix recommendation engine) are not able to learn or conceive of the past or future, so it responds to identical situations in the exact same way every time.
-
“limited memory” AI absorbs learning data and improve over time based on its experience, using historical data to make predictions. It’s similar to the way the human brain’s neurons connect. Deep-learning algorithms used today is the AI that is widely used and being perfected today.
-
“theory of mind” is when AI acquires decision-making capabilities equal to humans, and have the capability to recognize and remember emotions, and adjust behavior based on those emotions.
-
“self-aware”, also called artificial superintelligence (ASI), is “sentient” understanding of of its own needs and desires.
https://www.forbes.com/sites/bernardmarr/2018/02/14/the-key-definitions-of-artificial-intelligence-ai-that-explain-its-importance/#22db12bb4f5d
What can AI do?
Microsoft “democratizes” AI and Machine Learning by providing a front-end that hides some of the complexities, enabling them to be run possibly without programming.
-
Microsoft’s Azure AI gallery of samples and users’ contributions:
https://gallery.azure.ai/browse
These are what are called “narrow” or “weak” AI.
-
Microsoft hands-on DEMOS at:
Case studies of how people are already making use of AI/ML to save time and money:
- Predictive Maintenance data science webinar
- Recommendation Engines (such as what Netflix)
- modsy.com 3D view
- Customer and Partner Success Stories for “bot”
Kinds of Cognitive Services
As of this writing, in various marketing and certification training DOCS, Azure Cognitive Services is categorized into these (which is the basis this article is arranged. Click on the bolded word to jump to it below):
-
Vision (Visual Perception) - interpret the world visually through cameras, videos, images
-
Speech - Text-to-Speech and Speech-to-Text to interpret written or spoken language, and respond in kind.
-
Language - aka Natural language Processing (NLP) to translate text (Text Analysis), etc.
-
Decision (Making) - supervised and unsupervised machine learning
-
An alphabetical list of Microsoft’s AI/ML services at Cognitive Services pricing page at https://azure.microsoft.com/en-us/pricing/details/cognitive-services:
PROTIP: The reconciliation below between marketing Feature categories and the Kind of Cognitive service speecified in CLI az commands provides quick access to web pages and reveals some contradictions that need to be clarified, by category: The table also has a column to summarize free machine time.
Kinds with ? are known in websites but not listed by this az command (which you can run after you’re setup to run CLI commands:
az cognitiveservices account list-kinds
PROTIP: CAUTION: Each service has a different maturity level in its documentation at azure.microsoft.com/en-us/downloads, such as SDK for Python open-sourced at github.com/azure/azure-sdk-for-python, described at docs.microsoft.com/en-us/azure/developer/python.
Also, individual products may be in preview or not available depnding on region.
Hybrid workflows
PROTIP: Although most Microsoft’s samples and tutorials usually focus on one service at a time, actual production work enjoyed by real end-users usually involves a pipeline of several services. For example: ingesting (stream processing) a newsfeed:
That and other flows are in Microsoft’s Architecture Center
IMPORTANT PROTIP: Microsoft allows its free “F0” tier to be applied to only a single Cognitive Service at a time. To remain free, you would need to rebuild a new Cognitive Service with a different “Kind” between steps.
Prequisites to this document
This document assumes that you have done the following:
- Get onboarded to a Microsoft Azure subscriptions and learn Portal GUI menu keyboard shortcuts.
- Setup a CLI scripting environment in shell.azure.com.
- Use CLI to Create a Cognitive Service to get keys to call the first REST API from among sample calls to many REST APIs: the Translator Text API.
- Setup PowerShell scripts
Learning Sequence: basics, simplest first
This document covers:
-
Automatically shut down Resource Groups of a Subscription by creating a Logic App.
-
Run an API connecting to an established endpoint (SaaS) you don’t need to setup: Bing Search.
-
Create Functions
-
Create a Workspace resource to run …
- Create a Workspace resource and
Create Compute instance to run
Automated ML of regression of bike-rentals. - Create Compute instance to run a iPython notebook
- Create ML Workspace in Portal, then ml.azure.com
-
Us cognitivevision.com to Create Custom Vision for …
- QnA Maker Conversational AI
- Train a Machine Learning model using iPython notebook
- IoT - “Hey Google, ask Azure to shut down all my compute instances”.
Automation necessary for PaaS
IMPORTANT PROTIP: As of this writing, Microsoft Azure does NOT have a full SaaS offering for every AI/ML service. You are required to create your own computer instances, and thus manage machine sizes (which is a hassle). Resources you create continue to cost money until you shut them down.
So after learning to set up the first compute service, we need to cover automation to shut them all down while you sleep.
So that you’re not tediously recreating everything everyday, this tutorial focuses on automation scripts (CLI Bash and PowerShell scripts) to create compute instances, publish results, then shut itself down. Each report run overwrites files from the previous run so you’re not constantly piling up storage costs.
Another reason for being able to rebuild is that you if you find that the pricing tier chosen is no longer suitable for your solution, you must create a new Azure Cognitive Search resource and recreate all indexes and objects.
When you use my Automation scripts at https://github.com/wilsonmar/azure-quickly/ to create resources the way you like, using “Infrastructure as Code”, so you can throw away any Subscription and begin anew quickly.
My scripts also makes use of a more secure way to store secrets than inserting them in code that can be checked back into GitHub.
Effective deletion hygiene is also good to see how your instances behave when it takes advantage of cheaper spot instances which can disappear at any time. This can also be used for “chaos engineering” efforts.
To verify resource status and to discuss with others, you still need skill at clicking through the Portal.azure.com, ML.azure.com, etc.
References:
-
VIDEO: To release IP address, don’t stop machines, but delete the resource.
-
VIDEO: shut down automatically all your existing VMs (using a PowerShell script called by a scheduled Logic App), by <a target=”_blank” href=”https://www.youtube.com/channel/UCAr20GBQayL-nFPWFnUHNAA”“>Frank Boucher</a> at github.com/FBoucher
-
Auto-shutdown by Resource Manager on a schedule is only for VMs in DevOps
-
https://www.c-sharpcorner.com/article/deploy-a-google-action-on-azure/
-
start/stop by an Automation Acount Runbook for specific tags attached to different Resource Groups: Assert: “AutoshutdownSchedule: Tuesday” run every hour. https://translate.google.com/translate?sl=auto&tl=en&u=https://github.com/chomado/GoogleHomeHack”>
Principled AI vetting
Using Microsoft’s API algorithms and data (such as celebrity faces, landmarks, etc.) means there has be some vetting by Microsoft’s FATE (Fairness, Accountability, Transparency, and Ethics) research group:
Microsoft’s ethical principles guiding the development and use of artificial intelligence with people:
- Fairness: AI systems should treat all people fairly.
- Reliability & Safety: AI systems should perform reliably and safely.
- Privacy & Security: AI systems should be secure and respect privacy.
- Inclusiveness: AI systems should empower everyone and engage people.
- Transparency: AI systems should be understandable.
- Accountability: AI systems should have algorithmic accountability.
-
DEMO: Hands on with AI/Guidelines for Human-AI Interaction: Click each card to see examples of each guideline
https://aka.ms/hci-demo which redirects you to
https://aidemos.microsoft.com/guidelines-for-human-ai-interaction/demo- Initially - make clear what the system can do & how well the system can do what it can do.
- During interaction - Time services based on context; show contexually revelvant info; Match revelvant social norms; Migrate social biases.
- When Wrong - support efficient invocation, dismissal, correction; Scope services when in doubt.
- Over Time - remember recent interactions; learn from user behavior; update and adapt cautiously; encourage granular feedback
PROTIP: Some fonts are real small. Zoom in to read it.
Resources:
- https://www.microsoft.com/ai/responsible-ai-resources
- VIDEO: Ethics an Artificial Intelligence at the DotNet Day Zurich May 28th, 2019, by Laurent Bugnion.
Azure AI certifications
Among Microsoft’s Azure professional certifications:
-
AI-900 is the entry-level exam ($99).
- AI-102 (with free re-cert after 1-year) replaces
- AI-100 (with free re-cert after 2-years) on June 30, 2021.
Both AI-100 and AI-102 are $165. The shift is from infrastructure (KeyVault, AKS, Stream Analytics) to programming C#, Python, and curl commands.
Previous exam 774 is now been retired. It was based on VIDEO: Azure Machine Learning Studio (classic) web services, which reflected “All Microsoft all the time” using proprietary “pickle” (pkl) model files. Classes referencing it are now obsolete.
DP-203 Data Engineering on Microsoft Azure goes into how to use machine learning in Azure Synapse Analytics.
LAB references DP-090 goes into implementing a Machine Learning Solution with Databricks.
Coursera video course: Developing AI Applications on Azure by Ronald J. Daskevich at LearnQuest is structured for 5 weeks. Coursera’s videos shows the text at each point of its videos. NOTE: It still sends people to https://notebooks.azure.com and covers Microsoft’s TDSP (Team Data Science Process) VIDEO:
- Business understanding (Charter, Objectives, Data sources, Data dictionaries)
- Data acquisition and understanding (Clean dataset, pipeline)
- Modeling
- Deployment model for use (testing) Flow: entry script to accept requests and score them, Deployment configs
- Customer Acceptance
AI-900
PROTIP: Here’s a must-see website: http://aka.ms/AIFunPath which expands to Exam definitions are at Microsoft’s LEARN includes a free text-based tutorial called “Learning Paths” to learn skills:
- Describe AI workloads and considerations (15-20%)
- Describe fundamental principles of machine learning on Azure (30-35%)
- Describe features of computer vision workloads on Azure (15-20%)
- Describe features of Natural Language Processing (NLP) workloads on Azure (15-20%)
- Describe features of conversational AI workloads on Azure (15-20%)
The MS LEARN site refers to files in https://github.com/MicrosoftLearning/mslearn-ai900
- Ravi Kirans’ Study Guide contains links to MS Docs.
Tim Warner has created several video courses on AI-900 and AI-100:
-
CloudSkills.io Microsoft Azure AI Fundamentals course references
https://github.com/timothywarner/ai100cs -
On OReilly.com, his “Crash Course” Excel spreadsheet of exam objectives.
-
OReilly.com references
https://github.com/timothywarner/ai100 -
CloudAcademy’s 4h AI-900 video course includes lab time (1-2 hours at a time).
-
https://www.udemy.com/course/microsoft-ai-900/
-
https://www.itexams.com/info/AI-900
-
Emilio Melo on Linkedin Learning
Practice tests:
- https://www.whizlabs.com/learn/course/microsoft-azure-ai-900/
- https://www.examtopics.com/exams/microsoft/ai-900/
- https://www.whizlabs.com/learn/course/designing-and-implementing-an-azure-ai-solution/
2-hour AGuideToCloud video class by Susanth Sutheesh
AI-102
AI-102 is intended for software developers wanting to build AI-infused applications.
AI-102 focuses on the use of pre-packaged services for AI development, not on the development of custom models using Azure Machine Learning, which is covered by exam DP-100.
First setup development environments:
- Visual Studio Code and add-ons for C#, Python,
- NodeJs for Bot Framework Composer and Bot Framework Emulator
AI-102 exam, as defined at Microsoft’s LEARN has free written tutorials on each of the exam’s domains:
- Plan and manage an Azure Cognitive Services solution (15-20%)
- Implement Computer Vision solutions (20-25%)
- Implement natural language processing solutions (20-25%)
- Implement knowledge mining solutions (15-20%)
- Implement Conversational AI solutions (15-20%)
PROTIP: Unlike the AI-100 (which uses Python Notebooks), hands-on exercises in Microsoft’s 5-day live course AI-102T00: Designing and Implementing a Microsoft Azure AI Solution (with cloud time) consists of C# and Python programs at https://github.com/MicrosoftLearning/AI-102-AIEngineer (by Graeme Malcolm).
Modules in the course, shown at https://microsoftlearning.github.io/AI-102-AIEngineer/. Modules:
- Introduction to AI on Azure
- Developing AI Apps with Cognitive Services
- Getting Started with Natural Language Processing
- Building Speech-Enabled Applications
- Creating Language Understanding Solutions
- Building a QnA Solution
- Conversational AI and the Azure Bot Service
- Getting Started with Computer Vision
- Developing Custom Vision Solutions
- Detecting, Analyzing, and Recognizing Faces
- Reading Text in Images and Documents
- Creating a Knowledge Mining Solution
Resources:
- Preview 45 min. Exam: Designing and Implementing an Azure AI Solution (AI-102)
- Ravi’s links still refer to AI-100
AI-100 Azure AI Engineer Associate
On June 30, 2021 Microsft retired the AI-100 exam in favor of AI-102 exam (avilable in $99 beta since Feb 2021). AI-100 exam, as defined at Microsoft’s LEARN has free written tutorials on each of the exam’s domains:
- Analyze solution requirements (25-30%)
- Design AI solutions (40-45%)
- Implement and monitor AI solutions (25-30%)
https://github.com/MicrosoftLearning/AI-100-Design-Implement-Azure-AISol
https://github.com/MicrosoftLearning/Principles-of-Machine-Learning-Python
- Plan and manage an Azure Cognitive Services solution (15-20%)
- Implement Computer Vision solutions (20-25%)
- Implement natural language processing solutions (20-25%)
- Implement knowledge mining solutions (15-20%)
- Implement conversational AI solutions (15-20%) - chatbots
Raza Salehi (@zaalion) created on OReilly.com an AI-100 exam prep “crash course” which references his <a target=”_blank” href=”https://github.com/zaalion/oreilly-ai-100 and https://github.com/zaalion/uy-cognitve-services-crash-course
Guy Hummel’s CloudAcademy.com 7hr AI-100 video course.
Raza Salehi created on Pluralsight.com a series for Microsoft Azure AI Engineer (AI-100)
Practice tests:
- https://www.whizlabs.com/learn/course/microsoft-azure-ai-100/
Sample ML Code
PROTIP: AI-102 is heavy on questions about coding.
Samples (unlike examples) are a more complete, best-practices solution for each of the snippets. They’re better for integrating into production code.
github.com/Azure-Samples from Microsoft offers samples code to use Cognitive Services REST API by each language:
https://docs.microsoft.com/en-us/samples/azure-samples/azure-sdk-for-go-samples/azure-sdk-for-go-samples/
A complete sample app is Microsoft’ Northwinds Traders consumer ecommerce store. install
Cognitive Services SDK Samples for:
Tim Warner’s https://github.com/timothywarner/ai100 includes Powershell scripts:
- keyvault-soft-delete-purge.ps1
- keyvault-storage-account.ps1
- python-keyvault.py
- ssh-to-aks.md - SSH into AKS cluster nodes
- xiot-edge-windows.ps1
- autoprice.py
Among Azure Machine Learning examples is a CLI at https://github.com/Azure/azureml-examples/tree/main/cli
Content Moderation
https://docs.microsoft.com/azure/cognitive-services/Content-Moderator/overview?WT.mc_id=Portal-Microsoft_Azure_Support#data-privacy-and-security
https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/client-libraries?tabs=visual-studio&pivots=programming-language-csharp
Response from the Text Moderation API include:
- What type of potentially unwanted words were found (Profanity)
- A list of potentially unwanted words found in the text.
- Possible personally identifiable information (PII) found in the text.
Content Moderation (Evaluate, Find Faces, Match, OCR)
- G+/ to Content Moderator”
-
“Create Custom Moderator”
Oops!
Could not create the marketplace item
This marketplace item is not available.
When working:
-
Select Create.
-
Enter a unique name for your resource, select a subscription, and select a location close to you.
-
Select the pricing tier for this resource, and then select F0.
-
Create a new resource group
- Select Create.
Create Cognitive Services
Azure Machine Learning 2.0 CLI (preview) examples
https://github.com/Azure-Samples/Cognitive-Services-Vision-Solution-Templates
BTW https://docs.microsoft.com/en-us/samples/azure-samples/cognitive-services-quickstart-code/cognitive-services-quickstart-code/ https://github.com/Azure-Samples/cognitive-services-sample-data-files
My script does the same as these manual steps:
- In Portal.azure.com
- G+\ Cognitive Services
- Click the Name you created.
- Click “Keys and Endpoint” in the left menu.
-
Click the blue icon to the right of KEY 1 heading to copy it to your invisible Clipboard.
-
Endpoint: https://tot.cognitiveservices.azure.com/
TODO: DOCS: Automate above steps to create compute and server startpup script.
PROTIP: These instructions are not in Microsoft LEARN’s tutorial.
“Your document is currently not connected to a compute. Switch to a running compute or create a new compute to run a cell.”
- Click the Run triangle for “Your document is currently not connected to a compute.”
- “Create compute”
- Virtual machine type: CPU or GPU
- Virtual machine size: Select from all options (64 of them) QUESTION: What is the basis for “recommended”?
- The cheapest is “Standard_F2s_v2” with “2 cores, 4GB RAM, 16GB storage” for Compute optimized at “$0.11/hr”. See Microsoft’s description of virtual machine types here.
- Next
- Compute name: PROTIP: Use 3-characters only, such as “wow” or “eat”.
- Enable SSH access: leave unchecked
- Create
- Wait (5 minutes) for box to go from “Creating” to “Running”.
Kinds of Cognitive Services CLI
You would save money if you don’t leave servers running, racking up charges.
You can confidently delete Resource Groups and all resources attached if you have automation in CLI scripts that enable you to easily create them later.
Instead of the manual steps defined in this LAB, run my Bash script in CLI, as defined by this DOC:
-
G+\ Cognitive Services.
- Click the ＋Create a resource button, search for Cognitive Services, and create a Cognitive Services resource with the following settings:
- Subscription: Your Azure subscription.
- Resource group: Select or create a resource group with a unique name.
- Region: Choose any available region:
- Name: Enter a unique name.
- Pricing tier: S0
- I confirm I have read and understood the notices: Selected.
TODO: Instead of putting plain text of cog_key in code, reference Azure Vault. Have the code in GitHub.
Azure has a cognitiveservices CLI subcommand.
https://docs.audd.io/?ref=public-apis
Content moderator
Content Moderator services detect potentially offensive or unwanted content.
https://www.youtube.com/watch?v=gVFiA6ZQNAw
Metrics Advisor
Metrics Advisor monitors metrics and diagnoses issues.
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-python/metricsadvisor-samples/
Personalizer
Personalizer creates rich, personalized experiences for every user.
- Raza Salehi’s Personalizer
NLP (Natural Language Processing) services
Intro: Tutorial: https://docs.microsoft.com/en-us/learn/paths/explore-natural-language-processing
NLP enables the creation of software that can:
- Analyze and interpret text in documents, email messages, and other sources.
- Interpret spoken language, and synthesize speech responses.
- Automatically translate spoken or written phrases between languages.
- Interpret commands and determine appropriate actions.
- Recognize Speaker based on audio.
Within Microoft, NLP consists of these Azure services (described below):
- LUIS (Language Understanding Intelligent Service)
- Text Analytics
- Speech
- Translator Text
SpeakerRecognition
Speaker Recognition for authentication.
In contrast, Speaker Diarization groups segments of audio by speaker in a batch operation.
LUIS (Language Understanding Intelligent Service)
Think of “LUIS” as Amazon Alexa’s frienemy.
Utterances are input from the user that your app needs to interpret.
https://github.com/Azure-Samples/cognitive-services-language-understanding
CHALLENGE: Add natural language capabilities to a picture-management bot.
- Create a LUIS Service Resource (in Python or C#)
- Add Intents
- Add utterances
- Work with Entities
- Use Prebuilt Domains and Entities
- Train and Publish the LUIS Model
https://www.slideshare.net/goelles/sharepoint-saturday-belgium-2019-unite-your-modern-workplace-with-microsofsts-ai-ecosystem
LUIS Regional websites
Perhaps for less latency, create the LUIS app in the same geographic location where you created the service:
- “(US) West US” in North America: www.luis.ai
- “(Europe) West Europe” and “(Europe) Switzerland North”: eu.luis.ai
- “(Asia Pacific) Australia East”: au.luis.ai
Create LUIS resource
NOTE: LUIS is not a service like “Cognitive Services”, but a Marketplace item:
- In the Azure portal, select + Create a resource.
- In the “Search services and Marketplace” box, type LUIS, and press Enter.
- In search results, select “Language Understanding from Microsoft”.
- Select Create.
- Leave the Create options set to Both (Authoring and Prediction).
- Choose a subscription.
- Create a new resource group named “LearnRG”.
-
Enter a unique name for your LUIS service. This is a public sub-domain to a LUIS Regional websites (above)
PROTIP: Different localities have different costs.
- For Authoring Location, choose the one nearest you.
- For Authoring pricing tier, select F0
- Set your Prediction location to the same region you chose for Authoring location
- For Prediction pricing tier, select F0. If F0 is not available, select S0 or other free/low cost tier.
- Select Review + Create.
- If the validation succeeds, select Create.
- After “Your deployment is complete” appears, click “Go to resource” (within Cognitive Services).
- Within the Resource Management section, select the “Keys and Endpoints” section.
- Click the blue button to the right of KEY 1’s value. That copies it to your Clipboard.
-
If you are making use of my framework, open the environment variables file:
code ../setmem.sh
-
Locate the variables and replace the sample values:
export MY_LUIS_AUTHORING_KEY=”abcdef1234567896b87f122281e9187e” export MY_LUIS_ENDPOINT=””
SECURITY PROTIP: Putting key values in a global variables definition file separate from the code.
authoring_key = env:MY_LUIS_AUTHORING_KEY authoring_endpoint = env:MY_LUIS_ENDPOINT
- Select a LUIS Regional websites (above)
- Login.
- Choose an authoring resource (Subscription). If there are several listed, select the top one.
-
Create a new authoring service (Azure service).
Create a LUIS app in the LUIS portal
-
Now at https://www.luis.ai/applications, notice at the upper-right “PictureBotLUIS (westus, F0)” where the Directory name usually appears in the Portal.
The pages displayed will be different if you have already created a LUIS app or have no apps created at all. Select either Go to apps, or the apps option that is available on your initial LUIS page.
-
Select “+ New app” for conversation Apps for the “Create new app” pop-up dialog.
The LUIS user interface is updated on a regular basis and the actual options may change, in terms of the text used. The basic workflow is the same but you may need to adapt to the UI changes for the text on some elements or instructions given here.
Take note of the other options such as the ability to import JSON or LU file that contains LUIS configuration options.
-
Give your LUIS app a name, for example, PictureBotLUIS.
-
For Culture, type “en-us” then select the appropriate choice for you language.
- Optionally, give your LUIS app a description so it’s clear what the app’s purpose is.
- Optionally, if you already created a Prediction Resource, you can select that in the drop-down for Prediction resource.
-
Select Done.
-
Dismiss the guidance dialog that may display.
DEFINITION: Each bot action has an intent invoked by an utterance, which gets processed by all models. The top scoring model LUIS selects as the prediction.
An utterance that don’t map to existing intents is called the catchall intent “None”.
Intents with significantly more positive examples (“data imbalance” toward that intent) are more likely to receive positive predictions.
Create a LUIS app with code
-
On your local machine (laptop), install Visual Studio Code for your operating system.
-
If you will be completing your coding with Python, ensure you have a Python environment installed locally. Once you have Python installed, you will need to install the extension for VS Code.
Alternately, to use C# as your code language, start by installing the latest .NET Core package for your platform. You can choose Windows, Linux, or macOS from the drop-down on this page. Once you have .NET Core installed, you will need to add the C# Extension to VS Code. Select the Extensions option in the left nav pane, or press CTRL+SHIFT+X and enter C# in the search dialog.
-
Create a folder within your local drive project folder to store the project files. Name the folder “LU_Python”. Open Visual Studio Code and Open the folder you just created. Create a Python file called “create_luis.py”.
cd cd projects mkdir LU_Python cd LU_Python touch create_luis.py code .
-
Install the LUIS package to gain access to the SDK.
sudo pip install azure-cognitiveservices-language-luis
-
Open an editor (Visual Studio Code)
Create entities in the LUIS portal
- Be signed in to your LUIS app and on the Build tab.
- In the left column, select Entities, and then select + Create.
- Name the entity facet (to represent one way to identify an image).
-
Select Machine learned for Type. Then select Create.
Create a prediction key
https://docs.microsoft.com/en-us/learn/modules/manage-language-understanding-intelligent-service-apps/2-manage-authoring-runtime-keys
- Sign in to your LUIS portal.
- Select the LUIS app that you want to create the prediction key for.
- Select the Manage option in the top toolbar.
-
Select Azure Resources in the left tool bar.
Unless you have already created a prediction key, your screen should look similar to this. The key information is obscured on purpose.
A Starter Key provides 1000 prediction endpoint requests per month for free.
Examples of utterances are on the Review endpoint utterances page on the Build tab.
Install and run LUIS containers
docker pull mcr.microsoft.com/azure-cognitive-services/luis:latest
-
Specify values for Billing LUIS authoring endpoint URI and ApiKey:
docker run --rm -it -p 5000:5000 --memory 4g --cpus 2 --mount type=bind,src=c:\input,target=/input --mount type=bind,src=c:\output\,target=/output mcr.microsoft.com/azure-cognitive-services/luis Eula=accept Billing={ENDPOINT_URI} ApiKey={API_KEY}”–mount type=bind,src=c:\output\,target=/output” indicates where the LUIS app saves log files to the output directory. The log files contain the phrases entered when users hit the endpoint with queries.
-
Get the AppID from your LUIS portal and paste it in the placeholder in the command.
curl -G \ -d verbose=false \ -d log=true \ --data-urlencode "query=Can I get a 5x7 of this image?" \ "http://localhost:5000/luis/v3.0/apps/{APP_ID}/slots/production/predict" -
Get the AppID from your LUIS portal and paste it in the placeholder in the command.
curl -X GET \ "http://localhost:5000/luis/v2.0/apps/{APP_ID}?q=can%20I%20get%20an%20a%205x7%20of%20this%20image&staging=false&timezoneOffset=0&verbose=false&log=true" \ -H "accept: application/json"
DEMO JSON responses
-
DEMO: voice control lighting in a virtual home.
-
Select suggested utterances to see the JSON response:
- Book me a flight to Cairo
- Order me 2 pizza
- Remind me to call my dad tomorrow
- Where is the nearest club?
Type instructions, use the microphone button to speak commands.
LUIS identifies from your utterance your intents and entities.
Entity types
- List - fixed, closed set of related words (small, tiny, smallest). Case-sensitve
- RegEx - (credit card numbers)
- Prebuilt
- Pattern.Any
- Machine Learned
LUIS CLI
Run my az-luis-cli.sh.
DOCS:
https://github.com/cloudacademy/using-the-azure-machine-learning-sdk
-
Create a Resource referenced when LUIS Authoring is defined. The resource name should be lower case as it is used for the endpoint URL, such as:
https://luis-resource-name.cognitiveservices.azure.com/
-
The Bot Framework CLI requires Node.js.
npm i -g npm
- Install Bot Framework Emulator per instructions at:
https://github.com/Microsoft/BotFramework-Emulator/blob/master/README.md
Download to downloads folder at:
https://aka.ms/bf-composer-download-mac - Install Bot Framework Composer per instructions at:
https://docs.microsoft.com/en-us/composer/install-composer
On a Mac, download the .dmg
https://aka.ms/bf-composer-download-mac -
Use Node.js to install the latest version of Bot Framework CLI from the command line:
npm i -g @microsoft/botframework-cli
https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-howto-bf-cli-deploy-luis?view=azure-bot-service-4.0
LUIS can generate from models a TypeScript or C# typw (program code).
Luis.ai
-
DEMO: https://www.luis.ai
-
Sign in and create an authoring resource refercening the Resource Group.
PROTIP: In the list of Cognitive services kinds, resources and subscription keys created for LUIS authoring are separate than ones for prediction runs so that utilization for the two can be tracked separately.
https://aka.ms/AI900/Lab4 which redirects to
Create a language model with Language Understanding which trains a (LUIS) language model that can understand spoken or text-based commands. He’s Alexa’s boyfriend, ha ha. -
Process Natural Lanaguage using Azure Cognitive Language Services
https://github.com/MicrosoftLearning/AI-102-LUIS contains image files for reference by https://github.com/MicrosoftLearning/AI-102-Code-Repos https://github.com/MicrosoftLearning/AI-102-Process-Speech
PROTIP: LUIS does not perform text summarization. That’s done by another service in the pipeline.
- Utterance is the user’s input that a model needs to interpret, such as “turn the lights on”.
- Entity is the word (or phrase) that is the focus of the utterance, such as “light” in our example.
- Intent is the action or task that the user wants to execute. It reflects in utterance as a goal or purpose. For example, “TurnOn”.
References:
- VIDEO: Starship commander enabled in-game voice commands using Azure.
-
Adding Language Understanding to Chatbots With LUIS by Emilio Meira
- github.com/chomado/GoogleHomeHack in slides by Madoka Chiyoda
Text Analytics API Programming
NOTE: The previous version references interactive Python Notebooks such as MS LEARN HANDS-ON LAB referencing “07 - Text Analytics.ipynb”.
-
Look at the DEMO GUI at:
-
Click “Next Step” through the various processing on a sentence:
- Sentiment analysis (how positive or negative a document is)
-
Key phrase extraction
- Entity Linking to show link in Wikipedia.
-
Bing Entity Search
- Text to Speech services:
- Language Detection (is it English, German, etc.)
- Translator Text
API:
Some Text Analytics API services are synchronous and asynchronous
Cloud Academy lab “Using Text Analytics in the Azure Cognitive Services API”
docs.microsoft.com/en-us/samples/azure/azure-sdk-for-python/textanalytics-samples
Azure Text Analytics client library samples for JavaScript has step-by-step instructions:
-
Logging into the Microsoft Azure Portal
<a target=”_blank” href=”https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.CognitiveServices%2Faccounts”“>Cognitive Services</a>
-
Select the service already defined for you.
Retrieving Azure Cognitive Services API Credentials
-
At the left menu click “Keys and Endpoint”. The Endpoint URL contains:
https://southcentralus.api.cognitive.microsoft.com/
The Endpoint is the location you’ll be able to make requests to in order to interact with the Cognitive Services API. The Key1 value is the key that will allow you to authenticate with the API. Without the Key1 value, you will receive unauthenticated errors.
-
In the Azure Portal, type Function App into the search bar and click Function App:
alt
- Click the only option:
alt
You’ll be brought to the Function App blade. While a complete summary of function apps is outside the scope of this lab, you should know the function apps allow you to create custom functions using a variety of programming languages, and to trigger them using any number of events.
In the case of this lab step, you’ll set up a function app to interact with the Cognitive Services API using Node.js. You’ll then configure the function to be triggered by visiting its URL in a browser tab.
-
On the left side of the blade, click on Functions and then + Add.
-
Click the HTTP Trigger option:
alt
Important: If you don’t see the HTTP trigger option, click More Templates and then Finish and view templates to see the correct option.
- Name the function language-detection and click Create:
alt
-
Click Code + Test in the left sidebar then look at the upper part of the console and switch to the function.json file:
-
Replace the contents of the file with the following snippet and click Save:
{ "bindings": [ { "authLevel": "function", "type": "httpTrigger", "direction": "in", "name": "req", "methods": [ "get", "post" ] }, { "type": "http", "direction": "out", "name": "$return" } ], "disabled": false }
The function.json file manages the behavior of your function app.
-
Switch to the index.js file:
-
Replace the contents of the file with the following code:
'use strict';
let https = require('https');
const subscription_key = "INSERT_YOUR_KEY_HERE";
const endpoint = "INSERT_YOUR_ENDPOINT_HERE";
const path = '/text/analytics/v2.1/languages';
module.exports = async function (context, req) {
let documents = {
'documents': [
{'id': '1', 'text': 'This is a document written in English.'},
{'id': '2', 'text': 'Je suis une phrase écrite en français.'},
{'id': '3', 'text': 'Este es un documento escrito en español.'},
]
};
let body = JSON.stringify(documents);
let request_params = {
method: 'POST',
hostname: (new URL(endpoint)).hostname,
path: path,
headers: {
'Ocp-Apim-Subscription-Key': subscription_key,
}
};
let response = await makeRequest(request_params, body);
context.res = {
body: response
}
};
function makeRequest(options, data) {
return new Promise((resolve, reject) => {
const req = https.request(options, (res) => {
res.setEncoding('utf8');
let responseBody = '';
res.on('data', (chunk) => {
responseBody += chunk;
});
res.on('end', () => {
resolve(JSON.parse(responseBody));
});
});
req.on('error', (err) => {
reject(err);
});
req.write(data)
req.end();
});
}
- Replace the contents of subscription_key and endpoint with the key and endpoint you retrieved in an earlier step, so that the variables look similar to this:
Copy code 1 2 const subscription_key = “4779caba69344c3d97bca9863d726af6”; const endpoint = “https://southcentralus.api.cognitive.microsoft.com/”;
-
Click Save.
-
While a deep understanding of code is outside the scope of this lab, you should take note of a couple of things:
The endpoint and subscription key that you set will allow the function app to communicate with your Cognitive Services API. The endpoint tells the app where to find the API, and the subscription key allows the app to authenticate. The path variable declares the exact path the app should request on the Cognitive Services API. This path system allows the API to offer many different services in one endpoint: Copy code 1 const path = ‘/text/analytics/v2.1/languages’;
The documents variable declares a list of documents that will be passed to the language detection API. The API will return the languages that these documents are most likely written in Currently there are three documents, and the goal will be to get the language used in each one:
let documents = {
'documents': [
{'id': '1', 'text': 'This is a document written in English.'},
{'id': '2', 'text': 'Je suis une phrase écrite en français.'},
{'id': '3', 'text': 'Este es un documento escrito en español.'},
]
};
The rest of the code simply manages the formatting of the data, requesting the language detection service from the API, and returning the result. Next, you’ll visit the function app’s URL to see it work.
-
Look at the upper command bar and click Test/Run.
-
Select GET as Http Method before clicking Run:
alt
- View the output in the Output tab:
alt
Notice that what’s returned is a JSON object with a “documents” object containing three results, one for each language you submitted to it. Each result has a “name” value with the predicted language and a “score” value with the likelihood that the decision is accurate. A score of 1 means that the Cognitive Services API was completely confident in its language detection. Here’s what the formatted JSON object looks like, for reference:
{
"documents": [
{
"id": "1",
"detectedLanguages": [
{
"name": "English",
"iso6391Name": "en",
"score": 1
}
]
},
{
"id": "2",
"detectedLanguages": [
{
"name": "French",
"iso6391Name": "fr",
"score": 1
}
]
},
{
"id": "3",
"detectedLanguages": [
{
"name": "Spanish",
"iso6391Name": "es",
"score": 1
}
]
}
],
"errors": []
}
- Using the Azure Text Analytics API for Language Detection
- Using the Azure Text Analytics API for Sentiment Analysis
Sentiment Analysis
The output is a number from 0 to 1, with 1 being the most positive language and zero being the most negative opinion expressed.
Key phrase extraction
Named Entity Recognition (NER) identifies entities in the text and group them into different entity categories, such as organization name, location, event, etc.
"documents": [
{
"id": "1",
"keyPhrases": [
"world",
"input text"
]
},
Detect Language
vs. Content Moderator - Moderate Detect Language to Auto Correct, PII, listid, classify, language. Classification of Profanity returns JSON with several categories:
- Category 1: sexually explicit or adult in certain situations.
- Category 2: sexually suggestive or mature in certain situations.
- Category 3: considered offensive in certain situations.
Input text is in a formatted JSON document files of up to 5,120 characters. Each file can contain up to 1,000 id’s, each associated with a text string.
As with GraphQL, the API returns the detected language and a numeric score between 0 and 1. Scores close to 1 indicate 100% certainty that the identified language is true. A total of 120 languages are supported.
"id": "3",
"detectedLanguages": [
{
"name": "Spanish",
"iso6391Name": "es",
"score": 1
}
Only one language code is returned for each document submitted. Mixed language content within the same document returns the language with the largest representation in the content, but with a lower positive rating, reflecting the marginal strength of that assessment.
Named Entity Recognition
Sample Response:
{
"documents": [{
"id": "1",
"entities": [{
"name": "Seattle",
"matches": [{
"wikipediaScore": 0.15046201222847677,
"entityTypeScore": 0.80624294281005859,
"text": "Seattle",
"offset": 26,
"length": 7
}],
"wikipediaLanguage": "en",
"wikipediaId": "Seattle",
"wikipediaUrl": "https://en.wikipedia.org/wiki/Seattle",
"bingId": "5fbba6b8-85e1-4d41-9444-d9055436e473",
"type": "Location"
}, {
"name": "last week",
"matches": [{
"entityTypeScore": 0.8,
"text": "last week",
"offset": 34,
"length": 9
}],
"type": "DateTime",
"subType": "DateRange"
}]
}],
"errors": []
}
Linked Entities from a well known knowledge base
Speech Services
Speech-to-text (STT) has two different REST APIs:
-
Speech-to-text REST API v3.0 is used for Batch transcription and Custom Speech.
-
Speech-to-text REST API for short audio is used for online transcription as an alternative to the Speech SDK. Requests using this API can transmit only up to 60 seconds of audio per request.
Microsoft Cognitive Services Speech SDK Samples
LAB:
https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/
https://docs.microsoft.com/en-us/learn/modules/transcribe-speech-input-text/?WT.mc_id=cloudskillschallenge_efc530c5-7105-4c12-8eb3-bc20ae3bee78
Speech Translation (Speech-to-text aka STT)
Get the monthly subscription mobile app on iPhone, Android, or Amazon. It has a Phrasebook of common phrases.
DEMO: Speech Translation recognizes and synthesizes speech, and translates spoken languages. REMEMBER: The sequence of services involves two APIs:
Speech-to-Text API -> Speech Correction -> Machine Translation -> Text-to-Speech API
“Speech Recognition” and Text Analysis are not involved in this use case.
Telephone voice menus use “Speech Synthesis”, defined by the Speech Synthesis Markup Language (SSML).
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/08%20-%20Speech.ipynb
https://github.com/timothywarner/ai100/tree/master/Speech-to-Text
https://github.com/MicrosoftLearning/AI-SpeechToText
The speech-to-text service includes multiple pre-defined voices with support for multiple languages and regional pronunciations, with language detection. In addition to standard voices, neural voices leverage neural networks to overcome common limitations in speech synthesis with regard to intonation, resulting in a more natural sounding voice.
PROTIP: Neural voices are created from samples that use a 24 khz sample rate.
Speech recognition can use a acoustic model of phonemes (sounds) or a language model that matches phonemes with words.
Custom voices can be created with the text-to-speech API.
PROTIP: Since you have to use your own subscription to follow this tutorial from Microsoft, skip clicking “Launch VM mode” and follow the Python notebook on Speech on the regular Portal.
SAMPLE: https://docs.microsoft.com/en-us/samples/azure-samples/cognitive-speech-tts/azure-cognitive-tts-samples/
-
PROTIP: In a CLI window, run my Bash shell script to Create a Cognitive Services resource and get its two keys:
cd ~/clouddrive/azure-quickly git pull ./az-cog-cli.sh
To synthesize speech, the system typically tokenizes the text to break it down into individual words, and assigns phonetic sounds to each word. It then breaks the phonetic transcription into prosodic units (such as phrases, clauses, or sentences) to create phonemes that will be converted to audio format. These phonemes are then synthesized as audio by applying a voice, which will determine parameters such as pitch and timbre; and generating an audio wave form that can be output to a speaker or written to a file.
To specify that the speech input to be transcribed to text is in an audio file, use AudioConfig.
Change the voice used in speech synthesis by setting the SpeechSynthesisVoiceName property of the SpeechConfig object to the desired voice name.
### Translator Text (text-to-text aka TTT)
Microsoft’s Translator service can translate text between more than 90 languages and dialects (including Klingon in Star Trek), specified using ISO 639-1 two-letter language codes and 3166-1 cultural codes such as “en-US” for US English, “en-GB” for British English, “fr-CA” for Canadian French, etc.
- Raza Salehi’s Translator
Hands-on tool without a compute instance:
-
Click on “Start conversation”, log in and enter your name and language.
-
Share the conversation code with other participants, who can join using the Micreosoft Translator app or website.
-
Speak or type in your language to communicate with other participants in the conversation. Other participants will see your messages in their own language.
https://docs.microsoft.com/en-us/azure/cognitive-services/translator/custom-translator/overview What is Custom Translator?
Foe Parallel Data, equivalent documents in different languages:
- https://docs.microsoft.com/en-us/azure/cognitive-services/
- https://docs.microsoft.com/ja-jp/azure/cognitive-services/
- https://docs.microsoft.com/ko-ko/azure/cognitive-services/
https://docs.microsoft.com/en-us/azure/cognitive-services/translator/custom-translator/how-to-upload-document
09 - Translation.ipynb
BLAH: You are asked to use your own Subscription anyway, so instead of the Exercise - Translate text and speech, use portal.azure.com directly.
A Python program can run from your laptop or mobile phone making API calls to the Translator endpoint at:
https://api.cognitive.microsofttranslator.com/translate?api-version=3.0
VIDEO; Raza Salehi’s 1 hr video course “Build a Translator system”.
VIDEO intro with sample code at https://github.com/microsoft/text-analytics-walkthrough
For response “script” : “Latn”, text was transliterated in English.
A custom translator is needed to train a model to recognize and translate domain-specific words and phrases in specific industries such as aerospace, automotive, chemistry, mechanical, etc.
portal.customtranslator.azure.ai
Training is done by have pairs of documents (English and French, etc.).
10,000 aligned parallel sentences are neede to train a translator.
In addition to Microsoft Office formats, files with extension .ALIGN for parellel languages are perfectly aligned. Translation Memory systems can export parallel documents in XLF, XLIFF, TMX, suffix. Microsoft’s LocStudio files have .LCL suffix.
Translation runs can each take several hours. So batch processing is supported.
If you don’t have admin
Need admin approval Mt Studio Web Prod Mt Studio Web Prod needs permission to access resources in your organization that only an admin can grant. Please ask an admin to grant permission to this app before you can use it.
References:
- Microsoft Azure Cognitive Services: Translator Text API” video course</a> by William Myers
Immersive Reader
- Raza Salehi’s Immersive Reader
Other sample code
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-python/documenttranslation-samples/
Vision services
App for the blind: VIDEO: INTRO: SeeingAI.com. Permissions for the “See It All” app are for its internal name “Mt Studio Web Prod”.
https://docs.microsoft.com/en-us/learn/paths/explore-computer-vision-microsoft-azure
HISTORY: In 2014, Microsoft showed off its facial recognition capabilities with how-old.net to guess how old someone is. At conferences they built a booth that takes a picture.
LEARN: https://docs.microsoft.com/en-us/learn/modules/read-text-computer-vision/
DEMO: Seeing AI app talking camera narrates the world around blind people.
- Semantic segmentation is the ML technique which individual pixels in the image are classified according to the object to which they belong.
-
Image analysis
- Face detection, analysis, and recognition
- Optical character recognition (OCR) for small amounts of text
- The Read API works asynchronously on images with a lot of text, to parse pages, lines, and words.
- Video Indexer service analyzes the visual and audio channels of a video, and indexes its content.
Computer Vision
Computer Vision” analyzes images and video to extract descriptions, tags, objects, and text. API Reference, DOCS, INTRO:
- Read the text in the image
- Detects Objects
- Identifies Landmarks
- Categorize image
https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/quickstarts-sdk/client-library?tabs=visual-studio&pivots=programming-language-csharp READ
Computer Vision demo
-
Select images and review the information returned by the Azure Computer Vision web service:
DEMO: https://aidemos.microsoft.com/computer-vision
-
Click an image to see results of “Analyze and describe images”. Objects are returned with a bounding box to indicate their location within the image.
- Click “Try another image” for another selection.
- Click “Next step”.
- Read text in imagery.
- Read handwriting
- Recognize celebrities & landmarks - the service has a specialized domain model trained to identify thousands of well-known celebrities from the worlds of sports, entertainment, and business. The “Landmarks” model can identify famous landmarks, such as the Taj Mahal and the Statue of Liberty.
Additionally, the Computer Vision service can:
- Detect image types - for example, identifying clip art images or line drawings.
- Detect image color schemes - specifically, identifying the dominant foreground, background, and overall colors in an image.
- Generate thumbnails - creating small versions of images.
- Moderate content - detecting images that contain adult content or depict violent, gory scenes.
Computer Vision API shows all the features.
-
- Right-click “Launch VM mode” for the “AI-900” lab on a Window VM.
- X: Click the Edge browser icon.
- X: Click to remove pop-ups.
- Go to portal.azure.com
- Sign in using an email which you have an Azure subscription.
- Type the password. You can’t copy outside the VM and paste into it.
- X: Do not save your password.
- Open Visual Studio to see
-
X: On another browser tab, view the repo (faster):
-
Follow the instructions in the notebook to create a resource, etc.
TODO: Incorporate the code and put it in a pipeline that minimizes manual actions.
- To take a quiz and get credit, click in the VM “here to complete the Learn module with a Knowledge Check.
Custom Vision
Azure Custom Vision trains custom models referencing custom (your own) images. Custom vision has two project types:
-
Image classification is a machine-learning based form of computer vision in which a model is trained to categorize images based on their (class or) primary subject matter they contain.
-
Object detection goes further than classification to identify the “class” of individual objects within the image, and to return the coordinates of a bounding box that indicates the object’s location.
LAB: Steps:
- Create state-of-the-art computer vision models.
- Upload and tag training images.
- Train classifiers for active learning.
- Perform image prediction to identify probable matches to a trained model.
-
Perform object detection to locate elements within an image and return a bounding box.
-
Open
DOCS:
-
MS LEARN HANDS-ON LAB:
aka.ms/learn-image-classification which redirects to
docs.microsoft.com/en-us/learn/modules/classify-images-custom-vision -
Load the code from:
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/03%20-%20Object%20Detection.ipynb
https://docs.microsoft.com/en-us/learn/modules/evaluate-requirements-for-custom-computer-vision-api/3-investigate-service-authorization Custom Vision APIs use two subscription keys, each control access to an API:
- A training key to access API members which train the model.
- A prediction key to access API members which classify images against a trained model.
https://docs.microsoft.com/en-us/learn/modules/evaluate-requirements-for-custom-computer-vision-api/4-examine-the-custom-vision-prediction-api
References: CV API
Ink Recognizer
Ink converts handwriting to plain text, in 63+ core languages.
It was deprecated on 31 January 2021.
https://docs.microsoft.com/en-us/azure/cognitive-services/ink-recognizer/quickstarts/csharp Quickstart: Recognize digital ink with the Ink Recognizer REST API and C#
QUESTION: Does it integrate with a tablet?
Video Indexer
Media Services & Storage Account:
-
In Portal, Media Services blade.
-
Specify Resource Group, account name, storage account, System-managed identity.
Video Indexer
-
In a browser, go to the Video Indexer Portal URL:
NOTE: Video Indexer is under Media Services rather than Cognitive Services.
-
Click the provider to login: AAD (Entra) account, Personal Microsoft account, Google.
PROTIP: Avoid using Google due to the permissions you’re asked to give:
-
Say Yes to Video Indexer permission to: Access your email addresses & View your profile info and contact list, including your name, gender, display picture, contacts, and friends.
NOTE: You’ll get an email with subject “Your subscription to the Video Indexer API”.
On your mobile phone you’ll get a “Connected to new app” notice for Microsoft Authenticator.
-
Click “Account settings”. PRICING: up to 10 hours (600 minutes) of free indexing to website users and up to 40 hours (2,400 minutes) of free indexing to API users. Media reserved units are pre-paid. See FAQ
-
Switch to the file which defines Azure environment variable VIDEO_INDEXER_ACCOUNT (in setmem.sh) as described in
https://wilsonmar.github.io/azure-quickly
- Switch back.
- In Account settings, click “Copy” to get the Account ID GUID in your Clipboard.
-
Switch to the file which defines Azure environment variable VIDEO_INDEXER_ACCOUNT (in setmem.sh) as described in
https://wilsonmar.github.io/azure-quickly
- Highlight the sample value and paste (Command+V).
-
Switch back.
-
Go to the “Azure Video Analyzer for Media Developer Portal”:DOCS
-
Click “Sign In”. Click “Profile”
NOTE: The UI has changed since publication of Microsoft’s tutorial, which says “Go to the Products tab, then select Authorization.”
- Click “Show” on the “Primary key” line. Double-click on the subscription key to copy to Clipboard (Command+C).
-
Switch to the file which defines Azure environment variable VIDEO_INDEXER_API_KEY (in setmem.sh) as described in
https://wilsonmar.github.io/azure-quickly
- Highlight the sample value and paste (Command+V).
-
Switch back.
Upload video using Portal GUI
Upload video using program
https://api-portal.videoindexer.ai/api-details#api=Operations&operation=Get-Account-Access-Token
-
DOCS: Select the Azure Video Indexer option for uploading videos: upload from URL (there is also send file as byte array by an API call, which has limits of 2 GB in size and a 30-minute timeout.
az-video-upload.py in https://wilsonmar.github.io/azure-quickly
-
Make an additional call to retrieve insights.
-
Reference existing asset ID
Search Media files
-
In “Media files” at https://www.videoindexer.ai/media/library
-
Click “Samples”, and click on a video file to Play to see the media’s people, topics (keywords).
NOTE: Search results include exact start times where an insight exists, possibly multiple matches for the same video if multiple segments are matched.
-
Click a tag to see where it was mentioned in the timeline.
Alternately, use the API to search: ???
Model customizations
Each video consists of scenes grouping shots, which each contain keyframes.
-
A scene represents a single event within the video. It groups consecutive shots that are related. It will have a start time, end time, and thumbnail (first keyframe in the scene).
-
A shot represents a continuous segment of the video. Transitions within the video are detected which determine how it is split into shots. Shots have a start time, end time, and list of keyframes.
-
Keyframes are frames that represent the shot. Each one is for a specific point in time. There can be gaps in time between keyframes but together they are representative of the shot. Each keyframe can be downloaded as a high-resolution image.
-
-
Click “Model customizations”
- Set as thumbnail another keyframe.
https://github.com/Azure-Samples/media-services-video-indexer
https://dev.to/adbertram/getting-started-with-azure-video-indexer-and-powershell-3i32
Azure Face
Azure “Face” is used to build face detection and facial recognition solutions in five categories:
- Face Verification: Check the likelihood that two faces belong to the same person.
- Face Detection: Detect human faces in an image.
- Face Identification: Search and identify faces.
- Face Similarity: Find similar faces.
- Face Grouping: Organize unidentified faces into (face list) groups, based on their visual similarity.
NOTE: On June 11, 2020, Microsoft announced that it will not sell facial recognition technology to police departments in the United States until strong regulation, grounded in human rights, has been enacted. As such, customers may not use facial recognition features or functionality included in Azure Services, such as Face or Video Indexer, if a customer is, or is allowing use of such services by or for, a police department in the United States.
A face location is face coordinates – a rectangular pixel area in the image where a face has been identified.
The Face API can return up to 27 landmarks for each identified face that you can use for analysis. Azure allows a person to can have up to 248 faces. There is a 6 MB limit on the size of each file (jpeg, png, gif, bmp).
Face attributes are predefined properties of a face or a person represented by a face. The Face API can optionally identify and return the following types of attributes for a detected face:
- Age
- Gender
- Smile intensity
- Facial hair
- Head pose (3D)
- Emotion
Emotions detected in JSON response is a floating point number:
- Neutral
- Anger
- Contempt
- Disgust
- Fear
- Hppiness
- Sadness
- Surprise
PROTIP: “Happiness: 9.99983543,” is near certainty at 1.0. 2.80234E-08” indicates 8
https://github.com/Azure-Samples/cognitive-services-FaceAPIEnrollmentSample
DEMO: LAB: https://github.com/microsoft/hackwithazure/tree/master/workshops/web-ai-happy-sad-angry
Create a Face API subscription
Subscribe to the Face API:
- Sign in to the Azure portal.
- Go to
- Create a resource > AI + Machine Learning > Face</a>
-
Enter a unique name for your Face API subscription name in variable MY_FACE_ACCT
Paste in setme.sh
export MY_FACE_ACCT=faceme
- Choose “westus”, the Location nearest to you.
- Select F0 the free or lowest-cost Pricing tier.
- Check “By checking this box, I certify that use of this service or any service that is being created by this Subscription Id, is not by or for a police department in the United States.”
-
Click “Create” to subscribe to the Face API.
- When provisioned, “Go to resource”.
-
In “Keys and Endpoint”, copy Key1 and paste in setme.sh
export MY_FACE_KEY1=subscription_key
The endpoint used to make REST calls is “$MY_FACE_ACCT.cognitiveservices.azure.com/”
https://github.com/wilsonmar/azure-quickly/blob/main/az-face-init.sh
https://docs.microsoft.com/en-us/learn/modules/identify-faces-with-computer-vision/8-test-face-detection?pivots=csharp
References:
https://docs.microsoft.com/en-us/azure/cognitive-services/Face/Overview What is the Azure Face service?
https://docs.microsoft.com/en-us/azure/cognitive-services/Face/
https://docs.microsoft.com/en-us/azure/cognitive-services/Face/quickstarts/client-libraries?tabs=visual-studio&pivots=programming-language-csharp
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/04%20-%20Face%20Analysis.ipynb
Azure Form Recognizer
“Form Recognizer” extracts information from images obtained from scanned forms and invoices.
- Raza Salehi’s Recognizer
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/06%20-%20Receipts%20with%20Form%20Recognizer.ipynb
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-python/tables-samples/
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-java/formrecognizer-java-samples/
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-net/azure-form-recognizer-client-sdk-samples/
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-python/formrecognizer-samples/
OCR
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/05%20-%20Optical%20Character%20Recognition.ipynb
-
Image classification - https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/01%20-%20Image%20Analysis%20with%20Computer%20Vision.ipynb
-
Object detection - https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/02%20-%20Image%20Classification.ipynb
Conversational AI
HISTORY: In 2015, Microsoft unleashed the Tay chat bot, then had to bring it down after hackers submitted enough racial slurs that they fooled the system into thinking that was normal and acceptable.
HISTORY: XiaoIce, a chatbot Microsoft launched in China, “has more than 200 million users, has engaged in 30 billion conversations, and has an average conversation length of 23 turns, which averages out to about half an hour, achieving human parity at translation from Chinese to English. Japan-based Rinna and the US-based Zo)
A Bot Framework enables the creation of Virtual Assistant
A LUIS app creates these types of entities:
- Machine-learned
- List
- RegEx
- Pattern.any
https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/concepts/plan?tabs=v1
Bots are extended by Skills
QnA Maker
The cognitive service name “QnA Maker” (Question and Answer Maker)</a> is a cloud-based API service that lets you create a conversational question-and-answer layer over your existing data. The service enables the building of knowledge bases of questions and answers that form the basis of a dialog between a human and an AI agent.
Microsoft created the QnA Maker portal to make it easier than writing code to create and manage knowledge bases using the QnA Maker REST API or SDK.
The knowledge base gets smarter as it continually learns from user behavior.
The knowledge base can be built by extracting questions and answers from your semi-structured content, including FAQs, manuals, and documents.
QnA Maker limits control the size of Knowledge base.
### Create QnA Service
https://github.com/Microsoft/BotBuilder-CognitiveServices/tree/master/CSharp/Samples/QnAMaker is only for C#.
-
View the DOCS:
View the v2 (previous release)
-
The Jupyter notebook:
https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/11%20-%20QnA%20Bot.ipynb
- On an appropriate browser profile (personal or work email) associated with the subscription you wnat to use.
-
Go to the QnA Maker portal at:
-
Sign in.
https://docs.microsoft.com/en-us/learn/paths/explore-conversational-ai/
- “Create a knowledge base” from the top menu.
- STEP 1: “Create a QnA service”, which opens another tab
-
Basics: During testing, do NOT click the checkbox for “Managed”. In prod, telemetry and compute are included automatically with your QnA Maker resource. If you do not select managed, you will be prompted to create an App Insights and App Service resources for the required telemetry and compute that you will have to manage for your QnA Maker resource. Read more <a target=”_blank” href=”“https://aka.ms/qnamaker-createoptions-description”>here</a>.
- Subscription: Your Azure subscription
- Resource group: Select an existing resource group or create a new one
-
Resource group location: “(US) West US”
- Name: A unique name for your QnA resource (App name)
- Pricing tier: both are “3 transactions per second, 100 transactions per minute”:
- Free F0 (3 managed documents per month, 5…)
- Standard S0 ($10 per month for unlimited documents …)
App Service details - for runtime :
- Azure Search location: Any available location
- Azure Search pricing tier: F
- Free F (50 MB in 3 indexes)
- Basic B (2 GB in 15 indexes, 3 scale-out units per service on 1 partition with 3 replicas)
- Standard S1 (25 GB in 50 indexes) - default
- Standard S2 (100 GB in 200 indexes)
- Standard S3 (200 GB in 200 indexes)
- Standard S3 (1000 Index/Partition)
- L1 (1 TB in 10 indexes, High Storage, 36 units)
- L2 (2 TB in 10 indexes, High Storage, 36 units)
Note: If you have already provisioned a free-tier QnA Maker or Azure Search resources, your quota may not allow you to create another one. In which case, select a tier other than F0 / F.
App Service details - for runtime :
- App name: Same as Name (“.azurewebsites.net” will be appended automatically)
-
Website location: Same as Azure Search location
App insights details - for telemetry and chat logs :
-
App insights: Disable, which will hide the “App insights location”, but appear in Review.
- Click “Review + create”. Create. NOTE: when these resources complete deployment:
- Microsoft.Web/sites/config
- Microsoft.CognitiveServices/accounts
- Microsoft.Search/searchServices
- microsoft.insights/components
- microsoft.insights/components
- Microsoft.Web/sites
- Microsoft.Search/searchServices
- Microsoft.Web/serverfarms
-
While you wait for the dots to stop flashing “Deployment in progress”, return to the QnA Maker portal tab. You may have timed out.
-
When “Your deployment is complete”, click “Go to resources” for “Congratulations! Your keys are ready.”
Connect QnA service to KB
STEP 2: Connect your QnA service to your KB.
- “Refresh” the list of available QnA service resources.
- Microsoft Azure Directory ID: The Azure directory ID for your subscription
- Azure subscription name: Your Azure subscription
-
Azure QnA service: The QnA service resource you created in the previous step
NOTE: In the Preview there is a checkbox “Enable language setting per knowledge base”.
-
Language: English
STEP 3: Name your KB.
-
Type a name: For example: “Margie’s Travel KB”. Spaces are allowed?
STEP 4: Populate your KB.
- ”+ Add URL”
-
Copy and paste this example URL:
https://github.com/MicrosoftDocs/ai-fundamentals/raw/master/data/qna_bot/margies_faq.docx
-
Add file
- chit-chat: Adding “chit-chat” to your knowledge base (by selecting a personality) automatically adds questions and responses to your knowledge base, which enables your bot to answer small-talk questions in a voice (personality) of your choice:
- None
- Professional « select this.
- Friendly
- Witty
- Caring
- Enthusiastic
QUESTION: What is the range of popularity?
QUESTION: Extraction? I’m stuck here.
-
Do NOT check “Enable multi-turn extraction from URLs, .pdf or .docx files.”
-
Click “Create your KB”. Wait for a minute or so while your Knowledge base is created.
-
Review the questions and answers that have been imported from the FAQ document and the professional chit-chat pre-defined responses.
Test the knowledge base
Make an API call
https://go.microsoft.com/fwlink/?linkid=2100125
https://go.microsoft.com/fwlink/?linkid=2100213 Coding</a>
Azure Bot Service
provides a platform for creating, publishing, and managing bots. Developers can use the Bot Framework to create a bot and manage it with Azure Bot Service - integrating back-end services like QnA Maker and LUIS, and connecting to channels for web chat, email, Microsoft Teams, and others.
Microsoft Bot Framework supports two approaches to integrate bots with agent engagement platforms such as Customer support service:
- Bot as agent distributes calls to bots on the same level as live (human) agents. Handoff protocols regulate a bot’s disengagement and transfer to live person.
- Bot as proxy presents a bot to filter interaction with live people.
-
DEMO: See a healthcare bot built using the Azure Bot Service:
<a target=”_blank”” href=” https://www.microsoft.com/research/project/health-bot/”> https://www.microsoft.com/research/project/health-bot</a>
Select the option to Try a demo of an example end-user experience. Use the web chat interface to interact with the bot.
MS LEARN: Create a Bot with the Bot Framework Composer
-
Run the Python Jupyter notebook
-
Sign in using the Microsoft account associated with your Azure subscription.
PROTIP: Use NVM to install Node
https://github.com/Microsoft/botbuilder-tools#install-cli-tools says to install Node.js version 10.14.1 or higher
https://github.com/microsoft/botframework-cli says to install Node.js version 12
Since the current version is now 16, we cannot use the command suggested in the doc:
npm i -g @microsoft/botframework-cli
Bot Framework
-
bf is the bot framework CLI command
One-stop-shop CLI to manage your bot’s resources. BF CLI and AZ CLI together cover your end-to-end bot development workflow needs. VERSION @microsoft/botframework-cli/4.13.3 darwin-x64 node-v16.1.0 USAGE $ bf [COMMAND] COMMANDS chatdown Converts chat dialog files in <filename>.chat format into transcript files. Writes corresponding <filename>.transcript for each .chat file. config Configure various settings within the cli. dialog Dialog related commands for working with .schema and .dialog files. help display help for bf lg Parse, collate, expand and translate lg files. luis Manages LUIS assets on service and/or locally. orchestrator Display Orchestrator CLI available commands plugins Install, uninstall and show installed plugins qnamaker QnA Maker
References:
Knowledge mining = Search?
Knowledge Mining Solution Accelerator
Search = Bing ?
-
Get on the Bing Resource portal GUI.
-
Define a Resource Group.
-
Price Tier: Free
PROTIP: Autosuggest requires the “S2” (Standard) pricing tier. Spell Check requires either S1 or S2.
- Click “Automation Options” link for a JSON Template.
- Click “Create” at the top.
-
Create
Notice the service name at the upper left is “Microsoft.BingSearch” and has a Global location. It’s Endpoint is: https://api.bing.microsoft.com/
Search for AI enrichment of docs
The Azure Cognitive Search service uses a Cognitive Search resource to support AI-powered search and knowledge mining solutions such as:
- Index documents and data from a range of sources.
- Use cognitive skills to enrich index data.
- Store extracted insights in a knowledge store for analysis and integration.
https://blog.api.rakuten.net/top-10-best-search-apis/
“Document cracking” during indexing extracts text content from unstructured text or non-text content (such as images, scanned documents, or JPEG files). The indexer accesses an Azure data storage service.
https://blog.scottlowe.org/2019/03/01/advanced-ami-filtering-with-jmespath/
Speech-to-text
Transcriptions can be done in real-time or in batch mode.
Batch mode is when audio recordings are stored on a file share, and a shared access signature (SAS) URI is used by a program to asynchronously receive transcription results.
-
Take the introductory tutorial:
Introduction to Machine Learning with Hands-On Labs
https://azure.microsoft.com/en-us/documentation/articles/machine-learning-studio-overview-diagram
-
Create a model.
-
Prepare Data:
As per this video using
- Clean Missing Data - Clip Outliers
- Edit Metadata
- Feature Selection
- Filter
- Learning with Counts
- Normalize Data
- Partition and Sample
- Principal Component Analysis
- Quantize Data
- SQLite Transformation
- Synthetic Minority Oversampling Technique
-
Train the model
- Cross Validation
- Retraining
- Parameter Sweep
-
Score and test the model.
-
Make predictions with Elastic APIs
- Request-Response Service (RRS) Predictive Experiment - Batch Execution Service (BES)
- Retraining API
https://github.com/timothywarner/ai100/tree/master/Speech-to-Text
Anomaly Detector
- Raza Salehi’s Anomaly Detector
https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-net/azure-anomaly-detector-client-sdk-samples/
Among Anomaly Detector API Samples
Anomaly Detection
Anomaly Detector identifies potential problems early on.
https://docs.microsoft.com/en-us/learn/modules/get-started-ai-fundamentals/3-understand-anomaly-detection
- Sensors in the car collect telemetry, such as engine revolutions, brake temperature, and so on.
- An anomaly detection model is trained to understand expected fluctuations in the telemetry measurements over time.
- If a measurement occurs outside of the normal expected range, the model reports an anomaly that can be used to alert the race engineer to call the driver in for a pit stop to fix the issue before it forces retirement from the race.
Data Science VM
https://docs.microsoft.com/en-us/azure/machine-learning/data-science-virtual-machine/overview#whats-included-in-the-data-science-vm
Resources
- Azure AI product page
- Azure case studies
- Microsoft Azure Blog
- Channel 9 AI Show
- AI for Good
- Seeing AI application
- The Future Computed book
Microsoft AI ML Community in Signapore
If you have an OReilly.com account:
On Udemy:
Notes to be inserted
Steps for data transformation:
- Feature selection
- Finding and removing data outliers
- Impute missing values
- Normalize numeric features
https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/limits-and-quotas
Microsoft’s History with AI
In April 2018 Microsoft reorganized into two divisions that offers AI:
-
The research division, headed by Harry Shum, put AI into Bing search, Cortana voice recognition and text-to-speech, ambient computing, and robotics. See Harry’s presentation in 2016.
-
Microsft’s “computing fabric” offerings, led by Scott Guthrie, makes AI services available for those building customizable machine learning with speech, language, vision, and knowledge services. Tools offered include Cognitive Services and Bot Framework, deep-learning tools like Azure Machine Learning, Visual Studio Code Tools for AI, and Cognitive Toolkit.
At Build 2018, Microsoft announced Project Brainwave to run Google’s Tensorflow AI code and Facebook’s Caffe2, plus Microsoft’s own “Cognitive Toolkit” (CNTK).
-
BrainScript uses a dynamically typed C-like syntax to express neural networks in a way that looks like math formulas. Brainscript has a Performance Profiler.
-
Hyper-parameters are a separate module (alongside Network and reader) to perform SGD (stochastic-gradient descent).
Microsoft has advanced hardware:
- [
This pdf white paper says the "high-performance, precision-adaptable FPGA soft processor is at the heart of the system, achieving up to 39.5 TFLOPs of effective performance at Batch 1 on a state-of-the-art Intel Stratix 10 FPGA."
Microsoft's use of field programmable gate arrays (FPGA) calculates AI reportedly "five times faster than Google's TPU hardware".
"Each FPGA operates in-line between the server’s network interface card (NIC) and the top-of-rack (TOR) switch, enabling in-situ processing of network packets and point-to-point connectivity between hundreds of thousands of FPGAs at low latency (two microseconds per switch hop, one-way)."
Microsoft Conversational AI Platform for Developers is a 2021 book published by Apress by Stephan Bisser of Siili Solutions in Finland. The book covers Microsoft’s Bot Framework, LUIS, QnA Maker, and Azure Cognitive Services. https://github.com/orgs/BotBuilderCommunity/dashboard
https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/
Additionally…
In July and August Microsoft Reactor ran a “Skills Challenge” to reward a badge for those who complete an AI tutorial.
https://learn.microsoft.com/en-us/training/challenges
- AI Builder
- Machine Learning
- MLOps
- Cognitive
https://www.youtube.com/watch?v=ss-kyogPRNo by Carlotta
More
This is one of a series on AI, Machine Learning, Deep Learning, Robotics, and Analytics:
- AI Ecosystem
- Machine Learning
- Microsoft’s AI
- Microsoft’s Azure Machine Learning Algorithms
- Microsoft’s Azure Machine Learning tutorial
- Python installation
- Image Processing
- Tessaract OCR using OpenCV
- Multiple Regression calculation and visualization using Excel and Machine Learning
- Tableau Data Visualization