AI-900: Intro to Azure OpenAI Service

What can you do in the Chat playground?

You can use the assistant setup to instruct the model about how it should behave. The assistant will mimic the response you include in tone, rules, and format you've defined in your system message

Basic ML process

- Feed data into algorithm - Use this data to train a model - Test and deploy the model - Consume the deployed model to do an automated predictive task

What are the popular artificial neural network typologies?

- Feedforward neural network - Recurrent neural network (RNN) - Convolutional neural network (CNN) - Generative adversarial network (GAN) - Transformers

Capabilities of OpenAI Models

- Generating natural language - Generative code - Generating images

What is Generative adversarial network (GAN)?

- Trained to create realistic content such as images - Made up to two networks known as generator and discriminator - both trained simultaneously - Used to solve problems like image to image translation and age progression

Three groupings of Azure AI's services and tools

1) Azure Machine learning platform 2) Cognitive services 3) Applied AI services

MSFT 6 AI Principles for usage of Azure OpenAI

1) Fairness 2) Reliability and safety 3) Privacy and security 4) Inclusiveness 5) Transparency

Azure & OpenAI's three main goals in partnership

1) Utilize Azure's infrastructure - including security, compliance, and regional availability - to help users build enterprise-grade applications 2) Deploy OpenAI model capabilities across MSFT products, including and beyond Azure AI products 3) To use Azure to power all of OpenAI's workloads

What are OpenAI code generation capabilities?

Code generation AI models are able to take natural language or code snippets and translate them into code. The OpenAI code generation model family, Codex, is proficient in over a dozen languages such as C#, Java, Perl, PHP, and is most capable in Python

What is machine translation?

Deep learning has expanded to two specific areas: - Automatic translation (translation of speech to text) - Automatic translation of images With the appropriate data transformation, a neural network can understand text, audio, and visual signals Can be used to ID snippets of sound in larger audio files and transcribe the spoken word or image as text

What is named-entity recognition?

Method that takes a piece of text as input and transforms it into a pre-specified class. The information can then be stored in a structured schema to build a list or serve as a benchmark (New information could be a postal code, a date, and a product ID)

AI Landscape: Generative AI

Models can produce new content based on what is described in the input. The OpenAI models are a collection of generative AI models that can produce language, code, and images

AI Landscape: Machine Learning

Models take in data, fit data to an algorithm, use framework to make predictions

AI Landscape: Deep Learning

Models use layers of algorithms in the form of artificial neural networks to return results for more complex use cases The learning process is deep because the structure of artificial neural networks consist of multiple input, output, and hidden layers.

Generative Natural Language: Embeddings:

Search, classify, and compare text

What is a Transformer artificial neural network?

- Suited for solving problems and containing sequences such as text or time-series data - Consist of encoder and decoder layers - What makes transformers different from other architectures containing encoders and decoders are the attention sub-layers - Used to solve natural language processing problems such as translation, text generation, question/answering, and text summarization

What is a Recurrent neural network (RNN)?

- Widely used for complex tasks such as time series forecasting, learning handwriting, and recognizing language - Save the output of a layer and feed it back to the input layer to help predict the layer's outcome - Great learning abilities

What 4 components does Azure OpenAI consist of?

1) Pre-trained generative AI models 2) Customization capabilities; the ability to fine-tune AI models with your own data 3) Built-in tools to detect and mitigate harmful use cases/users 4) Enterprise-grade security with role-based access control (RBAC) and private networks

What are the 5 pillars of Cognitive Services?

1) Vision 2) Speech 3) Language 4) Decision 5) Azure OpenAI Service

What is Convolutional neural network (CNN)?

- Layers are organized in three dimension: width, height, and depth - The neurons is one layer connect not to all the neurons in the next layer, but only to a small region of the layer's neurons - The final output is reduced to a single vector of probability score, organized along the depth dimension - Used in areas such as video recognition, image recognition, and recommender systems

Deep learning use cases

- Named-entity recognition - Object detection - Image caption generation - Machine translation - Text analytics

Describe Azure OpenAI

Azure OpenAI service is a product offered by MSFT Azure that provides REST API access to OpenAI's language model including GPT-4, GPT-3, Codex, and DALL-E models. It allows customers to apply large language models and generative AI to a variety of use cases. Runs on Azure global infrastructure - providing Azure security, compliance, and regional availability - Also provides built-in features to ensure you're using AI responsibly

What is Image Variation?

Can be created by providing an image and specifying how may variations of the image you would like. General content

What is the benefit of code generation?

Code generation can help devs code faster, understand new languages, and focus on solving bigger problems in the application. Devs can break down their goal into simpler tasks and use Codex to help build out those tasks using known patterns

What is Codex?

Codex models are based off of GPT-3 and optimized to understand and write code. Models have been trained on both natural language and billions of lines of code from public repositories It's able to generate code from natural language instructions such as code comments, and can suggest ways to complete code functions

What is object detection?

Comprised of two parts: 1) Image classification: IDs the image's objects 2) Image localization: Provides specific location of the objects Already used in industries such as gaming, retail, tourism, and self-driving cars

What families of models are within Azure OpenAI?

GPT-4 GPT-3 Codex Embeddings DALL-E **All models can be trained and customized

What is Generative AI?

Generative AI is a type of AI that can generate text, images, or other media in response to prompts. GAI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics, typically using Transformer-based deep neural networks. Potential applications in every industry. Concerns of misuse in the form of deep fakes, fake news

What is important to consider when making business decisions around which model to use?

How time and compute needs factor into machine learning training The 'learning' portion of training requires a computer to ID an algorithm that best fits the data The complexity of the task the model needs to solve for and desired level of model performance all factor into the time required to run through possible solutions for a best-fit algorithm

What are OpenAI's image generation capabilities?

Image generation models can take a prompt, a base image, or both and create something new - either realistic or artistic

AI Landscape: Artificial Intelligence

Imitates human behavior by relying on machines to learn and execute tasks without explicit direction on what to output

Deep learning vs Machine learning

In machine learning, the algorithm needs to be told how to make an accurate prediction by consuming more information In deep learning, the algorithm can learn how to make an accurate prediction through its own data processing, thanks to the artificial neural network structure

What is Azure OpenAI Studio?

In the Azure OpenAI Studio, you can build AI models and deploy them for public consumption in software applications - made possible by specific generative AI models, optimized for different tasks; some models excel at summarization and providing unstructured responses, others are built to generate code or unique images from text input

What is a playground?

In the Azure OpenAI Studio, you can experiment with OpenAI models in playgrounds

What is text analytics?

Involves analyzing large quantities of text data, recognizing patterns, and creating organized and concise information out of it Companies can use deep learning to perform text analysis to detect insider trading and compliance with gov't regulations

How does GPT model for natural language generation work?

It takes an input, or prompt, to return a natural language, visual, or code response. GPT tries to infer, or guess, the context of the user's question based on the prompt

Deep learning requirements

Number of data points: Needs to use large amounts of training data to make predictions Hardware dependencies: Depends on high-end machines. It inherently does a large number of matrix multiplication operations. A GPD can efficiently optimize these operations Featurization process: Learns high-level features from data and creates new features by itself Learning approach: Moves through the learning process by resolving the problem on an end-to-end basis Execution time: Usually takes a long time to train because a deep learning algorithm involves many layers Output: The output can have multiple formats, like a text, a score or a sound

Machine learning requirements

Number of data points: Small amounts of data to make predictions Hardware dependencies: Can work on low-end machines, does not need a large amount of computational power Featurization process: Requires features to be accurately ID'd and created by users Learning approach: Divides the learning process into small steps then combine the results from each step into one output Execution time: Takes comparatively little time to train, ranging from a few seconds to a few hours Output: The output is usually a numerical value, like a score or classification

What are GPT models best used for?

Summarizing text Classifying text Generating names or phrases Translation Answering questions Suggesting content

Describe Azure OpenAI's access and responsible AI policies

Teams and individuals tasked with developing and deploying AI systems should work to identify, measure, and mitigate considerations for safe and fair use

How are OpenAI's natural language capabilities trained?

Trained on words or chunks of characters known as tokens - Ex: "Hamburger" gets broken up into the tokens ham, bur, and ger, which a short and common word like 'pear' is a single token - Tokens are mapped into vectors for a machine learning model to use for training. When a trained natural language model takes in a user's input, it also breaks down the input into tokens

What is Editing an image?

When provided an image, DALL-E can edit the image as requested by changing its style, adding or removing item, or generating new content to add Edits are made by uploading the original image and specifying a transparent mask that indicates what area of the image to edit. Along with the image and mask, a prompt indicating what is to be edited instructs the model to then generate the appropriate content to fill the area

What is a feedforward neural network?

- The most simple type of artificial neural network - Information moves in only one direction from input layer to output layer - Feedforward neural networks transform an input by putting it through a series of hidden layers (Every later is made up of a set of neurons, and each layer is fully connected to all neurons in the layers before) (The last fully connected layer (output layer) represented the generated predictions

When do you use Azure's existing language service and when do you use Azure OpenAI's service?

There's no strict guidance - Azure's existing language service can be used for widely known use-cases that require minimal tuning Azure OpenAI's service may be more beneficial for use-cases that require highly customizable generative models, or exploratory research

What are some of the overlapping features shared by Cognitive Service's Language service and OpenAI's service?

Translation, sentiment analysis, and keyword extraction

What can you do in the Completions playground?

Type prompts, configure parameters, and see responses without having to code

Image capabilities generally fall into what three categories?

1) Image generation 2) Editing an image 3) Image variation

What is transfer learning?

A technique that applies knowledge gained from solving one problem to a different but related problem By repurposing the domain or problem, you can significantly reduce the amount of time, data, and compute resources needed to train the new model

What does ChatGPT portal consist of?

ChatGPT's portal consists of the front-end user interface (UI) users see, and a back-end that includes a generative AI model - the combination can be described as a chatbot The model provided on the back end is what is available as a building block with both the OpenAI API and Azure OpenAI API

Generative Natural Language: Text Completion:

Generate and edit text

What is Dall-E?

In addition to natural language capabilities, generative AI models can edit and create images. Model that works with images is called Dall-E. NOTE: Access to Dall-e is currently invite only

What is Image Generation?

Original images can be generated by providing a text prompt of what you would like the image to be of simpler while still fulfilling what is asked for

How to you use Azure OpenAI?

Apply for access, then use the service by creating an Azure OpenAI resource - like you would in other Azure services. Once created, you can use the service through REST APIs, Python SDK, or web-based interface in the Azure OpenAI Studio

What are some well-known implementations of transformers?

- Bidirectional encoder representations for transforms (BERT) - Generative pre-trained transformer 2 (GPT-2) - Generative pre-trained transformer 3 (GPT-3)

What is the training process of Generative adversarial network (GAN)?

- Generator uses random noise to create new synthetic data that closely resembles real data - The discriminator takes the output from the generator as input and uses real data to determine whether the generated content is real or synthetic - Each network is competing with each other - the generator is trying to generate synthetic content that is indistinguishable from real content and the discriminator is trying to correctly classify inputs as real or synthetic - The output then used to update the weights of both networks to help them better achieve their respective goals