Session 6: Generative AI

Generative AI

"Generative artificial intelligence (AI) describes algorithms that can be used to create new content, including audio, code, images, text, simulations, and videos."

Prompt engineering

An AI technique that involves crafting the input instructions to help LLMs understand the task they need to perform and elicit the desired response.

Model Parameters (text)

LLM outputs are affected by the configuration hyperparameters of the model, which control different aspects such as how "random" it is. By adjusting them you can influence the output to be more creative, diverse or interesting which control various aspects of the model, such as how 'random' it is.

Text / LLMS Applications (ChatGPT, Google's Bard)

Large Language Models (LLMs) must be capable of capturing the sequential nature of text data. Transformer-based architectures are particularly effective in this regard due to their ability to account for long-range dependencies within the text data.

Anatomy of a good prompt (text):

Role Task description Ex 1 Ex 2 Ex 3 Context Question

Anatomy of a good prompt (image)

Subject Medium Style Artist Website Resolution Color Additional details

Diffusion Model

a kind of machine learning model that uses a process of gradual change* to turn a simple, known type of random information (like Gaussian noise) into another type of information that we're more interested in (like images or text). This process happens in stages and goes in reverse order, similar to how heat spreads through an object. Once the model is trained, it can make new data that looks like the data it learned from quite precisely.

Transformer-based Model

a type of machine learning model that employs the concepts of self-attention and feedforwarding, meaning they numerically weigh the importance of each element in a sequence in relation to all others. Once a Transformer model is trained, it can understand and generate data that is contextually similar to the training data, maintaining the structural and semantic properties of the input.

Variational autoencoder (VAE)

a type of machine learning model that learns to reproduce its input and map data to latent space*, which contains a compressed representation of the input data. Once trained, VAEs can generate new, synthetic data that is similar to the input data by randomly sampling points from the latent space and decoding them.

Generative adversarial Network (GAN)

a type of machine learning model that uses deep learning techniques to generate new data based on patterns learned from existing data.

Generative Audio Models (GAMs)

are required to capture the time-sequential patterns and rich nuances within audio data. The very last models are using a combination of Transformers-based + Diffusion models with good results.

Video and Audio / GVMs (Meta's make a video, AudioLm)

need to be able to comprehend the underlying temporal dynamics and spatial relationships within video data. Still young technology, but Transformer-based models are emerging as a promising solution.

Images Applications / Generative Image Models (GIMS) (Midjourney, DALL-E 2)

need to be capable of capturing the inherent structure and intricate patterns within image data. They have significantly evolved over time, witnessing a major transition from GANs and VAEs to more recent Diffusion models (that offer a blend of the strengths of GANs and VAEs).