Encoder Decoder Terminology
Ace your homework & exams now with Quizwiz!
encoder-decoder architecture
A machine that consumes sequences and outputs sequences. Comprised of two stages: Encoding stage and Decoding stage. Each stage can be implemented with a different architecture.
decoder stage
Creates a human readable output (i.e. sentence) from the input vector representation.
BERT
First used in 2018
T5
First used in 2019
GPT-3
First used in 2020
PaLM
First used in 2022
encoder stage
Produces a vector representation of the input sequence.
sequence-to-sequence architecture
Takes a sequence of words as input and outputs a sequence of words.
LLM
abbreviation for Large pre-trained Language Model. First used in 2017.
RNN
abbreviation for Recurrent Neural Network
