transformer, encoder and decoder

 

A transformer architecture has two main segments: an encoder that primarily operates on the input sequence and a decoder that operates on the target sequence during training and predicts the next item

Reference: 

How to build a GPT model (leewayhertz.com)