GPT model

Transformer is the predecessor of GPT and BERT. The optimization of Google and OpenAI in natural language processing technology is based on this model. BERT and GPT are very important models in the field of natural language processing in recent years. The difference between the two is that BERT is a bidirectional pre-training language model + fine-tuning (fine-tuning), while GPT is an autoregressive pre-training language model.+Prompting (indication/prompt). Pre-trained language models and self-supervised pre-trained language models, which are then fine-tuned for specific downstream tasks. GPT uses a unidirectional Transformer decoder, while Burt uses a bidirectional Transformer encoder. Both BERT and GPT adopt the "pre-training + fine-tuning" paradigm, and downstream tasks are still classic NLP task forms such as classification, matching, and sequence labeling. This paradigm lays the foundation for natural language processing technology。