Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Generative pre-trained transformer
open-in-new
<p>A generative pre-trained transformer (GPT) is a type of <a href="/facts/Large_language_model/WnogWVJY">large language model</a> (LLM) and a prominent framework for <a href="/facts/Generative_artificial_intelligence/ykT3GGyT">generative artificial intelligence</a>. It is an <a href="/facts/Neural_network_(machine_learning)/6V1jMlkx">artificial neural network</a> that is used in <a href="/facts/Natural_language_processing/1hjMKsSN">natural language processing</a> by machines. It is based on the <a href="/facts/Transformer_(deep_learning_architecture)/cDbjx6a8">transformer deep learning architecture</a>, pre-trained on large <a href="/facts/Data_set/eDvegUTK">data sets</a> of unlabeled text, and able to generate novel human-like content. As of 2023, most LLMs had these characteristics and are sometimes referred to broadly as GPTs. </p><p>The first GPT was introduced in 2018 by <a href="/facts/OpenAI/V7WVK1t4">OpenAI</a>. OpenAI has released significant GPT foundation models that have been sequentially numbered, to comprise its "GPT-<i>n</i>" series. Each of these was significantly more capable than the previous, due to increased size (number of trainable parameters) and training. The most recent of these, <a href="/facts/GPT-4o/6WnEYHRa">GPT-4o</a>, was released in May 2024. Such models have been the basis for their more task-specific GPT systems, including models <a href="/facts/Instruction_tuning/WnogWVJY">fine-tuned for instruction following</a>—which in turn power the <a href="/facts/ChatGPT/ptB1y6J5">ChatGPT</a> <a href="/facts/Chatbot/kASkMjs7">chatbot</a> service. </p><p>The term "GPT" is also used in the names and descriptions of such models developed by others. For example, other GPT foundation models include <a href="/facts/EleutherAI/CKaKGUSY">a series of models</a> created by <a href="/facts/EleutherAI/CKaKGUSY">EleutherAI</a>, and seven models created by <a href="/facts/Cerebras/Xu4HUfJl">Cerebras</a> in 2023. Companies in different industries have developed task-specific GPTs in their respective fields, such as <a href="/facts/Salesforce/jEqXM3Gr">Salesforce</a>'s "EinsteinGPT" (for <a href="/facts/Customer_relationship_management/Y74q0uXB">CRM</a>) and <a href="/facts/Bloomberg_L.P./2NCtubRH">Bloomberg</a>'s "BloombergGPT" (for finance). </p>