History of artificial neural networks

<p><a href="/facts/Artificial_neural_networks/6V1jMlkx">Artificial neural networks</a> (ANNs) are models created using <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> to perform a <a href="/facts/Neural_network_(machine_learning)/6V1jMlkx">number of tasks</a>. Their creation was inspired by biological <a href="/facts/Neural_circuit/pgngqMLf">neural circuitry</a>. While some of the computational implementations ANNs relate to earlier discoveries in mathematics, the first implementation of ANNs was by psychologist <a href="/facts/Frank_Rosenblatt/fdPnkXdt">Frank Rosenblatt</a>, who developed the <a href="/facts/Perceptron/ArxdkAC1">perceptron</a>. Little research was conducted on ANNs in the 1970s and 1980s, with the <a href="/facts/Association_for_the_Advancement_of_Artificial_Intelligence/BjScke2s">AAAI</a> calling this period an "<a href="/facts/AI_winter/DLpU1wxl">AI winter</a>". 
</p><p>Later, advances in hardware and the development of the <a href="/facts/Backpropagation/lCsIdKHc">backpropagation</a> algorithm, as well as <a href="/facts/Recurrent_neural_networks/bx7hBVB1">recurrent neural networks</a> and <a href="/facts/Convolutional_neural_networks/kHEdnmGU">convolutional neural networks</a>, renewed interest in ANNs. The 2010s saw the development of a deep neural network (i.e., one with many <a href="/facts/Hidden_layer/uKzUskIX">layers</a>) called <a href="/facts/AlexNet/FUFE1Xov">AlexNet</a>. It greatly outperformed other <a href="/facts/Computer_vision/Tl2Yyk66">image recognition</a> models, and is thought to have launched the ongoing <a href="/facts/AI_spring/DLpU1wxl">AI spring</a>, and further increasing interest in <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a>. The <a href="/facts/Transformer_(deep_learning_architecture)/cDbjx6a8">transformer architecture</a> was first described in 2017 as a method to teach ANNs grammatical dependencies in language, and is the predominant architecture used by <a href="/facts/Large_language_models/WnogWVJY">large language models</a> such as <a href="/facts/GPT-4/MuXdd4sW">GPT-4</a>. <a href="/facts/Diffusion_models/ay78DDLn">Diffusion models</a> were first described in 2015, and became the basis of <a href="/facts/Image_synthesis/YYxfnmAq">image generation</a> models such as <a href="/facts/DALL-E/rkWL8P49">DALL-E</a> in the 2020s.
</p>

History of artificial neural networks open-in-new

History of artificial neural networks