NCU News
Exact sciences
Artificial Intelligence for the Nobel Prize
The Nobel Prize in Physics was awarded to John J. Hopfield and Geoffrey E. Hinton. Recognised by the Royal Swedish Academy of Sciences, the research on machine learning using artificial neural networks is commented on by Prof. dr hab. Włodzisław Duch from the NCU Faculty of Physics, Astronomy and Applied Computer Science.
The Royal Swedish Academy of Sciences honoured two scientists for 'fundamental discoveries and inventions that enable machine learning with artificial neural networks.' The award is shared by Prof. John J. Hopfield from Princeton University and Prof. Geoffrey E. Hinton from the University of Toronto. They were recognised for, among other things, using physics to construct methods that helped lay the foundations for today's powerful machine learning. John Hopfield created a structure that can store and reconstruct information. Geoffrey Hinton invented a method that allows independent discovery of data properties, which has become vital in today's large artificial neural networks.
Imitating the brain
- Today, I rarely hear words such as machine learning or neural networks mentioned in the public debate when talking about artificial intelligence (AI). And in fact, they are the foundation for AI,' explains Prof. dr hab. Włodzisław Duch. - For many years, roughly until the end of the last century, artificial intelligence developed by trying to describe the world using natural language, words, symbols, logical statements. Reasoning consisted of inventing strategies, manipulating various rules to arrive at conclusions. This was the 'symbolic' approach to artificial intelligence, today called 'good old fashion AI' (GOFAI), i.e. good traditional artificial intelligence understandable by humans.
However, it was not possible to create a good system for natural language dialogue, recognising images or analysing them, or converting human speech into text. The only system that was capable of doing this was the neural networks in human brains.
Thanks to the pioneering work of the winners of the just-awarded Nobel Prize in Physics, this system has also moved into artificial neural networks, explains Professor Duch. - Thanks to this and some more recent algorithms, it has been possible to create large neural networks that process very diverse information: texts, images, video, speech - at many stages. These are networks with up to a couple of thousand billion parameters that allow them to learn everything. This seems like a lot, but we still have many more equivalents of such parameters in our heads, with around 100,000 billion parameters, or connections between neurons that change as a result of learning or remembering new events.
Foundations for artificial intelligence
- Without doubt, both John J. Hopfield and Geoffrey E. Hinton are both pioneers in the field of machine learning. In the 1980s, when hardly anyone was looking into neural networks, Hopfield tried to publish an interesting paper about his model of associative memory, now called the Hopfield model, explains Professor Duch. - At one conference, he recounted that this paper was not accepted anywhere, it seemed too simplistic. He finally managed to get it published in the journal Proceedings of the National Academy of Sciences, but only because he himself was an editor there.
It was the simplest model and its operation resembled what happens in the brain when we recall something.
The model of the network he created allows the rest to be reconstructed on the basis of partial information. Just like people do. Sometimes a clue is enough for us to recall the details of an event related to it,' explains Professor Duch. - In the Hopfield network, the activation patterns of its elements (corresponding to neurons) allow it to remember images and different information. It is enough to show it a fragment of the pattern for the network to reconstruct the entire information presented during learning.
The second award winner, Geoffrey E. Hinton, published the paper 'Learning Representations by back-propagating errors' in the journal Nature in 1996 with David Rumelhart and Ronald J. Williams. It was considered as groundbreaking.
Prior to this discovery, neural networks were not, for example, able to learn to recognise from medical data whether a person is sick or healthy. To determine this, a neural network has to produce an internal representation of knowledge about the world based on the information that is used to train it. And this is what these scientists have succeeded in doing, they have created an algorithm that allows the network to learn automatically, explains Professor Duch. - Their method, called 'backpropagation of errors', involved changing the parameters of the network based on the network's error, rolling back information about that error through all the elements of the network so that on the next attempt the error would be smaller.
For many years, it was mainly this method that scientists relied on to teach neural networks. Unfortunately, it only works for fairly simple networks. However, for complex networks, where there are many transformations, something new was required.
- In 2015, Geoffrey Hinton, together with Yann LeCun and Yoshua Bengio, presented an algorithm in one of their papers that showed how to teach networks with multiple transformations of input information. This algorithm was called deep learning, explains Professor Duch. - This is a foundational method for solving complex problems, such as understanding the meaning of words or recognising what is in pictures.
This is just the beginning
In physics, we use artificial neural networks in a wide range of fields. The work of the winners has already brought great benefits, emphasises Ellen Moons, chair of the Nobel Committee for Physics.
The Nobel Prize-winning discoveries play an important role in many scientific studies, including the sorting and analysis of vast amounts of data. This has made possible, among other things, the discovery of the Higgs particle. Other applications include noise reduction in the measurement of gravitational waves that originate from colliding black holes; it is also useful in the search for exoplanets. The technology is also being used to determine the properties of biomolecules, for example to calculate the structure of protein molecules, or new materials that could already contribute to more efficient solar cells in the near future.
