(Long read ~3750 words, best enjoyed with a cup of fairtrade coffee or tea. This text was originally written in Norwegian, translated to English by means of unsustainable Artificial Intelligence, and modified by sustainable Human Intelligence)
The progress we have seen in artificial intelligence (AI) over the last decade has been impressive, but it is not sustainable, neither in an environmental, social or economical sense. It is still possible to adjust the path, because recent research towards more human inspired AI indicates that AI can become both more sustainable as well as more efficient. However, more human-like AI also brings ethical challenges that society must address, today.
Artificial intelligence (AI) has come to everyone’s attention in recent years. Although the concept and the methodological development have been with us for almost 70 years, it was not until about ten years ago that AI made the biggest leap forward in terms of utility and usability. This was primarily a result of improved algorithms, growing computing power and large amounts of readily available data. At the very least, the big tech companies like Google, Facebook and Amazon have been able to take full advantage of this. Since then, the development has mainly been about increasing computing capacity, increasing the amount of data used in the training of the models and expanding the use to new areas.
This has given us impressive examples of what AI can do in very specific situations. It can beat world champions in board games such as chess and GO, it can to some extent take control of our cars, and recently Google Deep Mind was able to present a model, AlphaFold, which with relatively high accuracy can predict the 3-dimensional structure of proteins, based only on the amino acid sequence of the protein. The latter example may be difficult to grasp the potential off for most of us, but it can potentially be a fantastic tool in biotechnological and medical research.
These are examples of so-called narrow AI. The trained models are apparently superior to humans, but only in very specific areas. If the Google Deep Mind’s “machine” AlphaZero, which is trained to play conventional chess, was challenged in a game on an alternate board with fewer or more squares, it would fail completely. To master the new game, AlphaZero had to be retrained from scratch. It can be said that AI can be super-intelligent in specific tasks, but useless in all other situations. Nevertheless, there is no doubt that AI will change our everyday lives to a greater extent in the time ahead by precisely streamlining automated and limited tasks.
AI and environmental and economic sustainability
Training the so-called deep learning algorithms that underlie AlphaZero and similar models is, however, very energy-intensive, and also costly and with negative environmental impact. Two years ago, researcher Emma Strubell and colleagues at the University of Massachusetts calculated the carbon footprint of one training session of such a model to be 284 tonnes of CO2, that is, five times the lifetime emission of a car. The fact that the trained models are static and therefore have to be run again every time they are to be adjusted or updated, and that models are constantly being developed for new tasks, of course means that the total cost and environmental impact is multiplied.
Moreover, since Strubell & co made their calculations in 2019, at that time on models with about 200 million parameters, requiring 32,000 hours of training, the AI models have grown in complexity and depth, and the number of parameters is now up to over one billion for Google’s latest language model! We’re now talking about weeks and months to train the model and the carbon footprint increases correspondingly. In addition, there is a significant environmental burden related to the storage of big data across the world. A calculation from 2016 showed that the world’s data centers at that time consumed as much energy as the entire aviation industry.
In 2020, a group of researchers at MIT published an analysis based on 1000 scientific papers that showed that the progress (measured as prediction accuracy) for AI models in recent years mainly is explained by an increase in computing power. They conclude that a similar development in the coming years will neither be economically or environmentally sustainable, nor technologically possible with current AI algorithms. Their conclusion is that new advances in AI will depend on either more effective methods, or switching to other methods in statistics or machine learning. More efficient computers like quantum machines or massively parallel machines with architecture inspired by the human brain, may also reduce training time, but the calculation costs are currently unknown for such unrealized future machines.
AI and social sustainability
Even from a social perspective, the development in artificial intelligence is not sustainable. Perhaps the biggest challenge in achieving the sustainability goals by 2030 lies in goal no. 10 (reducing inequality) and its interaction with the other goals. The UN has launched the concept of “leave no one behind”, which specifically addresses how to work with the sustainability goals in light of the fact that everyone should have equal opportunities.
Technology development is an area where this is particularly relevant. There is a great danger that the technological gap, both in and between countries in the world, will increase if one is not aware of this. AI is unfortunately an example where technological development so far seems to contribute to increased inequality. As we have seen above, R&D activities in artificial intelligence require such enormous resources and amounts of data that only resourceful countries and the big tech companies can afford it.
Discrimination and prejudice have also proved to be a problem. Many AI models are trained on enormous amounts of observational data, often collected from the internet. In addition to useful and relevant information, the data can hide unintended biases between groups (gender, ethnicity, etc.) or even unfortunate/immoral tendencies expressed by groupings in society. Biases and prejudices can then be adopted by the models and thus reinforce the biases or prejudices by their use.
This problem is due to the fact that the methods are mainly based on “bottom-up” learning, that is, learning only from observational data and not taking instructions “from above”, so to speak. In addition, the trained models are not transparent (black box models), which makes it difficult to detect “bad habits” picked up from big data. The latest Facebook algorithm, Seer (SElf-supERvised), is an impressive example of how AI can teach itself to categorize objects from just watching pictures on Instagram with minimal human intervention. This billion-parameter model, which requires “gargantuan amounts of data», is literally left to teach itself. Although the potential for improved performance in object classification is big, the undesirable bias problem may also become a serious issue, unless better ways to control biases are developed.
The bias problem is addressed in international strategies (e.g. EU, OECD) for artificial intelligence. Actions to reduce biases and discrimination are called for, like data cleaning, where attempts are made to wash or balance the data in advance to overcome the problem. A catch here is that for the many-dimensional problems that the AI models often address, for example in language processing or in Facebook’s Seer algorithm, it is not possible to clean the data in all thinkable ways. The pursuit of unbiased AI is therefore basically futile, although one should of course seek to minimize the problem. This means that we must be able to cope with the fact that AI will be biased to a greater or lesser extent also in the future.
How can AI become more sustainable?
If AI is to follow a more sustainable path towards greater performance and application, we must solve the challenges outlined above. The question is how new advances in AI methodology and technology can be compatible with changing course towards a more sustainable AI.
What we need is AI that is energy efficient, that learns quickly, preferably from little data. It should also be dynamically learning. We should to a large extent be able to manage biases and ethical/moral aspects, and we should also know the uncertainty associated with the decisions that AI makes for us.
Where do we start?
Well, probably a fruitful strategy is to seek inspiration in the human brain, because between our ears we have a deep learning “machine” with all these properties. Ever since the beginning of the development of AI, the human brain has been an inspiration for developers, albeit to varying degrees. We now see that researchers in AI and neuroscience seek together to learn from each other (e.g. Zador Lab). Neuroscience can learn about the brain from the properties of AI models, and AI developers draw inspiration from neuroscience in the pursuit of what is called artificial general intelligence. This is AI which in principle can solve a wide range of intellectual problems, as humans can, that is, not just play chess, translate texts or fold proteins. But in this context, an equally important point is that this development can also be the path to a more sustainable AI.
Four principles of human learning to make AI more sustainable
There are several principles of the human learning that have, to only a limited extent, influenced development within AI. We’ll look at four of them here and relate them to recent research within artificial intelligence.
1) Top-down control. As humans we learn through an interplay between bottom-up sensing and top-down control. The focus within AI research has mainly been on bottom-up learning for the last 40 years. As humans we have an enormous ability to utilize previously acquired knowledge in new situations. Learned patterns, categories and concepts are adapted to new situations so that we can learn from very few instances, whereas AI as of today mostly has to be trained from scratch for each new problem. For example, when we first got purple carrots in the grocery stores a few years ago, most of us were a bit perplexed for a moment, but due to our knowledge of orange carrots, just one single observation of purple carrots was enough to expand our mental category of “carrots”. In this way learned and re-used concepts make humans into incredible fast learners. In fact, some concepts (reflexes, instincts) appear even to be “programmed” from birth. The sucking reflex in babies is one example, while in the animal world we find many examples of instincts that contribute to animal survival.
Relevant research. AI, as it is typically is constructed today, is predominantly bottom-up learners and would need a large number of examples to learn a new concept. However, a recently published article by Rule and Riesenhuber shows the potential for top-down learning in AI, as well. By using previously learned categories in image recognition, while learning new similar categories, e.g. recognizing car wheels by first learning to recognize bicycle wheels, they could point to effective learning from training with a significant reduction in the number of necessary observations.
Sustainability aspects: Herein lies an enormous potential for fast and efficient learning, and thereby a reduction in the need of costly and energy demanding computations. Furthermore, the availability of AI would depend less on access to big data.
2) Social transfer of concepts. Perhaps the most important key to human success is that we are social and able to learn concepts and ideas from each other. In fact, we do not even have to experience it ourselves! Many skills, such as learning to ride a bike, do happen through a lot of trial and error, and learning takes place more or less like a robot would do today. However, when it comes to questions of right and wrong, morality, ethics and values, we mostly learn in a more top-down way, and from others. When a mother or father tells their child that it is wrong to steal, the child usually accepts it, perhaps after answering some “why?” questions, but without testing it out repeatedly at the local mall. This means that there should be opportunities for controlling the ethical and moral aspects of AI by controlling the biases in a top-down manner rather than just leaving it to bottom-up learning from cleaned or balanced data.
Relevant research. Although it seems like much research still focuses on a bottom-up approach to AI, there is a growing focus now on how to get machines to obtain deep understanding and possess a kind of common sense. More research remains to find out how such a transfer of higher-order concepts can be effectively done in deep learning AI, but a key here probably lies in human language that can be used to represent and convey general thoughts, ideas and opinions. Research in what is called Natural language processing (NLP) linked to transfer learning and representative learning is becoming a central field in the search for artificial general intelligence.
Sustainability aspects: Transfer learning increases the sustainability benefits of top-down learning by adopting concepts and categories from other “learners”.
3) Dynamic learners. The brain is plastic, that is, changeable through learning. Thus, human learning is a dynamic, continuous process in which our learned concepts or biases (opinions, perceptions, interests, prejudices, knowledge), are continuously adjusted based on what we sense and experience. In statistics this process would fit the description of a Bayesian filter, where prior assumptions turn into posterior beliefs in light of experience. In this way we become increasingly wiser, hopefully, at the same time as our individuality and personality are strengthened throughout life.
Relevant research. Principles for dynamic learning, which has proven effective for learning in humans, has not been explored to any great extent within deep learning contexts in AI. A recent exception is a research group that constructed a deep learning model with a defined structure inspired by the neural system of the nematode C. elegans with its 302 neurons and 7000 connections. They describe the method as a liquid neural network because it has the property that it can learn continuously by strengthening/weakening connections as a result of experience and can adapt to a changing environment.
Sustainability aspects: With dynamic, continuous learning, it is reasonable that the necessity of re-training models for updates or adjustments could be reduced, and again, here lies a huge potential for a reduced carbon footprint of AI. In addition, availability to AI would increase by providing access to previously trained models as templates for further development through dynamic learning.
4) Confidence assessment. Predictive coding is a central theory in cognitive brain research that states that through life and based on experience, we build mental models that are used to predict what will happen next, or what we will sense or experience in the immediate future. The discrepancy between what is experienced and what was predicted is used to adjust our internal models, if the experience is found credible and we trust our senses. The theory can also explain how strong internal models can twist sensory impressions and lead to illusions or misconceptions, if we have little confidence in or doubt what we observe. As human beings, we thus have a system that links uncertainty assessments to our perceptions and judgements.
Relevant research. This is more or less absent in deep learning algorithms, but can give important information about the uncertainty in the decisions that AI makes for us. If there is great uncertainty, more information should be gathered, if possible, for example to make the most responsible choices. Here we see the need for statistical considerations to be included in the methods (again). Uncertainty analysis for artificial neural networks is not new, but have proven to give heavy and time-consuming calculations, and new, fast methods are needed for increased sustainability. More research is needed here on how the brain links so-called confidence to its assessments, but a research group at MIT/Stanford has recently published a promising and rapid method called deep evidential regression that can lead to safer and more robust AI systems in the future, for example for self-driving cars or in medical diagnostics.
Sustainability aspects: Confidence information may increase the trust in AI models and their predictions and potentially prevent unfortunate consequences, like instances of discrimination, prejudice or even fatal accidents. Confidence assessments can potentially also limit the need of gathering more data if data anyway are noisy and do not increase the confidence in the model predictions.
Challenges and opportunities with human inspired AI
Ethical principles, reliability, transparency, explainability and accountability are emphasized in international strategies for AI as important aspects that must be safeguarded. It is very important that guidelines and legislation are at the forefront of technological development. A development towards a more human inspired AI is apparently inevitable, and as discussed above, also desirable from a sustainability perspective, but at the same time it increases the importance of regulation.
As mentioned earlier, it is more or less impossible to create AI that is completely unbiased, and just as it is natural for people actually to be biased, it will also be so for AI with built-in top-down control, because it is precisely the biases that define the overall control function. But here also lies an opportunity for more transparent, overarching and integrated handling of ethical and moral principles. Eventually, it may be possible, as part of a dynamically learning model, to instruct AI at a higher level through the use of appropriate a priori representations like “it is wrong to steal”, or “women and men should be evaluated equally in recruitment processes”. Or to quote John Connor in the movie Terminator 2 in which he morally instructs the terminator played by Arnold Schwarzenegger: “You can not just go around killing people!”.
The challenge is, of course, the same for a top-down moralization as the one we face if it is to be given through cleaned data; whose moral standards and ethics should apply? What is perceived as right and wrong is both time and place dependent. Here, the world community must agree upon wat is ethical and morally acceptible. Furthermore, it points to the importance of AI being developed in interdisciplinary environments composed of expertise not only within AI, statistics and technology, but also from social science areas, such as law and philosophy.
On the horizon of fast and dynamically learning AI, there is another issue that needs attention, namely that learning of a machine or robot will depend on its experiences, and individual machines could develop in different ways and in different directions as they «grow up». It is not inconceivable that in a few years, robots will be referred to and treated more as individuals for this reason. This opens up new and demanding questions about responsibility for the actions of machines. Already today, questions are being asked about who is responsible for traffic accidents involving self-driving cars; the driver or the car manufacturer? This problem will not be easier if the car’s built-in AI system has become accustomed to new habits, perhaps even from the driver, through many hours in traffic since it rolled out of the factory. Such individualization can easily trigger our imagination and be an inspiration to Science Fiction literature, but future scenarios, where talking about “schooling” rather than “training” of artificially intelligent robots, is not unlikely. In fact, Facebook’s Seer algorithm is already well seated on the school bench. Soon we may even talk about robots with different types of personality. For example, a robot with clearer top-down control than bottom-up influence from data, is likely to score high on the personality variable Openness to experience in the widely used Big five inventory in personality psychology. The psychoanalyst Carl Jung would probably type this robot’s personality as intuitive.
Such scenarios as AI with personality, can seem both distant and perhaps frightening. Many fear what the future will bring in this field. Will robots eventually develop consciousness and reach the point where they surpass human general intelligence and become a threat to humanity? Experts in consciousness research mainly agree that this is not a likely scenario. The human component in this equation is probably much more important to control, because it is AI in the hands of humans that can go wrong if we are not precautionary and agree on how the future AI should be regulated. It will be necessary to develop both ethical, responsible, robust , transparent AND sustainable AI.
A broader strategy for sustainable AI
As we have seen examples of, there are various research groups around the world that work with some of the issues that are highlighted here. Also here in Norway more energy-efficient algorithms, explainable AI, human-inspired AI and moral/ethical principles for AI are areas of research, for instance within the NORA network. Focus on sustainability is also increasing, but mainly within applications of AI to strenghten the progress towards achieving many SDGs. It is positive that Norway in the National Strategy for Artificial Intelligence aims to take the lead in the work of producing ethical aspects of AI. The strategy states (my translation): “Norwegian society is characterized by trust and respect for fundamental values such as human rights and privacy. The government wants Norway to take the lead in the development and use of artificial intelligence with respect for the individual’s rights and freedoms.” Here, Norway could have taken the lead and even clearer responsibility for global, social sustainability and fronted the “leave no-one behind” vision on the technological side. Also when it comes to the economic and environmental aspects of AI, Norway could aim to lead in a more sustainable direction, but then we must have a clear idea of what is the right direction. We have seen that the environmental consequences associated with energy use and data storage are growing, and a further investment in computational power and data storage does not seem to be sustainable, even though this is precisely where the last ten years’ success in narrow AI has come. In this article, we have pointed out that investing in a more human inspired AI is not only a sensible path towards artificial general intelligence, but also to sustainable AI addressing the entire Agenda 2030.
