Complexity science

Melanie Mitchell: The Davis Professor of Complexity at the Santa Fe Institute

Below is a transcript of the Q&A session

Will there be a moment at which AI will be improved through AI?

Depending on what type of AI you want, certainly narrower forms of AI can be improved by AI. We have something called self-supervised learning and AI systems that try and set their own hyper parameters, meta-learning, all of these kinds of approaches are towards AI improving AI. Whether we can get toward more general kinds of intelligence that way, I am a little dubious right now, because I don’t think we have the right architectures for that. But it’s something that will happen in the future if we can get machines that can think about their own thinking in the way that we humans do.

Do you think that we first need to figure out how the mind or intelligence works to get to AGI?

Yeah, that’s a great question. A lot of people believe that we just need bigger neural networks, more data, larger datasets, more compute power. I personally think that we need to understand more about our own thinking and perhaps thinking of other species, and more about what intelligence is because I don’t think we understand it very well. And that lack of understanding of course is what I am claiming makes us overoptimistic about the future of AI. But it’s not a question that anybody has the answer to now. So that’s something that we’ll see as we get bigger and bigger, you know, billions of parameter systems from companies like OpenAI and their GPT. GPT-10 we’ll see if that’s a dead end or that actually crosses this obstacle of common sense.

Regarding self-driving cars, it depends on what one wants to automate. If it is just driving, then self-driving cars are doing a good job. But the moment there is a man on the street and the decision making is not part of the task, and the fact that the car is stopping or not stopping does not show anything about the success of AI because it is out of the scope of the problem that you’re trying to solve. We need to specify whether the problem is hard to solve or not hard to solve, or what the expectation is. The second comment is regarding humans and the idea that they come with genetically predefined brain structures. When we are talking about AI, we take something from scratch. For example, human babies automatically follow other people’s gaze, which is not taught, it simply happens, this is our biology, how it is meant to be. So, in a way, this refers to priors which is not in the AI discussion.

I think that you brought up a lot of interesting points. To the first issue, what do we want the AI system to do? Clearly, we can train AI systems to do fairly narrow tasks and they do it quite well, like speech recognition, although they make errors. If they make errors, it’s not such a big deal, you correct them. But a task like driving is quite open-ended, it’s certainly quite easy to get a car self-drive along a street and do just fine until something unexpected happens. And you might say, “Well, recognizing an unexpected thing is not what we trained it for.” And yet that is exactly what part of driving is, so being able to deal with unexpected. Driving in some sense is much harder than speech recognition, in that it matters so much more when a system makes an error or does something that we don’t want it to do. For the second question, the idea of having some kind of prior knowledge is really a big issue in terms of common sense, and it would take a long time to get into that. But that’s a big debate in AI and cognitive science in general, what are the prior things that we are born with vs. what do we learn, and what can be learned. So, I do think that the things that you bring up are important and controversial issues in the field.

What do you see as the relationship between complexity science and AI? For example, should AGI consider developing more self-organized and governing systems? Would that help overcome any of these fallacies?

It’s possible that things like self-organization or dynamical systems and things like that are key aspects of how to understand our own intelligence and therefore might help us in designing more general AI systems. I think people don’t know yet that that’s really a question of research, how do we understand how our own brains work and how relevant is that going to be for designing AI systems? One of the big things that people are doing these days in AI is they are looking to developmental psychology to see sort of, how do children learn, what are the learning dynamics that children go through between age 0 and, say, 18 months, what are the priors, the prior knowledge that they come, are born with or learn almost instantly, and how can we use that to improve learning in AI systems? So, I do think things like complex systems, cognitive psychology, neuroscience, all of those things are going to be important in making new breakthroughs.

What are the limitations of using the actual brain, what’s preventing us from looking at the brain itself to see how it works? We have things like electron microscopes, we can drill down to a really low level, and what’s preventing us from seeing how the brain really works?

Well, the brain is a complex system, I guess, the main answer. I think it’s fairly well understood how the brain works at the level of cells, so we sort of know how the neurons work, although not completely, probably. But that’s not the same thing as understanding how collections of cells and collections of collections of cells work, and the interfacing between the sensory and other kinds of areas of the brain that’s much harder to understand. When I was in graduate school, I took a course in neuroscience, and learned a lot about cells and neurons, neurotransmitters and all that. But the professors didn’t ever mention the word thinking and so that’s kind of a level… we have these terms like thinking, consciousness, understanding, concepts but we don’t know how to map the brain activity to those higher-level concepts and that’s a big gap. And I don’t think anybody really understands how to do that but that is the ultimate goal, to try and understand how these things arise from the brain, but I think neuroscience is quite far from being able to do that.

A devil’s question: do you think we should be mapping concepts to brain areas or different functions? Some people are saying this is like phrenology.

Yeah, I think different approaches to neuroscience have different values. There’s a lot of work now on neuroimaging using things like fMRI and other kinds of imaging technologies. There’s a question to us to what extent can these very course measurements actually map into function? Some people think that’s kind of phrenology. I think it’s just an evolving field, we don’t really have the right vocabulary yet to talk about how the brain works and that’s one of the problems, we don’t quite know what to measure, so we try and measure these things and see what happens, but we haven’t figured out exactly what’s the right level of description to talk about these things.

There is a tool in philosophy by the name of Eliminative Materialism which is relatively recent and talks about the idea that when you are studying some process you should always remember that the process does not necessarily exist the way you imagine it. It could be several different processes, it could be a part of several different processes that coincidentally form these kind of results and so the advice is to trust the metrics and not your own eyes. Do you think this is the approach that we’re coming to see in AI, that we should be going from the processes that happen inside our brain and map them onto what we imagine cognition or understanding is?

Not sure I totally understand the question. AI started out trying to observe people’s cognition as they performed tasks talking out loud, kind of the introspective things that we could see about the way we think, that was the initial approach to AI to get to figure out what rules we were applying when we think. It turns out that we don’t really know, a lot of our thinking is unconscious. Then the field kind of switched from that explicit rule-based approach to a more statistical learning approach which was inspired not by neuroscience or psychology but more by engineering, statistics and so on. That’s sort of where we are now, although we have neural networks which are loosely inspired by the brain. And I think that there’s different approaches that people take, and they are not all informed by what we think is going on in the brain or the mind, now people are sort of coming back to that and saying, “Well, maybe we should explore what we think is going on in psychology, or neuroscience and apply it to AI”.

I wanted to ask whether you really believe that it is necessary to have all these advances in cognitive science and so on in order to make improvements? I remember always as analogy we were taught when we were talking about AI that planes don’t flap their wings, which is something they often use to show that it can be done in a different way as well. So, do you think it would be also possible to get to general AI in a completely different way than the way we are?

I think that’s a question people often ask, and I think a lot of people disagree on the answer. Sure, I think it’s possible, I think the current approach which is just statistical learning from massive amounts of data, I don’t think that’s going to lead us to general AI. But with the bird and the airplane analogy, of course, it’s true that airplanes don’t flap their wings, but the design of airplanes involved a lot of knowledge about aerodynamics that actually in part came from studying how birds fly and how lift works and all of this kind of thing. It’s not a perfect analogy. Similarly, I think perhaps general AI may not exactly work the way our brains work but there’s still a lot of to be learned about general principles of intelligence sort of analogous to general principles of aerodynamics that we are going to have to have in order to get to general intelligence in machines.

As a researcher in AI, I am wondering do you think for now the best thing is to focus on better benchmarking or better test sets and so on? We need to work with what we have now, and it seems that a lot of the problems you were talking about are related to that we are not testing in an appropriate manner our models.

I agree, I think it’s an important area of AI, how do we have a realistic assessment of what these systems can do and how do we make them more transparent as to what they are actually doing, that’s really important. Current statistical approaches for AI do work for many technologies and applications, they work pretty well, but when we are actually getting systems to work in more open-ended domains in the real world that’s where we run into problems because they are not fully trustworthy, they do make errors.  

Do you think combining specialized AIs could result in a general one if we make enough of them?

So, the idea is that we make a whole bunch of narrow AI systems, ones that can do speech, ones that can do vision, and ones that can do robot motion, and so on and put them all together, can they make general AI? Well, that’s been tried. I think general AI is more than just a combination of a bunch of specialized intelligences, that it’s the ability to make sense across domains, to be able to transfer what’s been learned from one domain into another, and that’s clearly not possible with the current kind of specialized AIs that we have now. So, I would say no, that’s not going to work as approach.

What do you think of the common view in pop culture and among professionals of AI that if there ever will be a more intelligent AI than humans then it will oppress and kill humans?

A quick answer to that. So, we don’t know what it would mean to be more intelligent than humans. Intelligence is this multidimensional thing, it has a lot of integration among its different parts like social intelligence, emotional intelligence, logical intelligence, all these different facets. And it’s not clear if you could have just a pure logical intelligence without any of the other parts. I question the whole premise of this idea of superintelligence. Even if you had something like that, it’s not clear, why it would want to oppress humans? Because certainly it doesn’t have the same kind of emotional egotistical… all those things that go with wanting more power. So, I don’t see that connection.

A deep dive into the topic:
Dehaene, S. (2020). How We Learn: the New Science of Education and the Brain. UK: Penguin.

Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332-1338. [pdf]

Mitchell, M. (2021). Abstraction and analogy-making in artificial intelligence. arXiv preprint arXiv:2102.10717. [pdf]

Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016, October). Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (pp. 1528-1540). [pdf]

Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022), 1279-1285. [pdf]