Explainable AI

By Stephen Downes
Dec 15, 2021
Transcript of Explainable AI

Unedited audio transcription from Google Recorder

Hi, everyone. This is Stephen Downes once again with ethics analytics and the duty of care. And as you can see, from the screen, today, we're going to talk about explainable AI. This is part of module seven. The decisions we make explainable AI is a difficult concept as something that's coming to vlog.

Perhaps only over the last five years or so. Although of course the concept of explanation has been around for decades. If not centuries, and it results from the fact that artificial intelligence and analytics applications can be, shall we say opaque? So this is led bodies like the United Kingdom House of Lords to make declarations.

Like we believe it is not acceptable to deploy any artificial intelligence system, which could have a substantial impact on an individual's life and less. It can generate a full and satisfactory explanation for the decisions. It will take so we can see here. There's a clear link between the ethics of AI and the explainability of AI for after all how indeed will we evaluate the ethics of a particular decision?

That an AI made if we don't know why it made it? So let's good reason in itself to look deeper into this concept and that's what we'll do in this video. Let's begin though by asking ourselves a more general question. What is an explanation? It's concept, really that if it originates anywhere originates in the philosophy of science and we can go all the way back to people like David Hume perhaps even to any decart looking for theories of explanations of phenomena in the world.

Just there are numerous ways of approaching it. I've got four here that can be drawn from the history of the philosophy of science. One is an explanation in terms of causality, What caused the accident at the corner. Now that sounds like a simple sort of question, but we're going to find very quickly that cause anal explanations run into difficulties because there are different ways of looking at what a causal explanation is.

I have indicated a few of these here, One is the distinction between internal causes and external causes and internal cause is something internal to excuse me. Internal to the person or objecting question, what? Cause the accident we're looking at the driver for an internal cause an external cause might be a condition like the sunlight, shining through the windshield, the sleeves on the road, the bush obscuring, the view, it's cetera.

A second aspect of causality, is the concept of causal chains. And the reason for this is events happening in a sequence and whatever we're trying to explain was only the latest of, in the sequence of events. So, you know, there's a lot of TV shows that have a lot of fun with us idea.

A person does something and then something happens, and something else happens. Something else happens, something else happens. I meant something happens to the person. Originally that started the sequence of causes. Sometimes we can talk about the most recent part of that chain, as the proximate cost or the efficient cost.

This is the the thing that we say is the cause responsible for the event, but because I'm already is a lot more complex even than simple change. There are complex causes where two things coming together creating an event. And we can think of you know multiple simultaneous conditions that are necessary in order for an event to happen.

It's like when the space shuttle challenge or exploded, right? You needed o-rings that were just wrong. You needed the temperature to be low, you needed a specific amount of stress on that particular part of the shuttle and then it explodes even deeper. We look at the causes. Well there were pressures to launch the shuttle early.

There were decisions made that shortcuted the engineering and inspection process a wide variety of things had to happen for that explosion to take place and that's normal. Not the exception. Finally, there's the concept of necessary and sufficient conditions and a lot of people have trouble with this. I know that because I actually had to write a short article on it, in order to correct people's confusion between the two unnecessary.

Condition is a condition that must happen for something else to happen. For example, in order to graduate university, it is necessary to pay your tuition. However, just because something isn't necessary. Condition does not mean that the effect will fall just because you pay, your tuition doesn't mean you're going to graduate university, other things have to happen as well, and we call these other things, collectively, the sufficient condition, if all of these happen together that is sufficient for the event to happen.

And what's interesting is there can be different sets of sufficient conditions leading to the same outcome. For example, one set of sufficient conditions is I pay my tuition I study really hard, I pass my tests and I graduate university. Another one is I pay my tuition, I cheat, I have someone take my exams for me, I graduate now.

You know, clearly one of them is on ethical, but both of them are sufficient conditions, graduate. So, console, explanations right off the back, create issues. I could go on for the rest of the day about cause electricians, but maybe I won't another way of looking at explanations, is as a product as boss, venison explained in 1980 in his book, the scientific image.

An explanation is an answer to a y question. Why did the accident take place? Why did I graduate? Why was my loan decline, right? We're looking for an answer to that question. The thing is, he explained the answer to why questions depending on a couple of things. First of all, they depend on presuppositions about causes and effects in the world.

We're looking for some kind of story which will fit what we already understand about the world. Second thing is, there's a set of alternatives where an explanation really is of the form. Why this instead of that, why are there roses in the garden instead of tulips? Why are there roses in the garden instead of nothing?

Why are there roses in the garden? Instead of a pile of smell, you see there can be different ranges of possibilities. And these ranges of possibilities determine what will count as an explanation. We can also think of explanation as a process. It's a type of reasoning called abduction. Now you may have heard of deduction where you infer from premises to a conclusion that necessarily follows.

And you may have heard of induction which is a type of reasoning where you have a bunch of events that happen and then by induction or projection, you predict, what the next event will be because I already is arguably. And inductive inference. Abduction is known as the inference to the best explanation.

That's the phrase that Gilbert Harmon gave it but really the idea originates with Charles Sanders pierce, the idea here is that if you see a phenomenon you come up with a variety of different explanations as to why that phenomenon happened. And then you pick the one. Well, what's the best one?

Because presumably all of the exponents would work. Well, that's where it gets interesting, best might be simplest, best might be most believable, best might be most convenient, You know, there are very, you know, best might be the explanation, not explains the most things. There are different choice criteria for picking the best explanation.

You know, it makes me think I was just waiting in my newsletter today about Larry Logan and the way Larry Logan describes scientific progress as through the solution to problems. That's really similar to a method of abduction being discussed here. And the idea here is, what counts as progress in science, is the ability to solve the next problem.

Whatever that is, however, you solve it, and then the next problem, and then the next problem. And, and that's what progress is. And so, what counts, as rational, or rationality on this model is whatever, solves the problem. And so, the inference to the best explanation is the explanation that most directly addresses, the problem that you face.

Finally, we can think of explanation in terms of justification And very often. This is what we really mean when we ask for an explanation, especially if we're asking a person for an explanation, Why did you pass through the red light? We're looking, you know, I mean we you know and acceptable explanation might be that it was by accident or that the driver didn't notice but really what we're looking for is some kind of reasoning.

Like there were no other cars. I was in a hurry. Somebody was dying. You know something like that. Something that can justify why this intentional action took place. So I think that that's going to be important later on. As we look at how we try to understand what kind of decisions computers make and in particular AIs and analytics systems make.

So why do we need this at all? Well, how grass gives us a few examples of of reasons? And, you know, we could argue that it all just comes down to human need. We just want to know but what this really breaks down into is, you know, a set of questions.

Like the five here, transparency, we need to be able to tell some kind of story in terms, you know, format. And in a language that we can understand just to be able to replay it in our head, or maybe we want to understand the underlying nature of the things that we're involved in whatever happened.

Typically, this is the causality picture or it might appeal to a natural law or an essential trait, you know, just Joe stole the shirt because Joe just is a, you know, I wouldn't think of that as a very good explanation but it is the sort of thing that provides us with, you know, an understanding of how the character of Joe would be what caused him to steal the shirt.

Another one that comes up a lot, is the question of bias in the question here is, how can we ensure that EAI system isn't biased? In some way, certainly, we've seen in all of the stuff that we've talked about so far in the course, there are many ways, many many ways for an AI or analytic system to be biased.

And really, it is argued. The only way we can be sure isn't biased is if we're able to explain how it is. That it came to the decision that it did. Similarly with fairness we can only understand. It could be argued how a system is fair if we understand how the decision was made.

And finally similarly safety, how can we trust our system? How do we know that the AI won't cope and shoot us with a machine gun or whatever, but we could prevent it from having a machine gun in the first place. But feeling that we need to be confident, the reliability, and the trustworthiness of our AI system and are hearably.

We could only be confident if we know why, it makes the decisions that it makes when we started with artificial intelligence back in the days, you know, the 60s 70s 80s when AI was ruled based and it was just a set of rules and you have an environment and you'd apply a rule to it and get a behavior of.

Then it was pretty simple to explain what was happening because you just look and see what rule was triggered. Why did the heater start? Whether there was a rule that said, if the temperature is less than 65 start the heater, this got a little bit trickier, okay? A lot trick here once we got into expert systems and things like decision trees, and so now it's not a single rule that fires, it's a whole sequence of rules, but we can understand that.

And then from the, the sequence of rules, we might be able to pick the rule. That's the most salient for the most relevant to the current situation. It's a lot like picking out a single item from a causal chain, no problem. But then we got into a neural nets.

Deep models onsembles and different machine, learning, algorithms, and even some into something, which Google is here. Calling meta learning, which basically is taking a whole bunch of these algorithms pitting them against each other, eliminating nine taking the tent, or I guess eliminating eight taking the mind. And so, as machine learning as evolved, we've moved further and further away from explainingability further and further away from being able to say, come up with the simple rule that we can point to that.

We could say the AI, followed this line of reasoning. And that's why I did that I talked earlier. It's and several times through this course, about how an analytics engine will take into account, 60,000 factors 60,000 variables of those. What is the one variable that constitutes a rule that allows us to explain the behavior?

It's really hard to identify that variable to begin with and secondly, to give it a name and to put it in a context that we could understand. And one that does not misrepresent what's actually happening in the AI engine.

So then what is explaining ability? Well, here's something from gunning from a few years ago, not too long ago. Trying to represent what explainable AI actually looks like. This is a, I think it's an IBM presentation again. So in the green boxes at the bottom, there we have different kinds of approaches to AI.

So you know, the deep explanation will provide deep learning the interpretal. More interpretable models will leading to things like Bayesian belief, nets, etc. And then model induction, like random forest or decision trees are examples of the role-based types of approach that we talked about earlier, whatever they are. We have all of these different techniques that blamed and mesh and get used to together alone or in an ensemble.

What we're trying to do is pull from that something relatively simple and straightforward, like this table, or this chart on the right hand side which gives us explainability. And so, you know, it's just, you know, from the red dot to the green dot, from the red dot to the green dot.

That's what provides us our explainability and it does. So, in terms of the nature and the accuracy of the prediction that it provides that that's the idea, right? The trick is going from these, you know, learning technologies, to that explainable model, it's not obvious how to make that leap.

The problem was intractable enough that the United States defense research agency Darpa actually launched a project that they called explainable AI or XAI. And basically what it wanted to do is replace the the learned function with a combination of explainable model and explanation interface. So that the person actually performing the task is able to understand why something happened, or why something didn't have happened, is able to know how to succeed or how to fail.

And when to trust and importantly is able to figure out when the AI model aired in some way. And the argument is simply machine learning models are opaque non-intuitive and difficult to understand a mechanism is needed to make that no longer the case. Okay? So you sketched out the reasons for this and kind of what's involved.

So how do we go about it? Well, what we want, is quote a sense of processes and methods, that allows human users to comprehend and trust the results and outputs created by machine learning algorithms. Something that explains the model or describes the model that's being used. Remember the model in an AI system, especially the neural nets that we've been talking about before.

That's the set of all the connections between the individual entities, The actual connections that were formed and the weights of those connections as the results of the training. So, but I say that I haven't really explained anything. What we need to do is explain why the model looks the way it does and the big challenges even in saying what the model looks like.

Because it's just a bunch of connections between thousands and thousands of neurons. The other thing it needs to describe is the expected impact, is there a way of predicting, what the the model will do? You know, if I had the model? And I had some input. Could I relyably predict what the outcome would be.

Now it's a little bit circular when you put it that way. Because of course, if I could predict what the outcome will be, I wouldn't need them model. But on the other hand, a big part of artificial intelligence, a big part of the motivation for AI is the idea that it's doing things and thinking like a human does.

So that a human doesn't have to do it and it can do what a human does many times faster much more efficiently. So, you know, I mean, it's artificial intelligence. So there should be a, in which a human who approached, the same thing would come to the same decision flag that and then third potential biases, right?

That's kind of like, being able to say if it goes round. Here's how it could go wrong. Yeah, here's my pre-trained model, here's my input. If there's a problem with it, here's how I would know that there's a problem with it. So the model, the impact, the biases, three pretty essential conditions of explainable AI and necessary.

Conditions, we might say. So what's the process? Well, there's no single process to producing explainable AI and, you know, it's going to be based on a variety of factors but here's a process that I've pulled out of a document by IBM, this fairly typical. We're looking at both the data and the model we're looking at what the base the data is based on, is it samples?

Is it features? Where did it come from? Etc. So that we have a handle on what the AI use in order to learn. Then the model are we looking at a local explanation based on something right here and now samples features, something like that or are we looking at a global explanation where we're able to talk about the model?

It's self and not the specific prediction that the model has made. And there's two basic types of of global explanations a pre-hawk and a post-hawk explanation. And we'll talk about those in a post-hoc system. What that's doing is producing and explanation for a specific decision and ideally to make it reproducible on demand.

Now, I've got a couple of examples here. Something called line local, interoperable model agnostic, explanations based on quote a class of potentially interpretable models, such as linear models decision trees or lure or rule lists. So, those are AI models that are basically, like those that we had at the beginning of the evolution of AI, where the AI itself is described as a set of rail based processes.

And then all we need to do is look at which rules were fired in the application of the AI. The all the other approach is black box. Explanations through transparent. Approximations what time? We'll come back and talk about this concept in a little bit, or this kind of approach in a little bit where we're optimizing for two things.

Right? Fidelity to the original model, but also interpretability of the explanations. So really, I mean, we're approximating we're not going to try to explain what every connection and every neuron in the model means, but we'll make an approximation of that. You know, I'll show in a little bit. How we do that with something like fuzzy logic.

The thing with post-hoc systems, is what's known as the, the fallacy of post-hawk ergo propter hawk, which is Latin for after this. Therefore, because of this, it's pretty easy to come up with an explanation of something after the fact. And, you know, when we see the input, we see the model.

We see the result, we can look at the model and say, well yeah, I can sort of see how this part of the model might lead to that or that part of the model might lead to that. So we're, you know, it's after the fact and we can't necessarily say that simply because these things happen before, and then this happened after that, these things were the cause of what happened.

And so we're going to need something that actually allows us to predict what the outcome is going to be before it actually becomes the outcome and that's how we get anti-hawk systems and probably the most obvious of these is what's called a glass box approach where when we build an AI, even a neural network, we're really able to say what every single, one of the neurons in the narrow network means, now we did an example of that, in the very, in the video that began this module, where we had the the number recognizer, and we had the, the input that came in of the diagram and then the first layer, where the, the, the the edges.

And then the second layer where the individual features, and then, the final layer was putting the features together to produce the numbers. We have a similar sort of thing here, right? Where we have primitives like lines that move in certain directions and in sub parts, which are combinations of these lines.

And then object templates, which are even bigger combinations of the line, leading to exemplars that puts out our actual output that we've produced. So glass box approach, we can see and understand everything that's happening. Problem is that glass box approach becomes really difficult when the number of neurons interests the thousands, the tens of thousands or even higher.

Still with anti-hawk methods. We still have various approaches that we can take. I really need to kill Microsoft teams since one person who keeps posting and posting and posting on teams and you won't stop. Okay, let me just there we go. Quit there gone now. I managed just won't know that I'm working but I'll live with that, okay?

So the boolean classification rules via column generation is kind of a way of taking a really complex thing and pulling out of that, some rules that we will say, are going to be what? I'm just stopped. You couldn't miss. I'm sure I quit.

How can that continue to be? You just doesn't seem to wait to be a way to stop teams from sending me messages even when I've quit teens. It's very annoying. Okay. I'm so I can't do anything about the law. Ding that's happening down there. All right. So a good example of this and and you've all seen.

This is before a football game and you have your analysts who give you say the three keys to victory. And here, we have an email honor of the Winnipeg Blue Bomber's winning. The gray cup last Sunday. We have one from the blue. Bombers.

There's a thought there. How there's no sound in my computer? I won't hear them at least So, you know, continue protecting nickels in the pocket, right? So, a week and express that as a rule something, like, if the blue bombers protect nickels in the pocket, then that increases their chances of winning.

And so it came up with three rules, three keys to victory. In this paper by Maliatov and Varsmy. They did one for tennis. I didn't I don't like tennis but but I wanted to put it in here because it's a little bit more precise, right? So the predictive decision rule for Federer defeating Murray in the 2013 Australian Open was win more than 59% of four to nine shot rallies, win more than 78% of points when serving at 30-30 or DC and serve less than 20% of serves into the body, okay?

But you know that, you know, always reminds me of the old joke, you know what's the key? What's the key to victory in this hockey game and the Alice comes along? Well Bob you got to score more goals than the other team. Yeah, you know it's it's a little bit circular in way if you do the things that leads you to win, then you're gonna win.

So it's not ideal but still it's the sort of thing that people would accept as an explanation after the fact as to why Federer defeated Murray or why Winnipeg won the football game, another approach is it can be characterized under the heading of contrast of explanations, and this actually opens up a whole range of possibilities, which I can't really go into.

But basically what it does is considers why in event happened, not in isolation, but instead of some other event now this should remind you and this is why I said it earlier of an explanation, as an answer to a y question, this is the mechanism that is used. So for example, we have from Jacobi this example, where we're trying to understand why it was decided, the higher person X and set a person.

Why? Well, we could say that, you know, person x was born person, x grew up person, acts finished high school. Now all the sufficient conditions for being in a position to have been hired. But really what were after here is, what was this significant difference between person x and person?

Why? Because they were both born. They both grew up. They both finished high school, they both have a relevant degree, but one of them had professional experience in the other one did not. So that becomes the contrastive selection, you know, it's not that the one person had the degree.

They both had the good degree. It's the one had the experience, the other one didn't. Now again this is going to depend a lot on the sort of difference that you think is important because it may also be true. For example that person acts was wearing a blue jacket and person.

Why was wearing a brown jacket and we know don't we that people who wear brown jackets? Don't get higher than to management. If I I had a director number of years ago who fell out of favor with upper management and in the months meeting up to the time when he was let coke, he expressed his protest by wearing brown jackets to meetings.

I don't know if you even knew he was doing this consciously but I could see it from the third party perspective. You know, he, he always wore the blue or gray jacket, or the black jacket, and then he suddenly switched around, and then he was gone. I don't think he was let go because he wore brown jackets.

But we wouldn't accept because somebody wore a blue jacket and not a brown jacket, as a reason, why they were hired. So, what's the difference here? Well, this is an example of modal reasoning strictly speaking. It's a counter factual because we have to consider the example of person. Why I'm ask, what would have allowed person why to win the competition?

Instead of person x even if x1 the competition or what would not have been sufficient to make person acts hired. So there's different analyzes of this. But really, it boils down to comparing what might be called possible worlds and impossible world, semantics, which you do is, you imagine the closest possible.

Possible world, a man. Ask yourself what happened in that world and the question now is what defines? The closest possible possible world. And I've mentioned this before and the answer to that is salience. So in this case, what is salient is the professional experience and not the brown jacket.

But you see what we've done here? Right? We've presumed in our definition of sailing ants, what the actual explanation will be. So we've kind of begged the question. And that's probably the biggest weakness of the contrast of explanations method as soon as you create these contrasting explanations, you're already putting in or embedding into your reasoning process.

Something that will eventually count as your explanation the way of knowing this is that in this case, I would not be able to rule out in any principle way, the brown jacket explanation. All of this gets even more difficult when we're trying to interpret deep neural networks. And I remember from the discussion of models, and interpretation before to interpret, something is to be able to provide well in a broad sense, the semantics of a thing or in a tighter sense, to be able to say what it means, when we say to neurons are connected or some neuron is turned on or whatever.

A way of doing that is to look at the data and they annotate it. So, here we have some pictures here from a scanning electron microscope and we can annotate those. We could say like there's a stocky thing here that's in across. There were some flowery things. Here, here's the stocky thing.

Again, here's the flowery thing. Here's the stocky thing. Here's the flower. You think here's something weird? Here's the flower. Anything only small. You see what I've done here, right? I've kind of, I've looked at those particular pictures and then I've given them an interpretation but given them a description that we can latch on to and use to distinguish one from the other.

And in fact, you know, I could say you know these are thick stocks, these are medium thick stocks and these are sparse stocks right? Big flower. Medium sized flowers. Small flower still always and you don't know what that is. So now if we just interpret the input layer, we've solved a big part of our problem with deep neural networks, right?

Because now I've got a set of properties at the the input layer that have labels and I have an outcome at the output layer, which I can also give labels to and then I could draw a set of rules and if I drew them cleverly enough then it wouldn't matter what's going on.

In sight, I could have rules of thumb for what's actually happening in the neural net. But first of all, how do we justify that? And secondly, how do we kind of make that work? Well, that's where a fuzzy logic comes in The idea is to think of our, you know, our layers and a neural network.

I really put layer etc in an approximate rather than a precise way. And that's what I did here, right? These are stocks. This is a flowery thing, right? So, we're not using precise mathematical terms to describe these things. We're just being vague about it. Or shall we say fuzzy A good example and this is the way humans.

Think typically a human, if you ask them, you know, how should you draw when the road is bad, say that, because it's freezing rain going on right now. So it's a perfect example. They will say something, like, if the distance to the car ahead is low or even more accurately.

If the car ahead is too close and the road is slightly slippery then? Slow down. Now what does too close mean? Well if you say like four car lengths or whatever, but as Sheldon Cooper famously said car length is not a proper measurement because cars can be variable lengths but that doesn't bother us.

We know the car length is. The road is slightly slippery. Well we're not going to do the coefficient of friction. It's either slippery or it's not you tap on the break. You slide a little bit that's slippery. I know I've done that many times similarly slow down. That doesn't mean reduce your speed 3.5 kilometers an hour.

It means just let up a bit so we can take precise numerical things and fuzzify them. And when we fundify them instead of having to account for 60,000 input pixels, we really only need to account for maybe a few hundred possibilities. I'm not the kind of thing that we can handle, right?

You know, the different pump, you know, the different ways of thinking of the distance. The different ways of thinking about the slipperiness of the road, the different ways of thinking slow down all of that adds up to if you hundred parameters and and these parameters are actually calculated differently for every driver.

And that's fine. We expect that a more experienced driver will think of slow as meaning, something different from a less experience driver or maybe the opposite. So that gives us a way of thinking about explaining ability, how is that going to cash out in practice? Well, we've got basically attacks on any of AI explainability that we can look at here.

One shot or interactive explanations come back to that. Are we looking at the data or the model? As we discussed earlier, explanations of samples or distributions, or features or being fuzzy descriptions explanations for individual samples, or overall global behavior. Each of these specifies, a different way that we're going to explain.

And then the different ways that we're going to explain indicate the different kinds of systems. We're going to use to actually produce the explanation and these systems are separate explainability algorithms. And I'm not going to do the list of explainability algorithms. There is quite a list, but it's not going to event.

It's not going to advance our knowledge to go through that list suffice to say answering these questions leads us to an algorithm selection process. So let's look at how this actually plays out. We've got an example from IBM here of explaining ability in practice. So what I'll do is bring this up on the screen.

So here it is. And why don't I maximize? So we get the best possible view. So let's suppose we're trying to explain the results of a loan application. For, let's begin with the bank customer. All right, so we're going to pick someone, they were called denied, so if you're pretty typical consumer experience, so let's pick Jason.

So several features in Jason's application fall outside, the acceptable range, so here's why he was denied. The value of consolidated risk markers is 65. The average age of accounting months is 52 and it needs to be 68 the value of months. Since the most recent credit inquiry, not within the last seven days is two and it needs to be three, which is kind of a weird criterion.

But I guess it, I guess they take that into account and here are the the relative weights of each of these three conditions. So the the consolidated risk marker, whatever that means, right? But you know it'll be a list of indicators that they just brought together. That's the most important factor.

So you provide this to Jason Jason says yeah. Okay. Jason will focus on the third one because people are irrational that way. But basically what this is saying is you got to wait some time and maybe reduce your risk and I don't get you alone. All right, good enough.

So what about the loan officer? Well, it's interesting in this example because here we have a case of Alice who is approved and Robert who is denied. And let's see what the loan officers shown. Well, here are a whole bunch of parameters risk estimate. Oldest trade, open, average. M in file, whatever, that is number of satisfactory trades that are maximum delinquency ever, etc.

So all of these are going to be factors and we can and they're being compared to other people who defaulted notice that they defaulted. And so you see, look at this, it's the same. You've seen these places where it's exactly the same. But even up here, external risk estimate, they're kind of all in the same range.

You know, here, you know, he's similar to some of them. Here's a little bit better than each others but but you know it doesn't seem to be a defining feature. So what we're getting here is a more detailed explanation for the loan officer so that the loan officer has a more comprehensive explanation of why the AI recommended denying the loan and then the loan officer will turn around to say to Jason or Robert.

In this case in sales. Sorry, Robert, you're just too much of a risk and Robert a goal. They unhappy. Meanwhile, we have the data scientist and I love this, because we've got directly interpretable models, or rules and linear terms learned from the fecode data, set, whatever that is. And then some of these, you know, rules and the chart that's associated with the rule etc.

So, this was actually the, the least reasonable the bunch. But still, you know they're asking what is the overall logic of the model and making decisions they did a scientist doesn't care about Jason or about Robert the data scientist presumably cares about the the operation of the model the model in general.

So what we see here, therefore are three different ways of presenting explanations that have very different characteristics to three different people and I find that very interesting. So, what does that tell us? Well, I think we can draw some insights from the social alliances here and there's a nice paper, three years ago from Miller on exactly that.

And I'm gonna put it in the newsletter because definitely worth the read. And we want to think about, you know, suppose the AI or the analytics engine was a person. Now I know they're not, We all know they're not but you know, we're people we tend to anthropomorphize our computers our systems anyways.

So, I'm not going to be a big stretch for us to try to explain why the computer is something in the same, sort of way, we explain why a person did something and even more to the point, we don't have access to the algorithm that's running the person. In fact, we're not even sure there is one, but probably is but still we don't have access to it and even if we did it's way too complicated for us to understand.

So I don't want to say well we make stuff up because that's not really exactly the case. But we do come up with, in fact, we have come up with. I mean, entire language for precisely this purpose. So how do we explain behaviors? Well, for example, we might explain them in terms of intentions and intentionality what we think they plan to do and how we think they think of things in the world.

What objects they think there are cetera, what they're person perception is right, which could be thought of, in this case up how we see them as a person. What they are as a person by comparison to what I am as a person. So you know I have certain intentions, I have a certain way of seeing the world.

I think there are chairs, I think there are tables. I think there is a coffee cup and I'm thirsty. So I form the intention to drink from it, I have done drunk from it and how I'm choking on it right back? Kind of language is called folks psychology and statements about beliefs desires hopes fears knowledge.

All of these cognitive phenomena are the the mechanisms or the tools that we use in order to explain someone's behavior. So why did John Robb the store? Because he believed he would get away with it and he needed the money perfectly satisfactory explanation, zero reference to an algorithm or causes or any of the, you know, it's it's a, you know, it's if you really press it, we really haven't learned anything, right?

Because we ask, well, why does why did he believe he would get away with it? Well, and we regress a bit further into these cognitive terms and then we regress further and we regressed further and we never do get to the level of mirrors and algorithms etc. Because, well, first of all, we can't we have to take what dammit would call an intentional stance.

We just, you know, it's not really reducible but just assume we're talking about the neurons and all that stuff when we're talking about the beliefs in the desires. But, you know, we can fuzzify the connection between them and that's fine. People do that all the time. And, you know, this actually goes back to homes point about causation, which I think is very relevant for this discussion and human says, you know, what is it for us to say that?

Something is a cause and something is an effect. Well, it's a habitual association that we draw between the two. When we perceive them together long enough over time, they form a natural attachment in their mind such that when we see the one we come to expect the other to take place.

It's a paraphrase, not a direct quote, but it's pretty close paraphrasing and our bill. Our statements about beliefs desires, etc. Are they're saying sort of thing. We're probably not referring to anything actually, real out there in the world, just as there are not for you actual causes out of these mysterious forces.

So also there are not really these actual beliefs or actual desires. They're interpretations that we create as as he says, useful fictions. And you might think of how can you use a fiction for an explanation? And my answer is well, why not does it solve the problem? You know, are we able to make progress with it?

So, you know, when we really want to get down to thinking about how to build any eye systems, and yeah, we can't use beliefs desires etc. That doesn't work but if we're trying to explain their behavior, then perhaps not language is perfectly acceptable. Let's suppose that it is then now we look at the social sciences which does work that level of explanation for the most part.

And look at the sorts of things that we accept as explanations in that language. Well, that would include things like norms and morals. For example, why didn't John Robb the store? Well, he knew it would be wrong. You know, why why did Fred wear the blue shirt? Because he had to wear something, I'm playing around with your alternative expectations there, right?

Collective intelligence. And, you know, again whole thing that could be made on collective intelligence. But you know, if you're asking why, you know, why didn't inflation rise? Well people thought that there was a scarcity of things so they were willing to pay more. That seem example of collective intelligence, right?

We're actually giving masses of things like society attributes, high beliefs, and desires. The market wants to see lower unemployment, this week, you hear that kind of language, in the news all the time, we could use what's referred to as Mali's models, to actually describe this. There's really you can describe you, you could approach Molly's models by thinking, first of all, of two ways that we talk about our expectations and how we describe them.

First of all, what we actually think there is going to happen. We'll call that the information I'll approach and then how that looks to other people, we'll call that the perceptual approach. We'll leave the perceptual approach to the side for now because it's not going to be relevant to the current discussion.

But then when we look at the information all approach, we're looking at four, major conditions information, requirements information, access pragmatic, goals, and functional capabilities. And so you can create this these explanations of behaviors in terms of these four parameters, if these four parameters map two, what we observe in the AI in the analytics, then that may provide us with a perfectly acceptable, sort of explanation.

Now, this is hypothetical, this whole field is just a few years old but it's seems you know especially if you like education where we're kind of working that way all the time. Anyways it feels like this sort of approach might serve as the best kind of approach, you know.

It would allow us to use the same kind of language and tools that we use to describe students and why they passed and fail to describe AI systems and analytics engines and why they succeed adults succeed. So, what are the factors in these different types of explanation and how can they go wrong?

Well, here's a list. Again, from the Miller article. I think, like, we take into account things like Adnormality temporality, controlability intent, social norms facts, foils or alternatives, responsibility, coherence, simplicity, and generality not helps us. Pick between alternative explanations and even things like truth and probability. How plausible is the explanation that we've provided?

But the danger in this kind of approach is the same danger that exists in AI. Generally, not surprisingly, our folk site collagical explanatory operators. That's a mouthful might be biased. And here we have from the garden, kunda a simple example of somebody being pushed, right? So you look at somebody being pushed, how are you going to interpret that?

How are you going to describe that to the police later on? Well, it turns out if the person that you see is black and if the person you see is white, you will actually draw a different interpretation and explain the action differently. The black person's push will be interpreted as aggressive.

The white person's push being negatively associated with the idea of it being aggressive. And so, the black persons aggressive push might be thought of as a violent push. Whereas the white persons push might be thought of as a jovial shove. Same physical action to very different explanations as to what happened.

And so the same thing that can happen in our explanations of AI can happen in AI itself. Or I should say that the other way around the same thing that happens in AI can happen in our explanations of AI. So Miller quoting Hilton, talking about conversational processes and causal explanations comes up with the idea of explanation as conversation and I found that intriguing.

So there are basically two stages to this approach. First of all, a diagnosis of causality and in which the explainer determines why in events happen and enter an action happened. And we might refer back to some of the mechanisms that we've talked about already, right? Which means it might also still be flawed, but then there's the actual stage of providing the explanation.

Now, here, it's depicted as resolving a puzzle in the explainy, or the listeners mind about why the event happened by closing a gap in his, or her knowledge. So the presumption here with this particular account is that the explanation is accurate, and the way you resolve the need on the part of the person to have an explanation is that you provide the explanation, then you feel in the details around the explanation, until the person is satisfied.

Of course, in that conversational process, we could expect the pushback to win. In other words, we provide the explanation but the person hearing the explanation, in the course of the conversation is able to raise doubt as to the suitability or dissatisfactioness if that's a word of the explanation. And so you have to you have to start over again and maybe start the whole inferential process again and that's a perfectly acceptable result.

But what's really key about this is that there's no simple set of criteria. No, simple rule. No simple mechanism that tells us how to explain, and then how to present the explanation of a decision by analytics or AI. And that's really important. That makes it very clear. That explanations are sorts of things that happen rather less by rule and rather more on a case by case basis, and why should we have expected any other result?

This is what we've been seeing pretty much consistently through this course about how all of these systems work and how we approach ethical issues in general. So why would we be expecting ability to be, you know, some sort of magic pill which solves a problem that gives us a simple process.

Would just provide an explanation and everything will be great. We don't need to worry about the fuzziness anymore. It's not how it works and this really becomes clear. When we think about what would count as explanations for different people, we saw an example of that. In the IBM example, if we think about the very different domains of discourse that happened among different players, or different stakeholders in the AI and analytics environment.

We can see that what counts as an explanation is itself, going to be very different from the engineering perspective. We might be interested in control performance and discovering information. Somebody deploying and analytics engine will maybe need to explain its rational or characterized strength or weaknesses and perhaps you know promote human machine cooperation.

Meanwhile somebody who's working at the governance, level will be more interested in trust protecting about protecting against bias. Following regular issues and part and policies and enabling human agency. These ironically enough all came from a Brookings Institute paper and if they're getting that right, that shows, how significant these different domains of discourse are They say in their paper?

This is why explaining ability will not save AI, and actually agree with that statement. Explainability won't save AI because explainability is the same thing as a AI. And that's perhaps the irony. When you throw out, explain ability as a criterion for, you know, the ethics or non-ethics of AI.

Basically, you've thrown out the bulk of how we interact with each other. As people You've thrown out far more, then you've kept of values, the classic throwing out the baby with the bathwater. I'm sorry for using such an obvious cliche, but these are all tools and mechanisms that we use every day and sure they're vague, they're fuzzy, they're context dependent.

You couldn't possibly put them down in a roll. And there are no simple explanations of why people see do or say things. And yet we still model through, we still interact with other people. We're even able to have court cases and find people guilty of an offense based on intentions expectations views of the world wants beliefs desires, etc.

If it's good enough for a human arguably, this kind of approach is going to be good enough for an artificial intelligence, and it goes back to what our expectations were the beginning of all of this. Remember, when we looked at the evolution of AI, we started with simple rules and then got into something more and more complex.

Our understanding of how computers work hasn't evolved along with the computers. We're still in a simple rule stage, or must be declared in a functional program, kind of stage of understanding of computers, but they're now far more complex in that not as complex as people obviously. But no longer resembling.

The simple rule following engines that recreated in the 1950s and the 1960s, we still use rules, just like we still use causality just like we still use terminology like beliefs and desires, and fears and hopes and we don't worry about whether these things are precise. The terminology we use to tell us when to slow down or when our behavior was unacceptable, or how to succeed at a baseball game are going to be fuzzy and general and probably not actual concrete representations of the situation.

Indeed, if they are representations of all, and that's fine. And I think that over time as we work with artificial intelligence and analytic systems, and as we become more used to how they operate and more comfortable with how they operate our demand for precise, explanations of why they did something.

Every time will decrease and, you know, we don't demand and explanation of why the car goes, a hundred kilometers an hour. We don't demand and explanation of why our modem, I guess, modems don't make sounds anymore, but when they did, we didn't ask for an explanation of it. I'm not asking for an explanation of my why my computer screen has put up an image on the screen that I can see.

And I think with our artificial intelligence, our comfort levels will increase to the point where, yeah, we can come up with these explanations and, and we will, and it'll be a fun exercise, but it will be about as important as identifying. The three keys to the game for the Winnipeg blue bommers.

That's it for explanation. This is the last really tough and technical section. I think in the course, I'm going to wrap up module seven in the next video and then we'll launch into module eight, where I'm really going to try to begin tying together. A lot of the different threads that we've unraveled through the first seven modules of this course.

And bring us to something like a satisfactory understanding of what we mean by ethics and analytics and the duty of care. I'm Stephen Downs. See you next time.

--------------

Include discussion of:

One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency

Kacper Sokol & Peter Flach, KI - Künstliche Intelligenz volume 34, pages 235–250 (2020)

https://link.springer.com/article/10.1007/s13218-020-00637-y

---------------

Also: Explainable Artificial Intelligence: a Systematic Review
Giulia Vilone, Luca Longo, arXiv, 2020 https://arxiv.org/pdf/2006.00093.pdf

Force:yes