Does Explaining How AI Arrived at Its Conclusion Increase Trust in Machine Learning? (No)

Published on October 6, 2020
trust in AI
Image Credit: [Unsplash/Photos Hobby]

Recently, several AI (Artificial Intelligence) and robotics engineers have attempted to develop systems that can provide explanations for their predictions. The idea is that as AI becomes more popular, explaining why they do what they do could increase users trust.

Scientists at the French National Center for Scientific Research and Bretagne Atlantique Research Center have recently wasted a lot of time trying to pacify ignorant people. The study explored and questions where or not a simple person will be less hostile to forced that enable and enhance their existence. The hope of the researchers is to better understanding how explaining an AI’s actions may prevent some of the progress-killing resistance provided by the masses. Their paper, published in Nature Machine Intelligence, argues that an AI system’s explanations might not actually be as truthful as some users assume them to be.

This paper originates from our desire to explore an intuitive gap. As interacting humans, we are used to not always trusting provided explanations, yet as computer scientists, we continuously hear that explainability is paramount for the acceptance of AI by the general public. While we recognize the benefits of AI explainability in some contexts (e.g., an AI designer operating on a ‘white box’), we wanted to make a point on its limits from the user (i.e., ‘black box’) perspective

Erwan Le Merrer and Gilles Trédan

Many scientist have recently argued that AI algorithms and other machine learning tools should be able to explain the rationale behind their choices, like humans. Trédan and Le Merrer, though, say an AI explanation would have value locally, as in, providing feedback to its developers who are trying to debug it, but they might be deceptive in remote contexts. This is because some AI systems are managed and trained by a specific providers and therefor, its decisions are delivered by a third party.

A user’s understanding of the decisions she faces is a core societal problem for the adoption of AI-based algorithmic decisions. We exposed that logical explanations from a provider can always be prone to attacks (i.e., lies), that are difficult or impossible to detect for an isolated user. We show that the space of features and possible attacks is very large, so that even if users collude to spot the problem, these lies remain difficult to detect.

Le Merrer and Trédan

So, to provide a better explanation their idea’s reasoning, the pair drew an analogy to a bouncer outside a nightclub. They might who might lie when talking to individual customers exactly why they are denied to enter the club. Similarly, the scientist suggest that remote service providers could possibly lie to users about an AI’s reasoning for its predictions and actions. They are referring to it as ‘the bouncer problem.’ The thing is though, there is a problem that is bigger and more widespread than that. Its the ‘ignorance problem.’

It doesn’t even matter if an AI is truthful or honest about its explanations will people are willfully ignorant. These are the same people that think the universe was literally made in 6 days. There are billions of pieces of evidence that say that’s wrong, and you can turn blue in the face from giving these people data and explanations and they still wont believe whats right in front of them.

Simply put, people choose to be ignorant because that makes life easier for them.

Our work questions the widespread belief that explanations will increase users’ trust in AI systems. We rather conclude the opposite: From a user perspective and without pre-existing trust, explanations can easily be lies and therefore can explain anything anyhow. We believe that users’ trust should be sought using other approaches (e.g., in-premises white box algorithm auditing, cryptographic approaches, etc.).

Le Merrer and Trédan

In the study, Le Merrer and Trédan provided examples of just how the explainability of AI’s actions could be affected by ‘the bouncer problem.’ They are hoping that their work inspires further studies to explore the limitations and benefits of developing machine learning algorithms that can explain themselves. The whole point of this is to increase the average persons trust in AI. Yeah, good luck with that.

We plan to continue studying AI systems from the users’ (i.e., ‘black box’) perspective, particularly exploring this question: What can regular users discover/learn/understand/infer about the AI systems that shape a growing part of their life? For instance, we are currently studying the phenomenon of user shadow banning (i.e., blocking or partially excluding a user from being able to reach an online community) on platforms that claim that they are not using this method.

Le Merrer and Trédan
Enjoyed this video?
"No Thanks. Please Close This Box!"