Where do we see Bias in AI models?
Björn: Welcome everyone, to AI Watch. My name is Björn. I’m the Chief Data Scientist in 2021.AI. And today I have with me Nermeen Louizi, a Tech Program Manager at The Lego Group. Welcome. Nermeen: Thank you. Thank you for having me. Björn: Today we’ll want to talk a little bit about bias in AI being a big topic these days. What is your experience with bias in AI models? Nermeen: I think I have both as a professional, but also as a consumer of products with AI in it. I think we forget that AI has always been around before ChatGPT. Even Google Translate, for example, uses elements of AI and Google Maps and all those technologies have been around for a longer time and these have already started showing signs of bias. For example, the most classical example that most people know is when Google Translate, when translating from a gender neutral language to gendered language with pronouns like he or she. For example, if you say, this is a doctor, this is a nurse, it would say he is a doctor and she is a nurse. This is like the most elementary of the examples. Another example that I think is really funny is, Google AI, when they came out with an experiment where they gave you a prompt. For example, it asks you to draw a shoe. So you do a little scribble and you try to draw a shoe. And if you drew a high heel, it would think it was a water slide or a chicken leg. And it’s because at a time when you started training the data, you didn’t have any high heels around because it was a mostly a male dominated team and you didn’t think about it. It was a very innocent mistake. But there’s also more serious cases of course in healthcare where if you are from a minority, racial minority in the data set. For example, if you’re black and you’re a woman, you would not be detected in terms of some medicines and diseases. Defining Bias & its limitations
Björn: So with bias. Bias is a special topic, I think, in AI machine learning, because AI is about detecting patents and patents in data, right? And if we look at certain applications, for example, the insurance industry, bias has always been there. You’re always expected to get a higher price depending on your gender and your driving experience, your age, and what kind of car you’re driving. How would you then define bias? What kind of patents are biased from your perspective, and which of the patents are non biased? What would we not say is biased. Nermeen: I think I would start maybe just saying that bias is not something we can eliminate. We are born with bias and we all have bias, and we’ll continue to have bias. But I think what is a little bit dangerous is that we are not aware of our bias sometimes. So I think it’s that element of, let’s be aware of it and try to mitigate it and be a little bit more representative of the real life data set that we have. For example, when you see that women are being forgotten in some of the data sets, considering that we are half of the population of the Earth, that is a bias that we could try to solve a little bit. Of course, it will always be biased, but at least try to represent the data set in our solution, similar to how it is looking in real life. What is dangerous is that we don’t know what we don’t know. And so I think just being aware of your bias is already the first step towards solving it, because you’d be like, yeah, well, those are the biases. Those are the strengths and weaknesses of algorithms, and that’s where the limits are. I think that’s what ChatGPT did really well when you first launched the first version, the better version of ChatGPT. You’re very open about the fact that it’s for research purposes and there’s a lot of things that you haven’t solved for. And I think that’s something that is a bit dangerous with companies like Google and Microsoft, when you just release it into their solutions without really notifying their users, the fact that the AI is being used and the fact that it is still something that’s developing. So I think it’s making both the consumers and professionals be aware of the fact that this is something that we’re still trying to master and make better. Obviously, they’ ll never be perfect, but we’re trying to aim for that. How Bias is detected, mitigated & annotated
Björn: I think it’s also interesting that you mentioned we need to define or discuss what is bias, right? Because that might also be very different depending on what kind of cultural background we have and things like that, which probably needs to happen before we’ll start thinking about the technical implementation. Now, there are obviously parts where I would say bias is already a kind of no, no. The EU Commission has, for example, raised the flag around credit rating models, insurance pricing models, generally pricing and product offerings being biased against gender, that this is not okay. How would you say, from a technical perspective now, if someone is using models to do these kinds of things or other things, how would you go about detecting bias? Nermeen: I think, first of all, just having a system, whether it’s something that you collaborate with another tool or just if you build it in yourself that continuously assesses and updates your models. I think that’s super important. I think a lot of people don’t actually have that first element of it and then obviously just also collaborating across the field. I think it’s not just the responsibilities of developers and people in tech to deal with this. This is something that we all should be collaborating on. The leaders, the business leaders who go out and want tech to come out really quickly, but then don’t want things to be cheaper. So do you want Indians to annotate? I think that’s a classic example. But if you let your already data set be annotated in India because it’s cheaper, then you’ll have some bias. Because of course someone in India will see the world differently than someone in Denmark. And not to say that their opinions are not as important, it’s just that we need to have a diversity of annotations, right? That’s the first element. And that goes a bit against. Even if we have the best intentions of mitigating the bias and assessing it, updating it, as long as the annotation of the data is also done with bias in it, that’s already the first warning sign. Right? And that’s, again, going back. That’s not something we can control. It’s something business leaders need to be aware of and decide that we might have to spend a bit more money and make sure that different opinions and different thoughts are being incorporated into the annotation and that we incorporate different people in it, which also costs money. Going back to what companies need to, and business leaders need to, accept that we might have to go a bit slower to make sure that the systems are at the quality level that we want it to be. Maybe I have to move more responsibly and try to make sure that people are not left behind in the process. Technical solutions: How to deal with the problems
Björn: A lot of the points that you’re mentioning are actually going quite beyond, you could say, just a technical solution, right? Where you would typically say, oh, we need to embed the equal opportunity difference measurement for this and we need to monitor that and dashboard there, which may be all fine and important to have, but it’s also a lot about the organization I’m hearing out, right? How do we start dealing with the problem? Who do we involve? How much time did we give us? Can you share maybe a bit on what are your perspectives on organizational structure that could support this kind of dealing with bias? Nermeen: I think, first of all, creating a culture where we teach about this and continuously discuss it and also welcome different opinions on it because there’s no right solution. I think because I’m talking about this, I mean, my view of the world or my solution is the best. I think we have a lot of those privacy workshops and training in a lot of organizations. I think we should have some similar elements of, like how do we want to develop code and how do we want to develop AI and how beyond operational AI, how do we want to include bias mitigation as part of a strategic move and not just as something that’s nice to have? And then obviously having a transparent culture where I think for a long time, developers are just seen as just coding something and we don’t fully understand it. I have a technical background, so I think it really helps me communicate with developers. But I think classically you would have program managers that wouldn’t really fully know what a scientist on the team or researcher or a developer does. And that’s really wrong that we have that culture where, oh, well, that’s something you’re looking at. That’s what designers are looking at. AI regulations
Björn: Going beyond, you could say, the technical profile, building up these responsibilities, creating not only awareness, but maybe also incentives, KPIs or requirements for program managers, project managers, to be aware of that. And be accountable for their decisions that they make then by advising the coders. Nermeen: Also because I think maybe that’s my ambitious take. I think everyone should be able to code and understand technology going forward. Otherwise, it’s really unfair to just put it on developers to hold accountable for a system breaking down or showing bias. It is a group effort. We always talk about one team in every organization. Well, then everyone should be a part of that. Björn: Yeah, I think that’s a really nice closing, nice statement to finish our session today. Thank you for being here. It was a pleasure. Very interesting. Thank you, everyone, for watching this episode of AI Watch. And we welcome you back next time.
