LLMs from the trenches Bias risk management cultural differences and all that good stuff

Transcript

I've never met anyone who asked me to help train a racist model. There are bad actors out there who will use AI for nasty purposes, but they were probably nasty before AI. That's a bigger problem. AI might make things a little worse because it scales and is lifelike, but the bigger issue is how we deal with bad actors, which isn't an AI problem. In the context of business and enterprise apps, you can approach this in two ways. One is to be super conservative and not use AI until it's 100% safe, setting up a governance committee to think about AI for 12 months. That's a bureaucratic and inefficient way. Some companies do this, and I tell them to ping me in 12 months when they have a document and have read the AI act. EU bureaucrats might have nothing else to do, but in a company, you have better things to focus on. A few guidelines and risks are fine, but then you should start experimenting. This is the approach I recommend. The only way to understand AI risk for your organization is by experimenting. Risk varies significantly from one project to the next. For example, if you're a bank with a customer-facing chatbot, the risk might be giving factually wrong or offensive answers. If you use AI for real-time trading, the risk could be the model doing something insane and losing a hundred million dollars in three minutes. You need to figure out what risk looks like for your specific use case. No one can do that for you. Risk is more subtle than just avoiding bias. In healthcare, risk means one thing; in banking, it's something else; in retail, it's another. If you're customer-facing, it's different from if you're strictly internal. You need to experiment, test, and push models to their limits to see what they produce, from nasty content to factually incorrect but seemingly correct answers. For a medical chatbot, giving a serious-looking but incorrect answer that leads to someone taking the wrong medication is a significant risk. I'm less concerned with typical bias issues, which are getting better understood and fixed. I'm more concerned with business problems like incorrect facts or tone of voice. For enterprise adoption, these are the real risks. We promote transparency, but in a business setting, you need to try things out and test models. An anecdote: a customer in Poland working for video game studios to detect hacking and fraud schemes found that all the models they tried refused to discuss criminal activity, even though that was their business case. They needed models to analyze criminal activities to understand them. This highlights the need for models that are not aligned, where safeguards are off, which is a problem the open source community can solve. Hugging Face, for example, has baseline models that are not aligned and can summarize criminal conversations, which is what you want in certain cases. Risk is a moving target, and internal assessment by the company is crucial to decide which model is better and how to realign it. Being pragmatic is key. While ethics and research are important, in the enterprise world, people need to get things done and manage risk effectively.