LLMs from the trenches Closed model builders have decided for you

Transcript

You don't know how they've been aligned. That's the thing. No names, but all those closed models have amazing capabilities. Some I like more than others, but it doesn't matter which ones. Their creators have decided for you. They have curated the dataset, designed the alignment process, and written the prompts and system prompts. If that works for you, great; it saves you time. However, it might stand in the way or sometimes become extremely stupid. We saw a closed model from another company generate historically incorrect figures. It was good fun to watch, and it was obvious those were wrong. The problem is, it could be more subtle. The human element is crucial. Everybody wants to fight bias and have safe models. But if the workforce aligning the model is not diverse enough, they might introduce another bias. It's a complex problem. The road to hell is paved with good intentions. Bias, risk, and alignment are tricky. From one country to the next, the context is very different. For example, in Singapore and Dubai, the perspectives can differ from those on the US West Coast. People in other parts of the world might not like some of your answers. They might build their own models, which will give them better language support. When you travel, you realize the importance of respecting cultural differences. In Rome, do as the Romans do, or don't travel. This applies to AI models too. There are differences across countries. Language support is an obvious one. Not everyone speaks English or one of the three major languages. For instance, in Indonesia, there are over 100 dialects. Similarly, India has many official languages. Big models can do an okay job with the most important languages, but for high-quality solutions in regions like APAC, India, or Eastern Europe, you often need to build your own models. A Singapore customer, for example, is working with AWS to build a regional model through AI Singapore. Cultural differences are also significant. In some parts of the world, people don't want a Western or US West Coast view on their country. They want their own perspective on their culture, religion, or history. Models need to account for these differences. If a model doesn't sound like a human expert from the local context, it won't achieve mass adoption or the highest quality. Local users need to build their models. You can't expect people from the US West Coast to understand your country and culture as well as you do. Public sector stakeholders need to own this process and partner with tech companies to speed up their efforts, but they must be in charge of the data and evaluation. Decentralizing the development of these models is crucial. We shouldn't have just three companies in the world building them.