LLMs from the trenches First and foremost it is a business discussion

Transcript

You've been speaking a lot with people about the democratization of Gen AI and LLMs. With everything that is going on with Gen AI nowadays, I'm wondering, how are you holding this C-suite conversation with the vast spectrum of customers that you work with? From startups to enterprises, what do you think these business and technical decision-makers have as their priorities? What makes them tick? That's a really good question. First and foremost, it is a business discussion. I'm an engineer, so I'm as big of a geek and bit head as you would find. But it is a business discussion, and technology makes no sense whatsoever, AI included, if it doesn't solve real-life business problems or improve the productivity of your organization or the customer experience, whether that's in education, healthcare, retail, or any other sector. This has been the starting point of all those discussions in the last 18 months: What can I really do with AI? There is certainly a lot of noise out there, and some organizations are extremely vocal about their AI efforts. There is such a thing as marketing BS, with claims that a model can do all sorts of things. Many organizations and stakeholders know better than that, but if you're running a company or large teams, you cannot spend a ton of time testing everything available or validating or invalidating all those claims. That's where I try to step in and be as neutral and data-driven as possible. When I was at AWS, if I thought the AWS account team was pitching something wrong for the customer, I would call it out loud in the first five minutes. I would try to convince the account team before the meeting that they shouldn't pitch it, but if they didn't, I would call them out in front of the customer, which led to interesting discussions and follow-ups. I'm doing the same here. If I think you can solve a problem with a closed model in five minutes, then go solve the problem with a closed model in five minutes. Don't try to use open source if it's too complicated. The right solution, the right tool for the job, is always my approach. First and foremost, it is a business discussion. Not all problems out there are AI problems, so don't try to use AI when it doesn't make sense. Don't use a larger language model if it doesn't make sense. Many ML problems, which I prefer to call them ML rather than AI, can be solved with simpler transformer models or traditional machine learning algorithms from 10 or 20 years ago. If you don't do that, you get too excited about tech and lose focus on cost and performance. At the end of the day, you won't get any ROI and will conclude that ML and AI are just BS. I try to provide clarity. Many understand this and just need guidance on the best option. Some are still flying high and need to land softly. Hopefully, I can give them that soft landing so we can have the right discussion about the tooling.