- Many AI models aren’t as effective as they’re marketed to be, report claims
- 95% of surveyed companies have seen very little impact from their LLMS
- Specialisation is the key to successful AI adoption
New research by MIT’s NANDA initiative has claimed the vast majority of GenAI initiatives attempting to drive rapid revenue growth are ‘falling flat’.
Of those sampled, 95% of companies deploying Generative AI are stalling, “delivering little to no measurable impact” on profit and loss.
It seems to be an all-or-nothing game, as the 5% of companies who are benefiting from generative AI are excelling – these are primarily, the lead author says, startups led by 19 or 20 year olds, who have seen revenues ‘jump from zero to $20 million in a year’.
It seems the key to success with AI models is specialisation. Successful deployment is about picking ‘one pain point’ and executing this well, and carefully partnering with companies using tools.
Specialised vendors have success around 67% of the time, but internally built models succeed only around a third as often. Highly regulated sectors like the financial industry see many organizations build their own AI systems, but the research suggests the companies are much more prone to failure when they do so.
When line managers are empowered to drive the adoption, they see success because they are able to choose tools that can adapt over time.
Allocation is important too, as most GenAI budgets are dedicated to sales and marketing – but the biggest ROI was seen in back-office automation.
This isn’t the first time that research has suggested that AI models aren’t working as they should. A significant number of companies have introduced layoffs of lower level workers and brought in AI systems – but over half of UK businesses who replaced workers with AI regret their decision.
Tangible benefits from these models are increasingly difficult to find, and security risks linked with the models are concerning organisations – as well as AI models making ESG goals much more difficult to reach.
