These days, we are drowning in conflicting information about the value of generative and/or agentic AI. I, myself am researching for good studies that dive into the ROI that is generated by this technology, with limited success. Most information is anecdotal, or comes from success stories, which cannot get used too literally.
Two major 2025 reports from MIT and Wharton, respectively, paint starkly different pictures of AI adoption and adoption success. While the meanwhile often quoted MIT NANDA “report” on the state of AI in business often gets quoted with 95 percent of all businesses not getting any ROI from their gen AI initiatives, a recent study by the Wharton Business School shows a very different result with 74 per cent of enterprises showing a positive ROI. Why is one so pessimistic and the other so optimistic? As I have written before, a closer look at the data reveals the 95% "failure" narrative is a myth, or even a scare, and the real story is probably a different and far more differentiated one, which Wharton names Accountable Acceleration.
Is GenAI really a 1-in-20 lottery ticket or is it rather a core business function? So, let’s have a look.
Methodology matters – debunking the 95% failure rate
In contrast to the NANDA “report” that relies on a fairly small sample of about 150 survey responses and 52 structured interviews, the. Wharton report bases on a large-scale, quantitative and longitudinal study. It surveyed around 800 senior decision-makers at businesses of different sizes and is tracking trends for the third consecutive year. Therefore, its data is built for statistically valid conclusions.
In addition, MIT NANDA’s rather sensational number of a 95 per cent failure rate is the result of a different focus. This “report” solely looks at custom, task specific agentic AI implementations that reach production state and measurable P&L impact. It does not look at the value of widely adopted generative AI tools like ChatGPT, Copilot or Perplexity. Instead, it dismisses their impact by labeling them as as tools that primarily enhance individual productivity, not P&L performance. And this, despite their obvious success that the report also identifies. 40 per cent of general purpose LLM implementations go into production with about 90 per cent of all employees using privately purchased licenses for ChatGPT, Perplexity, et. al. This is an indication that the corporate implementations are not exactly looking at the right pain points.
In contrast to this, the Wharton report identifies this as the primary source of value creation. Its data shows the top use cases are data analysis, content creation and its summarization and presentation (p. 30). This is the ROI. The MIT report is like arguing 95% of companies get no return from email because it only boosts individual productivity. This doesn’t make much sense as an increase in productivity is a return. Wharton, in turn, identified how businesses identify and measure ROI and found that almost 75 per cent of all businesses report a positive return on investments, with smaller businesses (p. 45) and some industry sectors (p. 47) seeing a higher return. But the picture is clear.
The ROI story – mainstream and measured
Given all this, the Wharton study seems to be a far more credible source of information about the current usage and ROI of generative AI. There is only a small fraction of businesses that sees a negative ROI, with some businesses seeing it as neutral or too early to assess.
The numbers vary across industries, but the general picture is clear. Generative AI is not a high-risk gamble. On the contrary, if implementations are managed appropriately, it does deliver results and consequently, Wharton concludes that we have now regular usage in core business operations, with embedded ROI metrics. More than 70 per cent of the surveyed organizations are formally measuring ROI, with a focus on productivity gains and incremental profit.
There is both, an increasing usage of generative AI and a continued significant investment, including into own research and development. Wharton dubs this phase as “accountable acceleration”, moving on from exploration- and experimentation-oriented phases in the previous years. I guess, there needed to be a catchy marketing phrase …
And, as said above, it’s showing results. Of the surveyed businesses, 74 per cent are seeing positive returns. This includes 35 per cent reporting "significantly positive ROI" and 39 per cent "moderately positive ROI". This data flat out contradicts the 95 per cent failure narrative of MIT NANDA. Apparently, we are not looking at a divide with a 95 per cent chasm but rather a far smaller spectrum of adoption speed and maturity.
Buy – or rather build?
Now that we are clear about well-managed generative AI initiatives showing a positive impact, the make or buy question looms again. NANDA quite unequivocally says “buy”, as they find that the failure rate of custom implementations is double the failure rate of strategic partnerships with a vendor and systems integrator.
Wharton roughly shows an equal distribution of efforts in new technology, enhancing existing technology and internal research and development efforts.
Which is not necessarily a contradiction, as it basically says, “buy and adapt”. Some scenarios can be supported by delivered software, some needs training or fine tuning, other models need to be specifically built.
Businesses are adopting a far more sophisticated hybrid strategy than simply make or buy. Firms are not just "buying" off-the-shelf tools; they are investing significant capital to build custom, proprietary solutions that drive competitive advantage.
The MIT report's "buy, don't build" advice, in contrast, is very simplistic. While I stick to the recommendation that I made in my article Beyond the Hype: Unlocking GenAI ROI in the Enterprise, the truth is more nuanced. It is basically the same as for the implementation of enterprise software in general. Stick to best practices where there is no significant or lasting competitive differentiator and invest into custom implementations where a differentiator with a competitive advantage lies.
The real barrier
Which leads us to what prohibits businesses from taking even more advantage from the use of generative AI. As Wharton shows, the main problem is not the technology, at least not only. There are credible studies that show a decreasing likelihood of success with increasing complexity of the modeled scenario, including CRM-Arena Pro and TheAgentCompany, but then these agree with the Wharton findings that success is very well possible.
The technology is mostly ready.
The real barrier is people and organization – essentially corporate culture. Management needs to set an example, people need to be educated and, very importantly, governance needs to be in place that both helps the users and protects the business. This requires change management. According to Wharton, the toughest challenge facing businesses are a lack of skills, employee fear for their jobs and management’s ability to constructively manage the necessary change by convincing the employees that the use of AI is not about technology grabbing their jobs.
From sensationalism to strategy
What we can confidently say is that the generative AI landscape is not as dystopian as the NANDA “report” makes it appear. This narrative is created on a too small data set and a questionable definition of “value”. It is fundamentally biased by an underlying agenda – which comes from the authors pointing a way into their solution of overcoming the technology problem. The Wharton "Accountable Acceleration" study provides the more data-driven, and optimistic view. Generative AI has gone mainstream. The return of investments can get measured, is being measured, and it is proving positive, with few exceptions.
The "GenAI Divide" isn't a chasm between success and failure, caused by poor technology.
It is rather a spectrum of maturity and of connecting investments into generative AI into strategic business KPIs. And, of course, selecting the right problems to solve. The real challenge is not to find a silver bullet that automagically solves business challenges. It's the boring work of business transformation. It is about investing into and aligning people, investing in training, setting smart guardrails, and executing a hybrid build and buy strategy.
This is the real, actionable roadmap to success.
If you want to explore, how this roadmap could look like for you, get in touch with me for an informal conversation.
Comments
Post a Comment