Despite advances in technology and increased spending, the number of new drugs approved per billion dollars spent on research and development has halved approximately every nine years since the 1950s. This trend was first identified in 2012 and humorously called Eroom’s Law (Moore backward).
While Eroom’s law is specific to drug discovery, the exponentially diminishing returns can be found everywhere. Some thoughts:
The Low-Hanging Fruit. Once the easy problems are solved the last 20% can take much longer to solve. In drug discovery, new drugs often are just incremental improvements. Smaller improvements mean larger clinical trials and more scrutiny against already-working drugs. In software, that’s the first optimization.
Increased regulation. This point is obvious when looking at the regulatory agencies for drugs (people’s lives are at stake). But it’s also true of software — antitrust laws, data privacy laws, and other industry regulations that weren’t in place when large technology companies were started.
Mythical man-month. More money and more research don’t automatically translate into more results. There might be more structural issues with the industry that prevent progress from occurring. However I wonder why the advancements in computing haven’t translated to computational biology (e.g., in computer science, we’ve found that you can just throw more computing power at problems to find breakthroughs).
You probably want to read what is working in big pharma like, stories from employees might help you divise the scaling culture.
When it's hard to make progress... Nobody expected quick or cheap progress
Do you happen to know what this looks like if it's inflation-adjusted? Probably still pretty stark, but much closer to where it was.