Let's say a clinic currently has eleven doctors and is able to treat X number of patients per week. Let's say one of them retires, and instead of finding a human replacement, the remaining ten doctors choose an AI clinical assistant to free up 10% of their workload. Now it only takes ten AI-assisted doctors to continue serving the same number of patients as eleven doctors used to do.

This is just an example to show that there is no meaningful difference between "assisting" and "replacing". Any time an assistant, AI or not, takes some workload off somebody's plate, they have partially replaced them, and it adds up.

If the AI can't do all the things that a doctor can do, then even when it can take up the slack for one doctor retiring, that doesn't tell you anything at all about whether or not it can take up the slack for two retiring.

Right now there's more work to be done than there are doctors to do it; this means that the same number of doctors are getting more things done as the AI improves… but not infinitely more, because there's still stuff the current AI can't do, that only humans can do.

We have a lot of things where technology has fully saturated demand: in food, this is why we've got an obesity problem in much of the world[0]; in medicine, this is how we wiped out smallpox entirely, and are very close to wiping out a few other diseases entirely; in telephony, this is why video conference calls are basically free.

But in each of those fields, there are other things we still have demand for, they're not complete post-scarcity: restaurants, old age, and bandwidth costs are still a thing.

[0] not so for transport, which is one reason why we simultaneously have some people starving

2. If a clinic can use an AI tool to make doctors 10% more productive, doctors become worth more rather than less. Firms are incentivized to hire more rather than less in this scenario. What you're invoking here is the "lump of labor" fallacy. There are market conditions where increasing efficiency really does reduce quantity demanded, but it's not clear that medicine really is one. As far as I can tell, far from there being a fixed lump of medical work, the general population in most of the West is under-serviced and struggles to get reliable, timely, cost-effective access to medical expertise.

We saw this play out in agriculture, in manufacturing, and now it is starting to happen in some services. I do not understand why would we think it will be any different this time around.

The thing you're missing here is that "healthcare services" and "doctor's labor" aren't the same unit. Ceteris paribus, efficiency increases allow the price of healthcare to decrease while the price of doctor's labor increases. The thing that makes this non-contradictory is that a single doctor can now produce "more" healthcare. Economics says the opposite of what you think it does here. Increasing productivity drives expansion in market size, which drives up the ratio of value in the market to its labor inputs which drives up salaries.

Like I said, there might be important real-world reasons why these scenarios won't play out in medicine the way the theory predicts. But so far, you haven't provided any.

Manufacturing has also seen the opposite of what you are saying here. Global manufacturing production value has exploded over the last century, quite literally lifting billions of people out of abject poverty. In particular the last 3 decades of enormous per capita income increases in China have been driven by industrialization. I'm guessing you're taking a US-centric view that is exclusively focused on the local collapse of US manufacturing. This is to do with globalization and free trade, not improvements to labor efficiency.

This isn’t even considering quality of care, only quantity.

For other professions, if there’s already a glut of supply,… well, we don’t really need more ads or reality tv shows or sensational/viral clickbait.

There are quite good specialized systems for medical applications that were thoroughly tested and vetted against quite high barriers for entry.

I hate the approach of ad companies to approach medical problems. Of course you need patient data for clinical studies, but far more interesting would be to collect data that hint to medical indications and offering up this knowledge to doctors that cannot know about all of them.

LLMs probably will just grow a new generation of hypochondriacs because they certainly will never say that you are healthy if diagnostic supports ever make it into production.

PS. Big fan of this person’s recent blog posts. He’s been on a roll!

Edit: I’d like to note that your writing was fun to read—my comment was instead leaking a bad mood I had at the time.

Happy to listen to any constructive feedback if you have any, though!

This article is less of a summary of a paper and rather commentary on what the results of the paper entails. After all, Hacker News is meant for discussion :)

I will say though that I do believe that I still stand by the "exponentially more valuable" portion. I think the fact that LLMs can fluke their way into "hitting a jackpot" in terms of test coverage is exactly why they're so valuable. When you have something constantly trying out different combinations, if it hits even one jackpot, like in the paper, it's extremely valuable to the team. It's a case that could have been either non-obvious or simply too tedious to write a test for manually. I think there's tremendous value in that, especially speaking as someone who has spend way too much time simply figuring out how to test something within a Big Tech codebase (F/G) when I already knew what to test.

You CAN'T have exponential growth that is not a function of some value or variable or input.

I suppose in this case you could argue you have exponential growth as a function of the discrete using-an-LLM or not-using-an-LLM, but I've never heard of exponential growth as a function of a discrete.

Often people using the term "exponential growth" in common English don't understand what it means. Sorry.

FWIW, exponential growth as a function of a discrete variable is very common (e.g. all of algorithmic complexity), but it has to be (at least modeled as) an unbounded numeric variable.

You can't have exponential growth as a function of a binary variable.

My problem is we have no clue what those lines actually were. If it was effectively dead code, then it's not surprising that it was untested, and the LLM-generated test wouldn't be valuable to the team. We have no clue what the value of the test actually was, and using a single stat like "lines of code covered" doesn't actually tell us anything. Saying the test was "exponentially more valuable" is pure speculation, and IMO not an especially well-founded one. (Sort of like saying people who write more lines of code are more productive.)

This speculation seems downright irresponsible when the paper specifically emphasizes that this result was a fluke. When the authors said "hit the jackpot" they did not mean "hit the jackpot with a valuable test", they meant "hit the jackpot with an outlier that somewhat artificially juked the stats." I truly believe if the LLM managed to write a unusually valuable test with such broad coverage they would have mentioned it in the qualitative discussion. Instead they went out of their way to dismiss the importance of the 1,326 figure.

Some of my comments within the article are more aspirational than realistic in this case, and I've made edits to reflect that.

I want to clarify that I view this LLM as a junior dev that submits PRs that pass presubmits and other verifiable, programmatic checks. A human dev then reviews the PR manually. In this case, the LLM + its processing is used to make sure that no BS is sent out of review - only potential improvements.

In no scenario should it's auto-generated code be auto-submitted into the codebase. That becomes a nightmare really fast.

One additional question - do you forsee any issues with this application where LLMs enter a non-value add "doom loop"? I can imagine a scenario where a test generation LLM gets hooked on the lower value simplistic tests, and yet management sees such a huge increase on the test metric ("100x increase in unit tests in an afternoon? Let's do it again!") that they continue to bloat the test suite to near-infinity. Now we're in a situation where all future training data is now training on complete cesspool of meaningless tests that technically add coverage, but mostly just to cover an edge case that only an LLM would create.

Not sure if that makes sense, but tl;dr - having LLMs in the loop for both code creation and code testing seems like it's a feedback loop waiting to happen, with what seems like solely negative repercussions for future LLM training data.

Rather, I prefer to view LLMs as a junior dev that submits PRs that pass presubmits and other verifiable, programmatic checks. A human dev then reviews the PR manually. In this case, the LLM + its processing is used to make sure that no BS is sent out of review - only potential improvements.

Why not treat every LLM as a dev contributing to git such that Humans, or other LLms need to gatekeep in case something like that happens? (start by treating them as Interns, rather than Professors with office hours)

It would probably be very interesting to compare then and now.

This was an article meant to resurface learnings from something that happened a decade ago, with added images and a clear distillation so that somebody doesn’t have to watch the 45 minute presentation to understand what happened.

I’m sorry you didn’t like it.

But I hope you can see the value I’m trying to provide here, as someone myself who doesn’t really have the time to sit through hour-long presentations to learn something.

Parent comment was also valuable to me, for seeing how important it is to listen to others before jumping to conclusions about the worthiness of something.

I like the 11 million unique users a month early in the company -- simple architecture, popular programming tools and languages, small team, and likely enough ad revenue to pay the bills and get some earnings!

I wrote my code using Microsoft's Visual Basic .NET, ASP.NET, SQL Server, and one use of platform invoke. The code appears to run as intended.

I like the mention of DB (relational data base): I wrote the code using a free version of Microsoft's SQL Server. Right, it's only for development work, has some severe limits on DB size, gets expensive for a production version, also a pain since have to count processor cores, was glad to see Postgres and MySQL since have been planning to use one of those. I liked seeing the mention, and remark on power, of key-value stores (e.g., Redis) since I wrote my own key-value store, with all the data just in main memory, using two instances of the .NET collection class.

Now with TB (trillion byte) main memories am even toying with the outlandish idea of keeping nearly all the DB data in main memory with SQLite -- outlandish!

And, liked seeing the steps up in capacity. That there are likely better architectural ideas now doesn't disappoint me!

The outlines of the architectures of even huge Web server farms was really good to see. You mean Google, Facebook, Amazon, Microsoft, etc. have surprisingly simple architectures??? WOW.

You can’t please everyone.

Watch some YouTube videos or read some CS articles.

If you have a project idea, break down the next steps into tiny actionable tasks and draw out how you would implement certain features.

But when I calculated out how much an exec would make for a diluted $1B exit compared to working at FAANG for the same amount of time... FAANG turned out better many times (unfortunately).

I think preventing dilution should be one of the highest priorities for founders for themselves and for retention. Every single dilution event sets the outcome bar higher and, thus, harder to achieve.

Say you co-found with 1-2 others, so you start at 30-51%. You want to attract some good talent, so you give away 10% to early hires. Each funding round you give up 15-25%. You can see how your equity can quickly get below 10%.

Zuck managed to hold onto ~30% but your typical control is much lower these days. Musk had 28% of Tesla at IPO. Kalanick had 8% of Uber at IPO. Foley had 6% of Peloton at IPO. Splunk 3 co-founders had 5-7.5% each at IPO. Butterfield had 8% at Slack IPO.

So you may be foregoing 10 years of $700K FAANG comp for a ~$80M payday (8% equity of $1B unicorn exit).

To me it raises the question if non-founder startup equity is basically worthless. If a founder ends up diluted down to 8% at exit, what happens to the 0.5-1% shares you get promised at hire? 0.06-0.125% ?? So your first senior engineer hire into a unicorn equity might end up worth a grand $1M at IPO? Ouch. Go grind some leetcode and work for FAANG and make that in 2 years.