"to generate copious amounts of source code that looks like it came from an offshore chop shop that whip cracked a thousand underpaid programmers to complete tasks under threat of violence so they'll fake the tests and cut corners but hide it with plausible bullshit"
As if it’s only one or another. Or you truly believe horde of low quality devs without any specs can come up with better product than Claude with quality specification?
In the abstract consumer point of view a car is exactly a faster horse. They both have high up front costs, both require continuous maintenance and fuel, and they're inconvenient to store when you're not using them.
Stationary gasoline engines were already changing the farm and reducing the head of horses necessary to feed a nation. It, too, was a faster horse for them.
Anyways.. it took the Detroit police to eventually deploy the first automatic stoplight. The real innovations seem to be often found downstream of the simple increases in capacity.
That all being said, it seems to me the current crop of LLMs haven't done this, their power and training budgets do not seem to be scaling favorably against adoption rates and profit margins. Absent a significant change in algorithm or computing substrate I don't think this strategy is the leap everyone hopes it will be.
I think this is a political and economic problem rather than a technological one.
I cannot think of a more important skill than surgery to continue training humans to do and to be wary of AI robotics replacing. Sure, some surgeries could likely be automated, but the entire point of specialist surgeons is to make choices and act in a timely manner in ambiguous situations with extremely high stakes.
What happens when the robot messes up? What happens when the internet is down, or the hospital is operating under abnormal circumstances? How do you teach, train, and collaborate with human medical workers and caregivers in a world where surgeons have been replaced by robots?
Most of the excess costs for healthcare and surgery aren't the humans doing the work. I think there's a lot of other areas we can optimize first, chief among those in healthcare being the cost structure around private businesses and insurers bloating the bill with administrative costs. There's a reason every other developed nation has a single-payer healthcare system and better outcomes, and I don't think an AI breakthrough is the only plausible solution to improving costs in the US. In fact, under the current system, an AI breakthrough in medicine would likely hurt the workforce more than it would improve costs.
2/3 of the costs are already wasted. Even if your robot is cheaper, the provider will hire more lawyers, admins, facilities staff, etc to keep the budget growing. Prices have been going up 15% yoy for 20 years do you think that will stop?
This is amazing, and there should be these for every profession and job in the world.
However, I also want to be able to auto-speedrun something like this, without intentionally "playing" a game. So that I can sit back and watch what's involved in a profession, without having to make lots of decisions.
Maybe, although it's actually giving me OCD, I think. It's really hard to tune out because of the irregular ticking. I implemented a regular mode to combat this, defeating the purpose somewhat.
Unpredictable things catch our attention - it's the exceptions that are important to survival, and our brains evolved to cope with the stimuli that this experiment messes with.
Something like this would be anxiety inducing for most people, I bet. That'd be an excellent experiment, track heart rate, EEG, and performance on a range of cognitive tasks with 2 minute long breaks between each tasks, one group exposed to the irregular ticking, another exposed to regular ticking, another with silence, and one last one with pleasant white noise.
It sounded fun (and it is)! My favorite mode is one that ticks each second imperceptibly fast, and then stalls for a second in one of the ticks (so that it lasts two).
It's just the right amount of "did that clock just skip a beat? Nah must just be my imagination".
If anyone knows of a way to use this code on a consumer grade laptop to train on a small corpus (in less than a week), and then demonstrate inference (hallucinations are okay), please share how.
It's true, the post lays out the details clearly, but a hands-on example can often make the concepts more tangible. Seeing it in action helps solidify understanding.
The post lays out the steps clearly, but implementing them often reveals unexpected challenges. It's usually more complicated in practice than it appears on paper.
This. I literally am asking for a step-by-step guide outlining every step (including an existing corpus that can be used on a consumer-grade laptop to train the model in under a week).
If the implementation details are clear, replicating the setup can be worthwhile. Sometimes seeing it in action helps to better understand the nuances.
I decided to make rotis every day for a month (am male of Indian origin who hadn't ever cooked breads), AND eat them. The first one was completely inedible. The 30th day's rotis were edible, but nothing like what women in my family make. But still, edible.
Eventually had the confidence to experiment with making Naan.
This led to experimenting with Asian-style Pot-Stickers.
The main benefit to me was confidence, and belief in pmarca's "you can just do things".
Why didn't you just ask the women in your family what they did to make them? It shouldn't take 30 attempts to get a basic flatbread recipe to be edible. It's not like all the women in your family devised recipes on their own - they just watched other women make them and learned how to do it that way.
This is like asking why someone built a text editor as a hobby project when Microsoft Word exists. There is value in experimentation and play, in trying to understand things from first principles. It would often be faster to just ask others, but if everyone did that, we would miss out on a lot of innovation.
If you're looking at it solely from perspective of "value" in experimentation and play, then that value mostly comes from expert level understanding of the ingredients and the process. To not ask for guidance from people right near you who know far more than you just seems like egotism.
Who has a better chance of developing an innovative omelette dish? Thomas Keller, or someone who can't make scrambled eggs without setting off the smoke alarm?
The point is, experts can bootstrap you so you can progress quicker than you can on your own. This is why mentors exist, and is the basis of Bloom's 2 sigma study.
It really depends on what your goal is. Some people just like "play." For example, I am terrible at video games--when somebody tells me that they "beat" a game, this usually seems unfathomable to me. The last video game I finished was Riven, back when Riven was new. I still do play now and then. I play Skyrim poorly. I like walking around and discovering things in the Skyrim world. If my goal had been to beat the game in some way, then I would be going about it terribly. But that's not my goal. It does not have to be because egotism.
In my experience, dealing with flour is an art, not a science. You just have to do it over, and over, and over, until you internalize the parameters and can adjust to them on the fly. The look, the feel, the temperature, the smell, etc.
Yes, and experts can guide you with feedback far quicker than you can perform these experiments and adjust the parameters on your own. This is exactly why mentors and apprenticeships exist.
Apparently a return to the policies of 14 years ago when people like Bill Clinton, Hillary, and Obama used to call for enforcing laws against illegal immigration.
People talk about "inbox zero". I want to learn "inbox Epstein".
Somehow, this guy had time to have regular email conversations with about 1,000 of the most powerful people in the planet (the head of the Nobel Prize Committee was one).
I keep one of those Amish hand-crank drills in my vehicle toolbox. I have one in the closet too.
I have a thing for old tools, but not much can substitute a drill when one is needed. And the ones I refer to are surprisingly effective, and built to last. Borders on art for me.
reply