Related story, while applying a firmware update to my Kawai CA49 piano, I bricked it due to flashing the wrong file (The process was broken, and I got desperate and tried something stupid, which bricked the piano). Claude walked me through looking for signs of life, and since OTA from the phone app wasn't working for me, it downloaded the Kawai Android APK, decompiled the Java, figured out the hardcoded key used for encrypting the firmware update. Extracted the piano firmware update, decrypted it, and then wrote a flashing script to program the piano from my laptop via bluetooth. My piano was back to working within an hour.
I can't imagine where we are headed. You understand every step of what it did and can appreciate the complexity but it'll only take a few generations for this to become something like magic to the tech priests beseeching the machine spirits for blessings
I think you're overestimating how much the average person knows about how technology operates today, or 30 years ago, or 1000. In some sense, we have been living with magic and tech priests since the Romans built the aqueducts. I wouldn't be surprised if widespread, cheap AI makes it easier for the average person to learn how things around them work, if they are so inclined.
I meet kids today who haven’t heard of Microsoft, who regularly play GTA and hand in assignments made in Powerpoint. 20 years ago I discovered that a friend didn’t know Xbox and Word were both from Microsoft. It’s really hard to understand what is common knowledge in different parts of society.
Kids today don’t even know where the files are stored or anything about partitions, drives, directory structure or even how much disk space is available.
They have some files, synced to OneDrive and do everything else fully online (Canva, etc.)
Most of them have never seen a computer with a drive other than C:
Kids today don't even know the most basic x86 assembly instructions! A whole class of third graders, and not one of them could tell me the difference between MOV and LEA!!!! Can you believe it?!?!
My son has never seen a C: drive before. Heck, we got him a Macbook Neo a few weeks ago and I don’t think he has left more than a few coding apps since then. Thankfully he isn’t using AI yet.
I don’t think much of them will be inclined to since information and “knowledge“ will be easily available via a prompt in any LLM. They won’t care to know what’s going on under the hood of anything. I mean, they already don’t want to and they have Google, Wikipedia and all sorts of things available to them.
To your point, we would think that just because we gained access to a vast mass of knowledge via web archives (such as Wikipedia), people would have become smarter overall, that no person with access to this would choose not to acquire the knowledge at hand. But here we are, surrounded by people who choose to stay ignorant. So, I imagine the same will happen with AI. The vast majority of people will keep using their chatbots and never in their lives will they solve their problems with these tools beyond asking general questions for troubleshooting.
> widespread, cheap AI makes it easier for the average person to learn how things around them work, if they are so inclined.
It looks to me that the far more common use case will be to manipulate technology rather than understand it.
The example with the synth is excellent. Today that kind of work demands somebody knowledgeable operate the AI harness. In short order, the AI may very well come up with the solution of looking online for example programs to decompile without the user even understanding what that means.
If religion and human technology are any guide, there will be a lot of this but it will never be the entire sum of human activity. Some of us are just too damn curious. We go straight for the curtain. I refuse to believe that very human pattern won’t continue.
"In the distant future, humans live in a computer-aided society and have forgotten the fundamentals of mathematics, including even the rudimentary skill of counting.
The Terrestrial Federation is at war with Deneb, and the war is conducted by long-range weapons controlled by computers which are expensive and hard to replace. Myron Aub, a low grade Technician, discovers how to reverse-engineer the principles of pencil-and-paper arithmetic by studying the workings of ancient computers which were programmed by human beings, before bootstrapping became the norm—a development which is later dubbed "Graphitics"." [1]
I’m all for the sci-fi extremes that we might lose valuable skills to cognitive delegation, but the idea that we as a society will forget how to count is… extremely stupid.
A post that lives rent-free in my head points out that a kid who is addicted to chatgpt is going to be more literate - and therefore likely better educated - than a kid who is addicted to tiktok
Has there ever been a modern time when this wasn't the case?
I mean: I can only go back so far, but I remember the 1980s well-enough. At that time, most of the new information that came into my brain from outside was sourced from public schools, newspapers, and the evening news on TV.
None of these sources were particularly unfiltered, uncensored, or unbiased. It was always an abbreviated approximation of someone else's idea of the truth.
Even in pre-modern times censorship was the norm. Heck, it wasn’t until the printing press was invented that the powers that be had to start doing it explicitly.
It's enough to make "explanation" a separate "educational" license to make it less broad used. Or disable it in some countries (this is happening already).
This is why locally running LLMs must be the future. We don't all need PHD level AIs to answer 99% of our queries, or to teach us a new thing. I'd encourage everyone to learn how to run and deploy local LLMs, even if they are not quite there yet in terms of performance.
There's a big difference between having something explained to you and developing expertise in it.
I don't see an AI-as-explainer future where expertise isn't sacrificed en masse.
Capitalism rarely supports a currently economically unproductive alternative for future good reasons.
The recent AI tech layoffs are a warning sign that corporate leaders will happily shoot their company's (and the future's) expertise to pad next quarter's financials and trust in 90% correct, but much cheaper, AI.
You jest, but I’m actually convinced education-tuned LLMs are (today) the only way education outcomes can actually improve in the AI era. As is, students are leveraging them for doing homework which makes homework useless, you want and economically need a model which can work as a 1:1 tutor with minimal supervision (and some hardware so lessons aren’t keyboard-driven).
Most kids can't use a keyboard and never will. Their Apple Pencil scribbles don't seem to make them particularly smarter.
Pen&pencil-> create something from (almost) nothing.
Stylus input-> subpar slow interface for computation.
Ipad data storage above par organisational help (no loosing lousy stuffed in bag paper).
I kinda liked the AI to transpose handwritten/drawn notes into digitally orderable artifacts. Seen a couple Show HNs. Are there any advances in the field (preferably OSS or one time purchaseable as alternatively)
(To add on to this: the utter physical imprecision of stylis pens is annoying. I can FEEL where a sharpt tip of a tool that is elongating my hand touches a surface and how it moves on a very fine scale/resolution.
Probably not a problem for people who have not developed highly sensitive sensomotor perception because they grew up with a lot of flattness in there surrounding and not much plasticity, but: my god are these things clumsy. I always want to reach for a sharpener when i use an apple pencil lol.
I've been writing code since my teens, I've studied assembly... yet the fact that _things_ start happening when I press the power button on my computer are pure magic to me and I like it this way.
I started digging a few times, but, I prefer the "magic".
- On startup processing begins at a known address, and you put the bootloader code over there. Hardware engineers can guarantee this for you.
- Every time you execute an assembly instruction, the program counter either explicitly jumps to a new location or else it just increments by 1. Hardware people can also make this happen as easily as implementing an adder.
Don't get me wrong, there are LOTS of layers between the hardware and most "useful" programs any of us will ever write. But all of them are pretty understandable. They're often not very complicated, just tedious.
Similarly for making a basic CPU that implements the logic you’re describing. In 2006 or so I made a super simple microcontroller on an FPGA for a course project. It had a whopping 256 bytes of RAM, 1kB of ROM, and I think four 8-bit registers plus a 16-bit program counter. You could only jump +/- 256 bytes. It was largely useless but also incredibly satisfying.
I'm genuinely puzzled by how you know enough about a system to even understand there is a basic assembly language, but still consider how "switching on" is 'pure magic'.
Doesn't the one explain the other ? It may be turtles all the way down, but at some point there's a fundamental turtle - be it LEA or CMP ?
Fair. I've just never come across someone who knows what assembly-language is, and who doesn't understand how a computer works. The journey of discovery is usually from the top-down, not the bottom-up.
I think it will be just like Dr. Know in Spielberg's "AI" movie from 2001 — I found it amazing how the oracle, though giving mystic-sounding obfuscated answers, was actually intelligent enough to figure out (a) what the kid was asking for and (2) give the correct answer.
It is amazing how Dr. Know projects where AI is likely to go. And a Kubrick script, no less. Even the commercial overlap, where you pump in coins as the only way to get answers. Did it not also have ads? Truly prescient.
Honestly, don't think so. That's certainly the path one might extrapolate if the next generation grows up exactly the same way as the current generation, but that's not how it works.
They will be exposed to this technology throughout childhood as their brains develop and they will develop unique ways to work with it we don't entirely understand just like GenY with cell phones and GenX with home computers. I think you deeply underestimate how adaptable we are as a species, but if you consider that we've been running the same OS and Bios as a species for the past ~40K years, perhaps you might be more optimistic?
Kids grew up on this man, they are master prompters. You’ll be asking them to fix your holoTV and your crypto phone when you’re too old to read the brainfuck.
If you have independent copies of the network learning gradients, then you’re effectively making the batch size smaller— unless you’re doing an all collect and making them sync, in which case there’s a lot of overhead
When you take a batch and calculate gradients, you’re effectively calculating a direction the weights should move in, and then taking a step in that direction. You can do more steps at once by doing what you say, but they might not all be exactly in the right direction, so overall efficiency is hard to compare
I am not an expert, but if I understand correctly I think this is the answer.
I want to also mention that the previous model was 3.7. 3.7 to 4 is not an entire increment, it’s theoretically the same as 3 -> 3.3, which is actually modest compared to the capability jump I’ve observed. I do think Anthropic wants more frequent, continuous releases, and using a numeric version number rather than a software version number is their intent. Gradual releases give society more time to react.
The numbers are branding, not metrics on anything. You can't do math to, say, determine the capability jump between GPT-4 and GPT-4o. Trying to do math to determine capability gaps between "3.7" and "4.0" doesn't actually make more sense.
He's not saying that all non fiction is bad, just that the incentives are misaligned, and to be fair at least in my experience, there are a lot of popular non-fiction books where each chapter is repetitive, and I feel the whole thing could have been written in 2-3 chapters, if publishing a 30-page nonfiction book wasn't taboo
I assume it’s easier to find an engineer who went to engineering school to learn how to build airplanes that are safe than it is to find an MBA who went to business school to learn how to build planes that are safe. (It’s not about the knowledge but about the root desire)
Similarly, I assume it’s harder to find an engineer who went into the field purely for money.
I do think on average engineers will prioritize safety (since they likely understand failure modes and production and long tail statistics better. We literally have to take engineering ethics classes), at the cost of doing a worse job at running the business. But when the business requires this level of safety, that IS doing a good job.
You need to be able to steer a large and complex organisation - being an engineer has nothing to do with that. And, yes, incentives matter, but those can be set.
No. This just says some engineers can (like some MBAs). There is no intrinsic link between being an engineer and being a great CEO. Most engineers are not executives.
This argument relies on false equivalence and isn’t even rational.
Except top executives are engineering degree holders. They are, and that’s a fact. The majority of top performing companies are headed by engineering degree holding CEOs.
But you keep making the same wrong points and trying to play devils advocate on positions that you don’t back up.
If you want to claim an intrinsic link between being an engineer and being a top performing CEO, you need to show something different anyway. For example, that the proportion of engineers that are great CEOs is higher than the proportion of MBAs or lawyers or chemist or ... that are great CEOs. Maybe that is true, but I haven't seen it.
Edit: we could also look at a narrower problem, for example: is the performance of engineer CEOs in "engineering companies" better on average than that of non-engineer CEOs in that sector?
I am not saying engineers cannot be good CEOs, just that the link between being an engineer and a good CEO is (probably) not intrinsic.
"If you have a great executive ticking all the boxes - splendid.
But there is no intrinsic link between being an engineer and being a good CEO (same holds for other disciplines, btw.). You could have engineers that qualify, lawyers, MBAs, mathematicians, physicists, ...
I think planes can still fly with the rudder loose? If the bolt falls out and it loses control, wind will push it into the neutral position and then flying will still be possible with other control surfaces? But I guess if the pilots don't know and it happens suddenly at a critical moment or if the bolt causes the rudder to get jammed, then that would be really bad. But I assume it falling out would result in the rudder loosely returning to neutral...
Planes can fly with the rudder inoperable, although with some restrictions -- you wouldn't want to do a serious crosswind landing, and you wouldn't want to stack it up with other failures, especially asymmetric engine failures.
However, that doesn't mean that planes can fly with the rudder /loose/. A significant risk in higher-speed airplane designs is that of aerodynamic flutter, where aerodynamic forces excite a vibration mode in the airframe, or a subset of it. You can find some impressive video of e.g. bending modes in sailplane wings being excited, with increasing magnitude bending until the wings are destroyed (or the excitation is reduced dramatically, or shifted to a different frequency). While aeroelastic modes get a lot of attention in flutter analysis, loose control surfaces can be much, much worse, because movement of the surface within the lash provided by the loose connections is effectively undamped.
The rudder is necessary for directional control specifically turns, and for flying straight in a crosswind.
I've heard of a few cases where applying more/less power on the right/left engines can sort of crudely achieve the same thing, and you might get lucky and get on the groud without crashing, but loss of the rudder would be a serious emergency indeed.
> The rudder is necessary for directional control specifically turns,
Only if you add a secondary constraint of coordinated turns, which are important for passenger comfort and efficiency, but not directly a safety concern. (You still need directional stability, but that's provided by the fixed portion of the vertical tail, not the rudder.)
> and for flying straight in a crosswind.
Only if you add a secondary constraint of alignment between body angle and flight path. This constraint is totally absent in normal flight -- it only comes up during takeoff and landing, where it's useful to have the plane lined up with the runway to avoid side-loading the landing gear. In the case of a known rudder failure, you'd head to an alternate where there's not much crosswind, to avoid this issue; but you wouldn't expect many issues getting there.
The third case where the rudder is actually critical is when combined with other failures, especially asymmetric engine failures. There are parts of the flight envelope where a single engine failure combined with a rudder failure would not be expected to be survivable.
In fact, the rudder does not do what new pilots think it does (it is NOT like a boat rudder at all, really, because the plane banks) that instructors will often make you practice flying without using the pedals at all.
If we have intelligent AI that can automate programming, then making really good robots will not be a problem. While not trivial, actuators and power systems are not the reason why we don’t have robots that can do all manual labor for us. Software is the reason, and the same kind of software that’s learning to code (machine learning) can also be adapted to washing dishes, folding clothing, doing craft labor or previously human manufacturing jobs.
Accelerating programming and information jobs also means accelerating the creation of robots that can do these trade jobs
Totally - but is the AI development of robots included in the timeline for what you're calling AGI? Can we get to AGI without having those robots, and then the AGI designs them?
I think they'll ultimately go hand in hand - this is more just a question of what we're defining AGI to be and whether robotics should be mandatory as part of the stated definition around doing 95% of work.
The AGI will have to run the robots, problem solving when things go wrong is what allows humans to run large organized endeavors without getting stuck, you need AGI to do that for generalized work. Before AGI robots will only be able to handle tasks with very simple error scenarios, and will still need humans to look after them for the rare cases where things go more wrong.
Recommend passing the speech-to-text narration through a round of GPT4 API to correct for any transcription errors (use some prompt giving context that it's speech to text)
Suggests there's other variables involved, like time of day taken, other supplements taken simultaneously, metabolic processes, diet, and maybe even the placebo effect.
