> reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper.
Or given a calculator. Which it's running on. Which it in some sense is. There's something deeply ironic about the fact that we have an "AI" running on the most technologically advanced calculator in the history of mankind and...it can't do basic math.
This is like saying it's ironic that an alternator in a car cannot combust gasoline when the gasoline engine is right beside it, even though the alternator 'runs' on the gasoline engine.
Or similarly having a gasoline engine without an alternator and making the observation that there's an absurdity there in that you're generating large amounts of energy, yet aren't able to charge a relatively small 12V battery with any of it. It's a very practical and natural limitation, yet in some sense you have exactly what you want - energy - you just can't use it because of the form. If you step back there's an amusing irony buried in that. At least in my humble opinion :-)
Thing is, a LLM is nothing but a prediction algorithm based upon what it trained. So it missing basic calculator functionality is a given. This is why tool usage is more and more a thing for LLMs. So that the LLM can from itself use a calculator for the actual math parts it needs. Thus increasing accuracy ...
If they were selling LLMs as “LLMs” instead of magic code-writing, answer-giving PhD replacements, the lack of basic arithmetic capability would be a given… but they aren’t. Judging a paid service using their own implied claims is perfectly reasonable.
Why is it a given? The universal approximation theorem should apply since addition is a continuous function. Now whether the network is sufficiently trained for that is another question but I don’t think it's a given that a trillion parameter model can’t approximate the most basic math operations.
I think the tokenization is a bigger problem than the model itself.
Easy to answer that one ... predictions are based upon accuracy. So if you have a int4 vs a float16, the chance that the prediction goes off is higher with a int4. But even with a float16, your still going to run into issues where your prediction model goes off. Its going to be a lot less, your still going to get rounding issue, what may result in a 5 being a 8 (just a example).
So while it can look like a LLM calculates correctly, its still restricted by this accuracy issue. What happens when you get a single number wrong in a calculation, everything is wrong.
While a calculator does not deal with predictions but basic adding/multiplying/subtracting etc .. Things that are 100% accurate (if we not not count issues like cosmic rays hitting, failures in silica etc).
A trillion parameter model is just that, a trillion parameters, but what matter is not the tokens but the accuracy as in, the do they use int, float16, float32, float64 ... The issue is, the higher we go, the memory usage explodes.
There is no point in spending terabytes of memory, to just get a somewhat accurate predictive calculator, when we can just have the LLM call a actual calculator, to ensure its results are accurate.
Think of a LLM more like somebody with Dyslexia / Dyscalculia... It does not matter how good you are, all it takes is to switch one number in a algebraic calculation to get a 0/10 ... The reason why i mention this, is because i often think of a LLM like a person with Dyslexia / Dyscalculia. It can have insane knowledge, be smart, but be considered dumb by society because of that less then accurate prediction (or number swiping issue).
Take it from somebody that wasted a few years in school thanks to that issue, it really does not matter if your a good programmer later in life, when you flunk a few years thanks to undiagnosed issues. And yet, just like a LLM, i simply rely on tool usage to fix my inaccuracy issues. No point in wasting good shoulder space trying to graft a dozen more heads/brains onto me, when i can simply delegate the issue away. ;)
The fact that we can get computer models, that can almost program, write texts, ... and do so much more like a slightly malfunctioning human, amazes me. And at the same time, i curse at it like my teachers did, and also call it dumb at times hehehe ... I now understand how my teachers felt loool
That's confusing basic arithmetic as a user feature and as an implementation requirement.
I guarantee that computer vision and email clients both use basic arithmetic in implementation. And it would be trivially easy to bolt a calculator into an email app, because the languages used to write email apps include math features.
That's not true of LLMs. There's math at the bottom of the stack. But LLMs run as a separate closed and opaque application of a unique and self-contained type, which isn't easily extensible.
They don't include hooks into math features on the GPUs, and there's no easy way to add hooks.
If you want math, you need a separate tool call to conventional code.
IMO testing LLMs as if they "should" be able to do arithmetic is bizarre. They can't. They're not designed to. And even if they did, they'd be ridiculously inefficient at it.
> Pretty sure the only thing computer vision does is math.
That is only marginally less pedantic than saying that the only thing computer vision does is run discrete electrical signals through billions of transistors.
Yes, everything that a computer does, it does using math. This does not imply that things running on the computer can do basic arithmetic tasks for the user.
> If plastic is burried it's not doing much more than it was when it was oil.
Unless it's sequestering something inside. Like water. It terrifies me to think about how much water is locked away for centuries in bottles underground.
We have plenty of water on the surface. Most of it is in the ocean. While that is not fresh water, it is where bassically all non sequestered water will end up. The limiting factor of fresh water is the how quickly the water cycle can produce it, which does not decrease if some water gets sequestered somewhere.
To the extent that net water in the water cycle is relevent, we would have the opposite problem where we are introducing far more water that had been sequestered (mostly in the form of glaciers), causing sea levels to rise. It would be great if we could find a way to sequester all of that extra water; but even putting a dent in that would nake carbon sequestration look trivial.
If underground water of that scale that terrifies you, go google "aquifer" some night when you don't want to sleep.
No, this is silly. The amount of water that human beings package for drinking is a tiny fraction of the amount we use for agriculture, which is itself an absolutely miniscule fraction of the amount of fresh water that cycles through the environment constantly (itself, of course, a tiny fraction of the water available in oceans).
Amounts of water aren't the problem. There will always be water. When people complain about water, it's not about how much there is in any one place, it's about flows: how much of this or that river basin or aquifer is being diverted for human use, potentially causing side effects like lake drying or desertification downstream. Those are important effects that need to be managed.
But Dasani bottles in land fills aren't hurting anyone.
I've thought about this too, but tbh the water sequestered in bottles should be nothing compared to what gets pushed under the mantle during tectonic plate movements, and the amount that falls into essentially unreachable and unknown aquifers.
Also in the long run that water will be squeezed out of those bottles by pressure and degradation, so I don't think that's too much to worry about.
Even if every bottle used this year was buried full, it would only be 0.06 cubic miles of water. Earth's lakes alone have about 20,000 cubic miles of fresh water, while there are millions of gallons of fresh groundwater.
Sounds like you live in a water scarce region (e.g. the southwest). If so, think about moving if water scarcity disturbs you, because there will be a lot more of that in the upcoming decades -- the problem is the number of people trying to live with the same finite resource.
Water is plentiful in many areas of the earth. Where I live, it was common just 20 years ago for sinks to just have continually flowing pipes with no faucets, since the water was diverted from streams or springs that just flow into the sea anyway. This rainy season we've had well over 200" of rain.
I think about this and drain any water bottle i pickup before either recyling or trashing them.
When you realize that a little bit of water in each bottle adds up to significant leakage from trash bags, what doesn't leak increases transport weight, and then what isn't lost landfill will live for ages (if not ruin the landfill liner over time)
- ed : are you thinking that microbes and germs and whatnot might lie there dormant until they're dug up again, unleashing some plague on unsuspecting descendants? If so, I imagine the chemicals leeching from the plastic will kill or retard their essential survival/replicative mechanisms...
Unfortunately you are doing the very thing that you accuse Musk of doing. L5 autonomy is not formalized (nor do I think it is able to be formalized) to the extent that would permit a rigorous proof showing it is isomorphic to the halting problem.
Your claim conflates a nebulous, squishy, human goal with a formally and rigorously proven mathematical problem. The only support offered is links to wikipedia and news articles, none of which help connect the two in an equally formal and rigorous fashion.
Some of us, governments included, believe that the presumption of innocence is a fundamental right.
"That it is better 100 guilty Persons should escape than that one innocent Person should suffer, is a Maxim that has been long and generally approved."
That quote is from Benjamin Franklin, one of the founders of the US government.
presumption of innocence is a legal principle that only applies to criminal court.
so i assume when youre equating the fundamental right to what governments believe, youre talking about specifically presumption of innocence before convicted criminally, and being jailed or executed.
the government does not fundamentally believe in presumption of innocence in civil law. for example, you could sue a bank for losing some of your records it's required to keep. the onus is on the bank (the defendant) to prove they did not lose those records, rather than on the plaintiff to prove that they don't have the records.
What's philosophically right about no-false-positives? There should be a debate about whether it's not more right to balance the different injustices (innocent people in jail vs guilty people going free), rather than install a very high bar for conviction as the desired outcome. Perhaps I just haven't come across the argument.
For me, it's more what's philosophically right about an innocent person enduring suffering or punishment through no fault of their own. To me, that appears to be a great (maybe even the greatest) injustice.
It's a sophist play on emotions. I don't think there's actual malice behind it, though. More that people struggle to assert themselves and provide direct, well-reasoned arguments. So, exaggerating or throwing meaning away altogether is an easy alternative. It is also particularly useful in obscuring a lack of any sort of coherent plan or proposed action(s) when one is angry/frustrated/etc.
To some extent I also believe it's natural to try to elicit an emotional response from the other "side," because then you can focus on attacking their words, tone, character, etc instead of engaging in reasoned discourse and debate. You get a bit of an automatic advantage if your opponent is already in an emotionally charged state. From what I've seen, the phrases you mention are pretty good at that too.
All of this ends up doing more harm than good most, if not all, of the time.
It is a time tested political tool to capture power by redefining the narrative. It's very effective as is the other strategy of manufacturing crises like the teaching of critical race theory in K-12 school or having to wear masks in schools.
I actually support masks in schools. There's solid scientific evidence backing it as a mechanism to help limit the spread of COVID. Until all ages are able to be vaccinated, it seems like a prudent course of action.
It's unfortunate that people are trying to turn such things into political statements.
What does "total" mean here? Is this some kind of "on a long enough timeline, everyone's dead" reasoning?
Limiting rate of spread is the same thing as reducing how many infections there are at any given time. Or, the total over a bounded time frame. I don't think "well there will still be about the same total number of infections by 2025 with or without masking" is a useful observation.
> I think it can be claimed that it limits the rate of spread, not total spread/infections.
Sure. We're probably all getting this thing at some point. But, that's why I said "until all ages are able to be vaccinated."
I really wish the US and other countries would take the Singapore route and require the unvaccinated assholes clogging up our medical system (and exhausting our highly trained/educated/caring physicians and nurses) to pay their own healthcare costs incurred due to COVID.
Having the unvaccinated directly realize some cost for their own ignorance seems highly appropriate here. Although I suppose at least some of them are paying the ultimate price with their life, which is also very unfortunate.
I believe the world becomes even worse when people are unable to speak candidly, frankly, and in their own way. Being able to extract and recognize the meaning behind someone's words, in spite of any perceived flaw in word choice or phrasing, is one of the hallmarks of a mature and intelligent individual.
This belief that controlling how people speak will somehow make the world a better place seems somewhat quixotic.
The United States of America is preserved today - intact - specifically because Lincoln valued coexisting with the confederates above almost all else. He made clear even in his first inaugural address that the preservation of the Union was paramount.
There is an incredibly important distinction between fighting against the ignorant temper tantrum of your brothers and sisters and exterminating them all.
Or given a calculator. Which it's running on. Which it in some sense is. There's something deeply ironic about the fact that we have an "AI" running on the most technologically advanced calculator in the history of mankind and...it can't do basic math.