Tom Segura has a standup bit in one of his specials about cop reality shows, and how people think talking to the cops is going to work out great for them. "Lawyer up. You can't handle that s**. Everybody's like, 'I'm gonna talk to the cops, and straighten this whole thing out.' You're gonna do 25 to life. Have fun with that, man."
> When you read plenty of papers you aren't going to read them again to cite them.
But in fact I do exactly that, exactly because experience has taught me that my memory of what is in a paper is fallible and I should at least cursorily review what I'm citing. In a few cases I've even just deleted something entirely because my premise was based on a recollection of what I intended to cite that was subtly wrong enough to fatally undermine my entire thesis.
I'm not saying you have to read an entire paper over completely every time you cite it but at least pulling it up and reviewing the parts that are informing your argument is definitely a best practice.
> I wish I could hammer one thing through the skull of every "AI SAFETY ISNT REAL" moron: if you only start thinking about AI safety after AI becomes capable of causing an extinction level safety incident, it's going to be a little too late.
How about waiting till after "AI" becomes capable of doing... anything even remotely resembling that, or displaying anything like actual volition?
"AI safety" consists of the same thing all industrial safety does: not putting a nondeterministic process in charge of life- or safety-critical systems, and only putting other automated systems in charge with appropriate interlocks, redundancy, and failsafes. It's the exact same thing it was when everybody was doing "machine learning" (and before that, "intelligent systems", and before that some other buzzword that anthropomorphized machines...) and not being cultishly weird about statistical text generators. It's the kind of thing OSHA, NTSB and the FAA (among others) do every day, not some semi-mystical religion built around detecting intent in a thing that can't actually intend anything.
If you want actual "AI safety", fund public safety agencies like NHTSA and the CPSC, not weird Silicon Valley cults.
> How about waiting till after "AI" becomes capable of doing... anything even remotely resembling that
I think it would pretty unfortunate to wait until AI is capable of doing something that "remotely resembles" causing an extinction event before acting.
> , or displaying anything like actual volition?
Define "volition" and explain how modern LLMs + agent scaffolding systems don't have it.
What people currently refer to as "generative AI" is statistical output generation. It cannot do anything but statistically generate output. You can, should you so choose, feed its output to a system with actual operational capabilities -- and people are of course starting to do this with LLMs, in the form of MCPs (and other things before the MCP concept came along), but that's not new. Automation systems (including automation systems with feedback and machine-learning capabilities) have been put in control of various things for decades. (Sometimes people even referred to them in anthropomorphic terms, despite them being relatively simple.) Designing those systems and their interconnects to not do dangerous things is basic safety engineering. It's not a special discipline that is new or unique to working with LLMs, and all the messianic mysticism around "AI safety" is just obscuring (at this point, one presumes intentionally) that basic fact. Just as with those earlier automation and control systems, if you actually hook up a statistical text generator to an operational mechanism, you should put safeguards on the mechanism to stop it from doing (or design it to inherently lack the ability to do) costly or risky things, much as you might have a throttle limiter on a machine where overspeed commanded by computer control would be damaging -- but not because the control system has "misaligned values".
Nobody talks about a malfunctioning thermostat that makes a room too cold being "misaligned with human values" or a miscalibrated thermometer exhibiting "deception", even though both of those can carry very real risks to, or mislead, humans depending on what they control or relying on them being accurate. (Just ask the 737 MAX engineers about software taking improper actions based on faulty inputs -- the MAX's MCAS was not malicious, it was poorly-engineered.)
As to the last point, the burden of proof is not to prove a nonliving thing does not have mind or will -- it's the other way around. People without a programming background back in the day also regularly described ELIZA as "insightful" or "friendly" or other such anthropomorphic attributes, but nobody with even rudimentary knowledge of how it worked said "well, prove ELIZA isn't exhibiting free will".
Christopher Strachey's commentary on the ability of the computers of his day to do things like write simple "love letters" seems almost tailor-made for the current LLM hype:
"...with no explanation of the way in which they work, these programs can very easily give the impression that computers can 'think.' They are, of course, the most spectacular examples and ones which are easily understood by laymen. As a consequence they get much more publicity -- and generally very inaccurate publicity at that -- than perhaps they deserve."
LLMs are already capable of complex behavior. They are capable of goal-oriented behavior. And they are already capable of carrying out the staples of instrumental convergence - such as goal guarding or instrumental self-preservation.
We also keep training LLMs to work with greater autonomy, on longer timescales, and tackle more complex goals.
Whether LLMs are "actually thinking" or have "volition" is pointless pseudo-philosophical bickering. What's real and measurable is that they are extremely complex and extremely capable - and both metrics are expected to increase.
If you expect an advanced AI to pose the same risks as a faulty thermostat, you're delusional.
It’s also already becoming an issue for open-source projects that are being flooded with low-quality (= anything from “correct but pointless” to “actually introduces functional issues that weren’t there before”) LLM-generated PRs and even security reports —- for examples see Daniel Stenberg’s recent writing on this.
"...There are fencers that seriously train that can barely score a point on me. I can barely score against the top guy in Australia. That guy can barely score against someone trying to make the Olympics, and that guy can probably barely score against the guy that actually won the whole thing."
Given that part of those "safeguards" were enshrining weird AI doomer logic in a federal agency's mission and trying to put a doomer in charge of enforcing it, some of them could use dismantling.
Not necessarily (although that doesn't necessarily mean I think this is OK). Payment-card-based verification is a longstanding method of doing prima-facie verification like this. When you give your credit card, you give your billing address and typically your phone number -- if the postal code is a US address and the phone number is a US area code and everything else is consistent with that, that might be all the KYC required. If you appear to be a foreign national operating outside the US, they can flag that and require additional paperwork only then.
This proposed rule looks to me like it basically requires providers to come up with their own verification plans, which may then differ from provider to provider, so as to be "flexible and minimally burdensome to their business operations".
[note for the following: I am not a lawyer. The following is not legal advice. Do not fold, spindle or multilate. Do not taunt Happy Fun Ball.]
The real danger, I think, with things like this is, there's an executive order that was issued, but it further specified a rulemaking process be conducted to determine the actual regulations that define compliance. The link in the title is to the proposed rule. There's nothing that says any amount of prior public input will necessarily influence the details of the final rule, or that rule can't change in the future through another rulemaking process, and if it does the only way to challenge it is either to sue the agency on the grounds that it exceeded its discretion (e.g. by making rules that require unconstitutional things) or that the enabling executive order is itself unconstitutional -- but these kinds of federal cases have a pretty high bar for what's called "standing" (the legal grounds to bring a particular lawsuit): you pretty much have to suffer concrete harm or be in obvious and imminent danger of suffering it to a grievous degree. (This is one reason you hear about "test cases" -- often somebody will agree to be the goat who is denied something, fined, or even arrested and convicted of a crime, so that standing to sue to overturn the law can be established.) Other times, if a lot of potential defendants already have standing, a particularly sympathetic defendant will be selected for the actual challenge. The US federal courts are also deferential to "agency discretion" by default, as a matter of doctrine.
What happens all too often with these things is, the initial rulemaking is pretty reasonable, and the public outrage (if there was any) dissipates. Then three years (or however long) on, the next rulemaking imposes onerous restrictions and strict criteria, and people suddenly (relatively speaking) wake up and find they're now in violation of federal regulations that they were in compliance with last week. (This is one reason public-interest groups are so critical -- they have the motivation and sustained attention to comb the Federal Register for announcements about upcoming rounds of rulemaking on various topics.)
Analyst consensus I've seen on long-term price has been floating around $32-34 per share. Take that with as much salt as you think it needs but it's at least interesting that it's within shouting distance of (but not over) the IBM offer.
