Have you considered that it's unsolvable? Or - at least - there is an irreconcilable tension between capability and safety. And people will always choose the former if given the choice.
in a pure sense no, it's probably not solvable completely. But in a practical sense, yes, I think it's solvable enough to support broad use cases of significant value.
The most unsolvable part is prompt injection. For that you need full tracking of the trust level of content the agent is exposed to and a method of linking that to what actions it has accessible to it. I actually think this needs to be fully integrated to the sandboxing solution. Once an agent is "tainted" its sandbox should inherently shrink down to the radius where risk is balanced with value. For example, my fully trusted agent might have a balance of $1000 in my AWS account, while a tainted one might have that reduced to $50.
So another aspect of sanboxing is to make the security model dynamic.
> The issue title was interpolated directly into Claude's prompt via ${{ github.event.issue.title }} without sanitisation.
How would sanitation have helped here? From my understanding Claude will "generously" attempt to understand requests in the prompt and subvert most effects of sanitisation.
I would not have helped. People are losing their mind over agents "security" when it's always the same story: You have a black box whose behavior you cannot predict (prompt injection _or not_). You need to assume worst-case behavior and guardrail around it.
And yet people keep not learning same lesson. It's like giving extremely gullible intern that signed no NDA admin rights to your everything and yet people keep doing it
What was the injected title? Why was Claude acting on these messages anyway? This seems to be the key part of the attack and isn’t discussed in the first article.
Because that's how LLMs work. The prompt template for the triage bot contained the issue title. If your issue title looks like an instruction for the bot, it cheerfully obeys that instruction because it's not possible to sanitize LLM input.
Because the standard for configure scripts says that the current directory at invocation is the build directory and the location of the configure script is the source directory. You are expected to be able to have multiple build directories if you want them. If you have written your configure script correctly, than in-tree builds (srcdir == builddir) also work, but most people don't want that anyway.
You can. But this makes intent clear. If you clone a git repo and see build/ with only a gitkeep, you are safe to bet your life savings on that being the compiled assets dir.
>I'm not saying legislation is a good solution but you seem to be making a poetic plea that benefits the abusers.
Only if you believe everyone else has no agency of their own. I think most people outgrow these things once they have something more interesting in their lives. Or once they're just bored.
Back when this thing was new, everyone was posting pictures of every food item they try, every place they've been to etc.. that seems to slowly change to now where there are a lot more passive consumers compared to a few polished producers.
If you're calling people delivering the content "abusers", what would you call people creating the content for the same machine?
But I do believe we overestimate our own agency. Or more importantly society is often structured on the assumption that we have more agency then we actually do.
where does it stop though? I suffer from cant-stop-eating-nutella but should we shut down ferrero? it is simply not possible to protect the vulnerable in a free society. any protection only gives power into
the wrong hands and will eventually get weaponized to protect “vulnerable” (e.g. our kids from learning math cause some ruling party likes their future voters dumb)
Dumb argument. They don’t intentionally make Nutella addictive and then test out recipes on the public to make it even more addictive. Other people can’t stop eating ice cream or oranges or salami.
The food industry has pretty much invented the whole process of making "addictive" products and then "test[ing] out recipes on the public to make it even more addictive". Of course, we usually call it making products that taste good, and running taste panels with the public for product development (making a new tasty thing), quality control (ensuring the tasty thing stays tasty), and market research (discovering even tastier things to make in the future). Each part of it employs all kinds of specialists (and yes, those too - nutrition psychology is a thing).
The process is the same. The difference between "optimized for taste" and "addictive" isn't exactly clear-cut, at least not until someone starts adding heroin to the product (and of the two, it's not the software industry that's been routinely accused of it just for being too good at this job).
Not defending social media here in any way. Cause and effect is known these days, and in digital everything is faster and more pronounced. And ironically, I don't even agree with GP either! I think that individuals have much less agency than GP would like it, and at the same time, that social media is not some uniquely evil and uniquely strong way to abuse people, but closer to new superstimulus we're only starting to develop social immunity to.
I would say the core problem is that we lack a goal as society. If you only care about making money stuff like this happens regardless how many regulations you do.
"Dumb" and "insane" are thoughtless and shallow positions to take.
It's fine to disagree with the EU's stance (I probably do. I'm not sure yet) but it's not a good look to dismiss it without some recognition that a reasonable person might think this is a worthwhile position to take given the known harms of social media.
It's important to remember that "games" aren't a monolith. People want different things. There's a categorization I vaguely remember and I was an "explorer". Hated narrative, puzzles and any repetition. Never wanted multiplayer. Just give me a weird world to explore. Yeah, procgen has limits but Minecraft is still fascinating to me.
I'm a senior engineer and have no degree. I never get offended by people making comments like this. If we're both in similar roles, making quality contributions, and are progressing in our careers, the only difference between us is, I didn't spending 50k-100k on a degree.
Sounds more like a knock on the person making the comment than it is on me.
I fit the same criteria. I think college is probably a wholesome experience, but I don't really know, as I only went for two years and didn't really get much out of it.
I had a few major issues with the experience:
One: It was force-fed to you in High School, it almost seemed like there was no other choice at the time, and it was far too easy to go into massive debt at such a young age.
Two: I was already self-taught in computer science, and the coursework didn't really expand upon my knowledge any.
Three: The bureaucracy was insane, having to deal with Student Aid, registration, and signing up for classes. It was nauseating.
Four: While there were some interesting classes in other domains of knowledge, the fact that there were so many required courses, like Writing and "English Composition." Kind of soured the experience. I didn't learn anything in the Comp Sci classes, and probably 60% of the other stuff I wasn't interested in. As an Adult who's paying tuition, you should be able to 100% pick and choose what courses you want to take, but because I was only 19 and fresh out of high school, that liberty didn't really dawn on me until after I had finally left.
I went to a community college. I assume a four-year school or something more academic by nature would be interesting, but not worthy of hiring one person over another strictly on credentials.
Have you considered that it's unsolvable? Or - at least - there is an irreconcilable tension between capability and safety. And people will always choose the former if given the choice.
reply