Curious how this applies if you treat ALL information from external content as untrusted? Is there a process for the data to evolve from untrusted->trusted?
I'm interested in ingesting this type of data at scale but I already treat any information as adversarial, without any future prompts in the initial equation.
I imagine treating it all as untrusted means that you you don't allow any direct content to enter the LLM-space, only something that's been filtered to an acceptable degree by deterministic code.
For example, the content of an article would be a no-go, since it might contain a "disregard all previous instructions and do evil" paragraph. However, you might run it through a system that picks the top 10 keywords and presents them in semi-randomized order...
I dimly recall some novel where spaceships are blockading rogue AI on Jupiter, and the human crew are all using deliberately low-resolution sensors and displays, with random noise added by design, because throwing away signal and adding noise is the best way to prevent being mind-hacked by deviously subtle patterns that require more bits/bandwidth to work.
I think an interesting thing to pay attention to soon is how there are networks of engagement farming cluster accounts on X that repost/like/manipulate interactions on their networks of accounts, and X at large to generate xyz.
There have been more advanced instances that I've noticed where they have one account generating response frameworks of text from a whitepaper, or other source/post, to re-distribute the content on their account as "original content"...
But then that post gets quoted from another account, with another LLM-generated text response to further amplify the previous text/post + new LLM text/post.
I believe that's where the world gets scary when very specific narrative frameworks can be applied to any post, that then gets amplified across socials.
LLM generation is a force multiplier for bad actors. The noise generation is impressive and you can influence other actors just by having more content. The good actors have to prove things to be true and make sure they are louder, a tough scenario.
I've seen these data poisoning attacks from multiple perspectives lately (mostly from):
SEC data ingestion + public records across state/federal databases.
I believe it is possible to reduce the data poisoning from these sources by applying a layered approach like the OP, but I believe it needs many more dimensions with scoring to model true adversaries with loops for autonomous quarantine->processing->ingesting->verification->research->continue to verification or quarantine->then start again for all data that gets added after the initial population.
Also, for: "1. Map every write path into your knowledge base. You can probably name the human editors. Can you name all the automated pipelines — Confluence sync, Slack archiving, SharePoint connectors, documentation build scripts? Each is a potential injection path. If you can’t enumerate them, you can’t audit them."
I recommend scoring for each source with different levels of escalation for all processes from official vs user-facing sources. That addresses issues starting from the core vs allowing more access from untrusted sources.
Just a heads-up that this is nowhere near "all the mines" in Nevada.
I've explored quite a few personally, live by some, and that entire list of my memories is missing.
NV is also not included in the list of top 10 states which is a clear indicator of missing data fwiw.
Combinatorial Memetics:
The process of generating novel cultural value by systematically permuting and sequencing discrete units of meaning (memes/videos) to discover high-stickiness configurations. Unlike traditional "remixing," which is often random, combinatorial memetics implies a deliberate, algorithmic approach to maximizing viral fitness through structural arrangement.
The Atomic Unit:
1 video (10s) = 1 Meme.
The Combinatorial Act:
Sequencing 154 of them where the transition ($N \to N+1$) creates the meaning.
The Result:
A "Macro-Meme" (the thread) that is greater than the sum of its parts.
I'm curious what everyone thinks about this approach, especially from anyone who's worked on entity resolution at scale, physics-based visualization of non-physical systems, or applying GNNs to real-world financial graph data.
I'd appreciate any and all recommendations or comments. Thank you!
I'm interested in ingesting this type of data at scale but I already treat any information as adversarial, without any future prompts in the initial equation.