Something I’ve realized lately is that AI makes some things so easy (like video generation) that you’re not required to do the hard work of planning and reflection that might occur with a more involved task. We’ve always been susceptible to rushed thinking but AI compounds this significantly.
I am open to changing my mind and I am looking forward for answers to your question - but I think it is like a click bait - looks interesting - but not really useful.
I am building my self-hosting llm-wiki system (https://gist.github.com/karpathy/442a6bf555914893e9891c11519...). My approach is to start with a theory of how such systems could work. Then since llms can interpret theory - this theory becomes an executable llm-wiki system itself.
"In the 890s, having recently converted to Orthodox Christianity, Boris ensured his church would be independent from the Patriarchate of Constantinople." --- I thought Orthodox Christianity was created by the Great Schism in 1054.
The orthodox would argue that it was catholicism that was created by the great schism. But for both sides the actual date of the schism matters little, the divide between east and west was already forming long before then. Tsar Boris the 1st actually sent letters to the pope in Rome and the patriarch in Constantinople asking both of them for Christian teaching, it's debated wether he did this cynically or innocently, but either way he was "playing both sides" even before the schism, if anything he accelerated it somewhat.
I concur - it does not make sense to do in llm prompts what can be done in code. Code is cheaper, faster, deterministic and we have lots of experience with working with code.
That's why you should just subscribe to multiple LLM vendors. One model to write specs, one to write code against the specs and another to validate the code. Problem solved. (I have heard this proposed at work.)
First you need to write these specifications and if you say just tell the llm to write them - then how would it be different from just tell the llm to write the program?
I guess you can argue that these are two independent processes so you can combine them to get something more reliable than both - this might be a viable path. But from what I heard writing formal specifications is just really hard - I haven't seen anything practical in this area.
reply