I think you are getting caught in marketing semantics. The scale is all you need movement was mostly a way to funnel money to LLMs. Did Ilya ever actually believe transformers with no other improvements would lead to AGI, or did he believe that it would lead to a much more useful AI and wanted to raise money for that but found it hard without claiming it would lead to AGI? At the end of the day it is probably a good thing that so much money went into scaling recently, because it did work, as long as your measure of success is more nuanced than “did it lead to AGI”. And even then it may lead to AGI as the amount of money spent on ai research is much higher now and that new money may be what is actually needed.
My issue isn't that they are wrong, my issue is with rewarding those who are wrong. But your argument is it's fine to reward those who lie? I'm not sure how this isn't worse.
I'm sure that you're right that many people used it as a vehicle rather than being just true believers (I know some people that do), but there were also a lot of true believers.
The movement also stopped a lot of research. It has also resulted in a lot of money being dumped into companies betting on it being true. If we are in fact in a bubble (and it looks this way) then all that damage is on the hands of the SIAYN crowd.
Not being a true believer makes it better, it makes it worse. A lie is far worse than being wrong. Being wrong isn't a big issue, especially in the world of research. But lying is a major issue. It ruins it for everyone. We don't have to do this cycle of boom and bust to get things done. That's literally destructive
The issue as I see it is that google invented transformers way before they were released publicly. Clearly there were not enough resources being spent on them which is why the scale movement came about. Would google still be hoarding transformer based LLMs today without Ilya’s hype? Seems like a real possibility to me.
No one was releasing a product based on the paper though. Ilya had to go and raise a bunch of money for that to happen. Maybe I’m just more cynical and accepting of lying as the way things are done compared to you.
