Arguably in the theology case it matters even more! If god exists your misaligned omnipotent AI already exists and has promised you infinite torment for not believing!
It’s just totally evidence free reasoning from axioms that are chosen by vibes alone.
Why evil AI god and not the Christian god? Why not Huitzilopochtli, who demands sacrifice?
The answer is that this is the wrong question. No argumentation can be usefully made either for or against.
It matters less because in the theology case it’s a lot easier to dismiss 1. - the divine religious arguments for a supernatural god are super weak so the details don’t matter. It’s much more likely humans just making up myths.
With AI we’re seeing the capabilities improve rapidly and the arguments about why AGI is impossible or will be constrained for some reason are the weak ones.
Yes, but towards what? How do we know that, say, Transformer based LLMs are closer to superintelligence than earlier architectures?
To make such an assumption, there would need to be something that we could measure to track the process. To the best of my knowledge, there is no accurate definition of intelligence, nor superintelligence.
So how would we know where on the scale of [intelligent-------superintelligent] a given system is, or whether it even is on that scale?
It’s just totally evidence free reasoning from axioms that are chosen by vibes alone.
Why evil AI god and not the Christian god? Why not Huitzilopochtli, who demands sacrifice?
The answer is that this is the wrong question. No argumentation can be usefully made either for or against.