I also have a tendency to say "Last century", thinking it comedically suggests "a long time ago" without it actually being that long ago. But as time goes by it obviously becomes legitimately a long time ago, and I suspect young people wouldn't see the attempted irony at all.
You can actually do that today. In fact I did that for some time, because I didn't want to configure e-mail client. The only hard thing is HTML. Average HTML e-mail is almost impossible to read and friction to extract it to a file to open in a browser is too much.
The entirety of 1999 and 2000 was a nightmare. "No, buddy, we won't change millenium next january." "Nope. We are still in the 20th century." And so on...
I think that's more or less when I lost faith in humanity.
But they were right and you were wrong. Your mistake is in thinking that years are cardinal numbers when they're actually ordinal.
The current year counting was based on the same way years were counted in the past, so you had things like "the 10th year of Caesar's reign". So the year 1 A D. was the first year of Christ's reign.
The year 2000 was the 2000th year of Christ's reign so that's what is celebrated.
Alternatively, you don't really care about that and it's called Common Era now anyway, but in that case it's entirely arbitrary. So either way, calling 2001 the New Millennium is wrong!