It's important that students and teachers become educated, even just slightly, about what ChatGPT is really doing and how it works.
Fundamentally, ChatGPT operates by predicting the most likely word to follow previous words. By doing so, it generates text that largely makes sense. By itself, it does not try to give correct answers to factual questions. It simply generates likely text based on its training distribution. And it incorporates randomness to generate variability, according to the distribution, so that it doesn't just regurgitate the same answer verbatim every time.
As others have noted, there is a “temperature” parameter that defines how much randomness gets injected in its process. With a temperature of 0 it’s results would be repetitive and lack any type of creativity. With a temperature set to max it would hallucinate a ton and have wild answers.
I just tested it in the playground using gpt4 and when I used the default temperature of 1, it’s answers carried from like 303k to 305k where each answer was slightly different. Then I tested it several times with temperature of 0 and it gave the same exact result every time 303820. So the algorithm thinks that 303820 is the most correct answer, but the injected randomness makes it pick slightly less optimal answers. I don’t know if the other answers are completely out of thin air or if it’s training data just had a bunch of different sources that varied slightly.
I don't imagine the average person will approach ChatGPT with this level of understanding. My class didn't struggle with the varied output of creative prompts, "write a story". We were confused regarding factual information.
The question is very slightly ambiguous. In England there is the city of Newcastle-upon-Tyne on the east side of the country and there is the town of Newcastle-under-Lyme on the west side of the country.
Most people presume Newcastle to mean the city, unless they happen to live close to or in the small one.
Another guess: People have lived in the Newcastle-upon-tyne area for at least 1900 years. There are probably lots of web pages about historical events that mention the population of Newcastle at some point in history.
I don't know if ChatGPT allows for the difference between a mention of the current population and mention of the population after William the Conqueror wreaked the place about 950 years ago.
This plus temperature/randomness is the answer. Language models are generally non-deterministic (you don't generally use them with 0 temperature) and if the question is ambiguous, it makes the variance higher.
There's also a lot of ambiguity in the boundaries of "Newcastle" , which may or may not include Gateshead on the opposite side of the river and contiguous towns like Wallsend depending on which level of municipal boundary, traditional understanding or geographical unit you're looking at
Fundamentally, ChatGPT operates by predicting the most likely word to follow previous words. By doing so, it generates text that largely makes sense. By itself, it does not try to give correct answers to factual questions. It simply generates likely text based on its training distribution. And it incorporates randomness to generate variability, according to the distribution, so that it doesn't just regurgitate the same answer verbatim every time.