I wonder if they actually do it to optimize inference. I maintain a corporate AI server and one of the tricks to reduce the load was to modify the system prompt to be as terse as possible so the average response completes faster and requests queue up less often.
If you remember how crazy verbose previous gpt models were… clearly there’s something going on unrelated to cost. It would restate the same thing in different words several times and fill the output with emoji or lists.
I wonder if they actually do it to optimize inference. I maintain a corporate AI server and one of the tricks to reduce the load was to modify the system prompt to be as terse as possible so the average response completes faster and requests queue up less often.