I think this is party true due to how well their test setup is. Being able to automated regression test every feature of the game[0]. Having a comprehensive test suite also helps you as a developer to dive into the code and refactor away with confidence that you don't break functionality. Thus allowing for more aggressive and efficient refactoring and experimentation. At the end, it's not the code that is your application, its the features. Code can flow into many different ways from these features.
That's really impressive. I'm not into the game industry at all but I get the impression that automated testing isn't so much of a thing with a lot of games, and they depend on manual testing a lot, either with armies of game testers or a community (for early access games), especially for games with a lot of randomness and "emergent" behaviour.
But for an indie game like Factorio with so much development, features, interactions, etc it's almost mandatory.
I'm sure most games have a big automated testing suite though, there's too much in there nowadays to warrant manual testing.
Given how weak some AAA titles are regarding bugs (Bethesda, Horizon Zero Dawn) I highly doubt there is _any_ automatic testing being done in most AAA games, let alone indies.
Guerrilla games actually does some extensive testing on their game[0]. Like AI bots playing the game continuously to uncover bugs.
But they are a Sony first party game studio which means they have a lot of experience to gain regarding the PC platform which is more diverse than a console and requires an even more extensive testing setups.
Also, a lot of these kind of games are also highly driven by art style. Which is really hard to capture in automated test suites. Add to that the complexity of a 3D world and all its interactions and associated math.
They released a game for 50€ that regularly crashes. That is not something a good studio does. Sure, they don't have experience with the PC, but that doesn't excuse the buggy release, and I'll dare call their testing for PC garbage if it let those kinds of bugs through.
>Also, a lot of these kind of games are also highly driven by art style. Which is really hard to capture in automated test suites. Add to that the complexity of a 3D world and all its interactions and associated math.
To be honest I wouldn't know where to start testing the art stuff, but the crashes should be doable automatically relatively easily.
For Horizon Zero Dawn, are you talking about the PC version? Because I didn't encounter any glaring or obvious (certainly nothing game breaking) bugs when I played the PS4 version. I don't actually remember encountering any, but I'm sure there were some minor ones that I forgot about. I certainly wouldn't give the PS4 version as an example of how weak AAA titles are regarding bugs.
Absolutely. There's no way anybody could hand-test all of Factorio. Certainly you could play it and find bugs, but to deliberately test every single feature on any kind of regular schedule would be incredibly tedious and error prone.
[0] https://www.youtube.com/watch?v=ivLFP2eApto