We've decided to go this route because based on billions of web-scraped page, Headless-based scraping is still a minority. And, it's way harder and more expensive to do at scale.