My go-to for scraping "like a human" is to use headless browser automation. Dump the DOM after a few seconds to allow react jank to settle, and then parse the HTML document for desired properties.
If you do things correctly this approach should be mostly indistinguishable from an actual user. The automated browser instance is just a normal build of chrome. The biggest trick seems to be patience. If you only make a request once every 60 seconds or so, it's very unlikely that someone or something is going to care about it.
My go-to for scraping "like a human" is to use headless browser automation. Dump the DOM after a few seconds to allow react jank to settle, and then parse the HTML document for desired properties.
If you do things correctly this approach should be mostly indistinguishable from an actual user. The automated browser instance is just a normal build of chrome. The biggest trick seems to be patience. If you only make a request once every 60 seconds or so, it's very unlikely that someone or something is going to care about it.