How do you get it to work with sites that are javascript-heavy or have heavy bot restrictions? I've been trying to make something similar but spent weeks just fighting edge cases on HTML not caching correctly.
I don't worry about it and just use a straight-up HTTP GET. That works a surprising number of times. If I was trying harder I would use Puppeteer or similar and try to grab a full height screenshot. Most of the time I'm just clicking the URL to get back to the page.
