this post was submitted on 02 Nov 2023
0 points (50.0% liked)

Self-Hosted Main

589 readers
1 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

For Example

We welcome posts that include suggestions for good self-hosted alternatives to popular online services, how they are better, or how they give back control of your data. Also include hints and tips for less technical readers.

Useful Lists

founded 2 years ago
MODERATORS
communick@selfhosted.forum
0
Web Crawler for Scripts (alien.top)
submitted 2 years ago by rfcity2@alien.top to c/main@selfhosted.forum
1 comments fedilink hide all child comments
 

I run the marketing for a website and don't have a direct connection with the web team, and we run into the problem where the web team does a website change and removes certain scripts we implemented - without notification. Is there a way to run a "script monitor" that checks external webpages to see if a specific script is loaded? And if not, to send an email?

We will always know the script names and webpage URLs. Most pages will have between 3-5 scripts. the scripts may either load directly from page source, or from a secondary js file that references it. It would be best to run the script hourly.

I'm happy to use some javascript or bash scripts with a cron job, but am also tinkering with Scrappy.

Does anyone have experience on this and some ideas on which path to start with?

you are viewing a single comment's thread
view the rest of the comments
[–] lilolalu@alien.top 1 points 2 years ago

You can let the scripts send heartbeat and monitor this with your favourite monitoring app. Or use something like

https://github.com/dgtlmoon/changedetection.io

to poll for website changes. It can even use a headless js capable browser to simulate user interaction.