This is an automated archive made by the Lemmit Bot.
The original was posted on /r/selfhosted by /u/bluesanoo on 2025-05-11 16:20:46+00:00.
Over the weekend, I have worked to fix several bugs, along with add a few requested features to the app.
- Added the ability to collect media from scraped sites (videos, photos, pdfs, docs, etc)
- By using the "Collect Media" option on the submitter, whenever the scraper hits the site, it will attempt to download and save all media found on the page.
- This could be useful for collecting images for training data, monitoring a webpage for new pdfs/docs, etc.
- Disable registration, and add a default user (optional)
- Added Cypress e2e testing in the pipeline (authentication, submitting jobs, navigation)
- Plan to add more e2e tests as features are developed
Bug Fixes:
- Worker not starting up
- AI chat job selector not loading in jobs
- Authentication being a little finicky
Github Repo: https://github.com/jaypyles/Scraperr
New Collect Media Option
Optionally Disabled Registration
