Hard Archivefag here.
I'm working on a new file, but had some challenges because
rentry.org seems to have shit the bed, and the .co site is behind Cloudflare and my scraper wasn't working without some finagling.
I did notice that since the last time it ran, a couple dozen stories more ended up being taken down, although some of those are either title changes/adjustments in the archive or new URLs Since v2, I don't check for 404s if the story is IN AA's list, but already has a hard copy available. But what it currently doesn't do is the opposite - look for a story that HAS been archived, but was de-listed from AA's list.
There's a process to find those but besides creating the warning file of what was there in the previous run versus what is there now, it doesn't do much. I guess the options are:
1) Leave it as-is, if someone's looking for a story, they can find the URL/hard file from the delta list.
2) Create a separate Index file like a "lost and found" of sorts
3) Merge the delta and master files and let the main process build the Index with existing hard files, or if they don't exist, mark them as 404/inaccessible (this one would cause some duplicates, Auntie's Little Secret for example comes to mind, or any stories whose title formatting changed between v4 and v5)
Also >15 minutes to wait for a post
Hiromoot you filthy jew
