/wsr/ - Worksafe Requests » Thread #1525560

[21 / 2 / 1]

62KiB, 1024x766, 1024px-PS2-8MB-Mem-Card.jpg

How to archive website for offline ?

Anonymous Fri 09 May 2025 22:39:53 No.1525560 View View Reply Original Report

Quoted By: >>1525620 >>1526442 >>1526528 >>1526665 >>1527014 >>1527049

I heard HTTrack could help, but is it good? Is there better alternative? This website for example https://vsrecommendedgames.miraheze.org/wiki/PlayStation_2

Anonymous

Anonymous Sat 10 May 2025 08:43:48 No.1525601 Report

Quoted By: >>1525603

right click (on any blank space on website) > Save as

Anonymous

Anonymous Sat 10 May 2025 08:51:09 No.1525603 Report

Quoted By:

>>1525601
Yeah, but sometimes it has problems. And it saves images as just thumbnails, some javascript simply dont work and etc

Anonymous

View Same Google ImgOps iqdb SauceNAO singlefile.png, 176KiB, 1431x775

Anonymous Sat 10 May 2025 09:19:32 No.1525604 Report

Quoted By: >>1525605

singlefile

Anonymous

Anonymous Sat 10 May 2025 09:24:31 No.1525605 Report

Quoted By:

>>1525604
I tried that, but saved pages have windows (openable windows) of page basically broken. How to fix that? And it has saving images as just thumbnail (very low size) issue

Anonymous

Anonymous Sat 10 May 2025 13:22:21 No.1525620 Report

Quoted By:

>>1525560
>Is there better alternative?
Server ftp access
>alternative?
Wget

Anonymous

Anonymous Sun 11 May 2025 01:33:46 No.1525673 Report

Quoted By: >>1525927

I'll tell you the best solutions if you agree to download/seed like 200 GB of important websites that I grabbed.

Anonymous

Anonymous Tue 13 May 2025 12:45:25 No.1525927 Report

Quoted By: >>1526198

>>1525673
Important websites?

Anonymous

Anonymous Wed 14 May 2025 18:00:46 No.1526198 Report

Quoted By: >>1526201 >>1526442

>>1525927
Subdomains of booru.org, including historic data from 10 years ago. It's WARCs containing the images+tags+webpages +etc. That's 240 GB, downloaded in 2024. I did a full site grab of an important forum website (also warc format), myBB-type forum; also historic, started in the 90s or 00s, didn't see others grab it. That's 160 GB, downloaded in 2023.

I could go into more details and post links to it, but I assume you're not interested because most people aren't.

Anonymous

Anonymous Wed 14 May 2025 18:04:24 No.1526201 Report

Quoted By:

>>1526198
Anons on /t/ and /g/ would seed something like this, no?

Anonymous

Anonymous Fri 16 May 2025 06:17:12 No.1526442 Report

Quoted By: >>1526524 >>1526645

>>1525560
Httrack is good, it should work
Wget is also OK but I think Httrack is better, you can also look into https://archivebox.io/, and recommend the site to be archived by ArchiveTeam in case its in danger of shutting down or a small website https://wiki.archiveteam.org/index.php/Frequently_Asked_Questions

>>1526198
Upload it to archive.org, and post the magnet here, maybe I can spare some space on my SSD

Anonymous

Anonymous Fri 16 May 2025 17:57:31 No.1526524 Report

Quoted By: >>1526578

>>1526442
>Upload it to archive.org
Already happened, but they DELETED that (the uploader of the item didn't delete it); see the details in the linked post.
>and post the magnet here, maybe I can spare some space on my SSD
Here is 360 booru.org subdomains as of 2024-05:
magnet:?xt=urn:btih:89d11d735d8f20d04879d4ef53aa3792ae54b522&dn=booru.org_360_subdomains_2024_05&tr=udp%3A%2F%2Fudp.tracker.projectk.org%3A23333%2Fannounce&tr=udp%3A%2F%2Ftracker.mg64.net%3A6881%2Fannounce&tr=udp%3A%2F%2Ftracker.mgtracker.org%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.sktorrent.net%3A6969%2Fannounce&tr=https%3A%2F%2Ftr.nyacat.pw%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker.x4w.co%3A6969%2Fannounce&ws=http://23.157.176.87:8080/ipns/k51qzi5uqu5diwzufekv38cynmdqlh0lwic1ve0d9w5a0glpy52qjc0l77h4t5/partial/not_bafybeienn2zxpo4zfyylscjrlqj2sxaou5wpnzi7uvmcruluyzwt63szxi/

Details:
>>>/t/1362276
>>>/t/1362278

Anonymous

Anonymous Fri 16 May 2025 18:05:57 No.1526528 Report

Quoted By: >>1527006 >>1527049

>>1525560
>I heard HTTrack could help, but is it good?
Pretty sure that doesn't do WARCs, so not so high fidelity. Use it anyways, or use Wget as suggested above. Wget gives you WARC+raws or just raws, and raws are just the webpage or web file minus any server-exclusive data.
>Is there better alternative?
Even better is this:
https://github.com/ArchiveTeam/grab-site
https://github.com/oduwsdl/ipwb

Use a combination of grab-site and a WARC replay system. This way you have a replay of the webpage and associate files plus the web server's -- the website's -- original filters or any data it had on it (like HTTP headers and stuff). I suggest using ipwb (requires that you have ipfs installed), but you can use other WARC replay software. I use and could post about other helpful software to make and view offline copies of sites and pages. Such as: for cases where there's anti-downloading/anti-accessibility crap in the site which makes grab-site and wget not work to download their pages and files.

Anonymous

Anonymous Sat 17 May 2025 01:40:12 No.1526578 Report

Quoted By: >>1526645

>>1526524
Pack up everything you have into a torrent and publish it on nyaa.si and share the magnet in booru communities you can find online.

Anonymous

Anonymous Sat 17 May 2025 07:34:41 No.1526645 Report

Quoted By:

>>1526442
>recommend the site to be archived by ArchiveTeam in case its in danger of shutting down
ArchiveTeam IRC channels suck. Reddit-coded people in charge. It's OK in for limited usage.

>>1526578
>Pack up everything you have into a torrent
I already did, see the magnet link that I posted in the post you are replying to.

>and publish it on nyaa.si and share the magnet in booru communities you can find online.
In other words, talk to a bunch of people, and share it in torrent site(s). I'd rather not put in that effort, but I can post about it in one booru imageboard and in /t/.

Anonymous

Anonymous Sat 17 May 2025 13:02:16 No.1526665 Report

Quoted By: >>1526676

>>1525560
https://www.httrack.com/page/2/
used to use this during my army service. i'd download entire tech related websites with tons of articles, then consume it in my free time on any pc. used to troll people with it making them think we actually had internet.

Anonymous

Anonymous Sat 17 May 2025 14:24:33 No.1526676 Report

Quoted By: >>1526690

>>1526665
Do you still have those files? How many gigabytes?

Anonymous

Anonymous Sat 17 May 2025 15:12:03 No.1526690 Report

Quoted By:

>>1526676
I do not have them. size was manageable enough to stuff enough material for 3 weeks on my 8gb sd card.

Anonymous

Anonymous Tue 20 May 2025 09:34:34 No.1527006 Report

Quoted By:

>>1526528
Thanks a lot

Anonymous

Anonymous Tue 20 May 2025 11:40:47 No.1527014 Report

Quoted By: >>1527049

>>1525560
Install Linux then install this
https://gitlab.gnome.org/GeopJr/Archives

Anonymous

Anonymous Tue 20 May 2025 19:10:52 No.1527049 Report

Quoted By:

>>1525560
This software uses Apache's CGI + Wget + IPFS to create local offline and online copies of pages:
https://github.com/ProximaNova/ipfs-kubo-rpc-api-for-cgi/blob/main/usr/lib/cgi-bin/mem

Example = capture of https://xmr.ditatompel.com/remote-nodes/ at:
https://magic.decentralized-content.com/ipfs/bafybeidnqvkfvme7einbhapswptx2k3o2lbvrznrtrbnnctzwsnd45t67q/memento/20250520185703/xmr.ditatompel.com/remote-nodes

Problems: doesn't work with "?" in the URL (for that, use >>1526528); bug with http://12.34.56.78:8080/ sites, likely due to ":" before the port.

>>1527014
>https://gitlab.gnome.org/GeopJr/Archives
That webpage uses anubis captcha
>https://archive.is/2025.05.20-190003/https://gitlab.gnome.org/GeopJr/Archives
so probably none of that software can capture that page. Gotta use Selenium or whatever to do that (stuff I've done).

Capcode	All Only User Posts Only Verified Posts Only Moderator Posts Only Manager Posts Only Admin Posts Only Developer Posts Only Founder Posts
Show Posts	All Only With Images Only Without Images Only Spoiler Images Only Non-Spoiler Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

On these archives

Your latest searches

How to archive website for offline ?