/g/ - Technology

install openbsd

[Make a Post]
[X]





Archives Nanonymous No.2305 [D][S][L][A][C] >>2325 >>2326 >>2328

I have in my posession 45 GB of 8chan archives (text+thumbnails) of the following boards (those marked as banned probably contains some cp so I'm going to delete them before publishing, and those marked as actively archived are kept up-to-date):
>/8teen/ (banned from 8chan)
>/a/ (actively archived)
>/animu/ (actively archived)
>/asmr/
>/aus/
>/ausneets/
>/b/ (actively archived)
>/baphomet/
>/boers/
>/christian/
>/co/
>/cow/ (actively archived)
>/creep/ (banned from 8chan)
>/danpu/
>/fascist/
>/fit/
>/g/
>/general/
>/girltalk/
>/greenbreeze/
>/hikki/
>/hwndu/
>/ipfs/
>/japan/
>/k/ (actively archived)
>/lain/
>/lewd/
>/loli/
>/n/ (actively archived)
>/newsplus/ (actively archived)
>/nnc/
>/pdf/
>/pdfs/
>/pol/ (actively archived)
>/polarchive/
>/polk/
>/polmeta/
>/poltech/
>/randamu/
>/redstick/
>/robowaifu/
>/shamedsluts/ (banned from 8chan)
>/soyboys/
>/sudo/
>/tech/ (actively archived)
>/wai/ (banned from 8chan)
>/waifuist/ (banned from 8chan)
>/younglove/ (banned from 8chan)
Anyone interested and how should I serve them? Should I just make a hidden service HTTP server with directory listing enabled? FTP server? There are 45501 files in the collection.
>inb4 make a torrent
No. Torrent exposes my IP.

Nanonymous No.2306 [D] >>2317 >>2319

Does archive.org allow anon uploading? (or anon account creation?)

Nanonymous No.2307 [D]

OP here. I just thought of rsync. Is running a rsyncd a good idea? That would allow others to keep their archives up-to-date from mine.

Nanonymous No.2311 [D]

onionshare, but probably just indexing would take ages

Nanonymous No.2317 [D] >>2320

>>2306
You're required to solve jewgle captcha, that means running niggerscript, so the answer is no.

Nanonymous No.2319 [D]

>>2306
Yes. Create an archive.org account, upload tars for each board.

Nanonymous No.2320 [D]

>>2317
You can bypass it by using the API.
There are other solutions either way, like creating mega.nz accounts and using megatools to avoid JS.

Nanonymous No.2321 [D]

>>2420
Would you like to elaborate?


Nanonymous No.2325 [D]

>>2305
Wow great, I only require the /pol/, /polk/, /polarchive/, /pdfs/, and /pdf/ archives. I'm not sure what's the best method for uploading... probably Archive.org they allow almost anyone to upload I think, but yeah you have to solve a Google reCAPTCHA in order to register.

Nanonymous No.2326 [D]

>>2305
What is the time-spam of the "archiving operation"?

Nanonymous No.2327 [D]

time-span*

Nanonymous No.2328 [D] >>2329

>>2305
I too have a meta-question: what is the value of this 45 GB archive? Like what is a good reason why we should download it? archive.org is already archiving 8chan daily. The only benefit would be lookup speed.


Nanonymous No.2329 [D] >>2330

>>2328
Archive.org almost always has incomplete archives of websites.

Nanonymous No.2330 [D] >>2335

>>2329
OK but still. What's the point of having the complete 45 GB? Spambot AI training?


Nanonymous No.2335 [D]

>>2330
Hrm, maybe scraping it for URLs?
>Spambot AI training
Or maybe seeing whether you can discriminate posts based on quality, I don't know.

Nanonymous No.2336 [D][U][F] >>2337 >>2340
File: cc39c6874a6026bc2160f2889df6e3462b3383a1eaf3e97ece0f760252680ce2.jpg (dl) (23.97 KiB)

There's a dedicated archive board >>>/ar/. But on the topic of archiving what do you guys think about archival DvD's? Hard drives constantly fail whereas with DvD+R's they are rated to last upwards of 60+ years. I've been considering buying a couple stacks of 100 discs and starting to archive anything and everything I value. I also need to learn about the best ways to get lossless compression of images and videos since DvD's have limited capacity. M-discs are allegedly rated to last a thousand years but I've read people are quite skeptical of them and most data hoarders still suggest DvD+R's for now.

Nanonymous No.2337 [D] >>2346

>>2336
4,7GB is quite small by today standards, but that depends what you want to archive. If 1080p Hi10P cartoons then better option is Blu-ray but for personal files in encrypted container that don't weight much it should be sufficient.
Other than that, they're quite fragile so you have to be careful to not damage them.
Stacks of 100 discs shouldn't be too hard to transport, but good luck hiding them unless you have lots of space.

Nanonymous No.2340 [D] >>2341 >>2346

>>2336
I've got ~100 DVDs but haven't yet used it for anything but installing Linux. I intended to use it for lightweight but important information, shit I don't want to lose, probably using a VeraCrypt (TrueCrypt) container.

Nanonymous No.2341 [D] >>2343 >>2346

>>2340
Make sure you memorize password - you don't to own random data.

Nanonymous No.2343 [D]

>>2341
Probably will hash the master password + password on DVD = key to decrypt DVD

Nanonymous No.2346 [D]

>>2337
>4,7GB is quite small by today standards
Eh, not really for me personally. I've read that consumer grade blue-ray writers aren't very good and can corrupt data. I know very little about compression but I tend to value small file sizes over having all my images and videos be the absolute best quality possible, so I think 4.7GB is manageable. It's certainly viable for archiving anything besides movies that's for sure. 100 discs is around $30 and you can store just about every document you'll ever need, pictures, stupid webm's, etc.
>Stacks of 100 discs shouldn't be too hard to transport, but good luck hiding them unless you have lots of space.
Yeah they're definitely a bit cumbersome. I wouldn't bother with it unless I knew I had the storage space. But I think everyone could do with a handful of dvds for some vital data (family photos stuff like that).
>>2340
I'm probably going to put an encrypted password manager onto one. After losing many passwords to a dead USB a few years back I have god knows how many backups of it now.
>>2341
I've been working on memorizing a 20 character random password myself.

Nanonymous No.2374 [D] >>2376

So OP are you uploading or still searching for a suitable host?

Please make it so that it can be downloaded through Tor. Better yet, manually split it up in roughly equal archives preferably under 1 GB each.


Nanonymous No.2376 [D] >>2377

>>2374
I think I'm going to make them available through both rsync and HTTP over a Tor hidden service. The rsync allows people to download the archive, and the HTTP allows people to view the contents of the archive conveniently.
However, I will need to look for a better host than my PC, since my internet is not good enough for people to download large amounts of data.

Nanonymous No.2377 [D]

>>2376
I think Archive.org is the best place to upload this.

Nanonymous No.2378 [D] >>2379 >>2403

What is the point of archiving imageboards? Aren't they meant to be temporary?

Nanonymous No.2379 [D] >>2441

>>2378
It could be useful to curate at some time, perhaps. I think the best archiving service was one of the early ones, where you voted on threads to be archived, so it only kept good threads and not the 90% of shitposts. Of course, that was a service of its time because it would likely just be gamed/botted to death today.

Nanonymous No.2403 [D] >>2405 >>2441

>>2378
There's some very valuable discussions and things posted on imageboards. I don't think anyone is particularly offended by archiving threads permanently. The temporary nature of imageboards is quite an outdated feature in all honesty. Thankfully it doesn't really come into play on good(slow) boards.

Nanonymous No.2405 [D]

>>2403
Agreed, I hate the ephemeral aspect of image boards to be honest.

Nanonymous No.2407 [D] >>2410

I2p torrent.

Nanonymous No.2410 [D] >>2424

>>2407
>25 KBps torrent downloads
yeah no, do you have a few months to download?

Nanonymous No.2413 [D] >>2421 >>2435

OP here. I've decided not to bother releasing these until 8cucks dies, since 8chan itself has an archive available. When I do that, I will make a tor hidden service with HTTP, FTP and rsync servers on it, all pointing to the archive directory. In the meantime I will have to find suitable hosting, which won't be any commercially available VPS since I don't think any of them offer lots of data capacity for low cost - which leaves a residential line as the only option.

Nanonymous No.2421 [D]

>>2413
>I've decided not to bother releasing these until 8cucks dies
Be careful how you define 8chan's "death": is it when it gets shut down? Is it when Jim sells it? Is it when it gets destroyed by normalfag redditors? What I'm trying to say is, if you're waiting for 8chan to go offline, you may have to wait a few years and maybe get hit by a bus in the meantime.


Nanonymous No.2424 [D] >>2434

>>2410
As a matter of fact it's 100-125KBps

Nanonymous No.2434 [D] >>2446

>>2424
The times I've tested I couldn't get more than 25 KB/s for the most popular torrent on some I2P site.

Nanonymous No.2435 [D]

>>2413
>OP here. I've decided not to bother releasing these until 8cucks dies
If you have been archiving for some time the great majority of the content you have is no longer available on 8chan. Just fucking upload it, what purpose does this thread serve?

Nanonymous No.2441 [D] >>2442

>>2403
The valuable discussion is drowned in the 99% percent of posts that are pure shit. Not just 90% of threads, but 90% of the posts within a good thread. Wading through that crap is sufferable when you can still respond to the few good posts, but it is completely pointless after the fact. If you've posted in a good thread that you want to keep around for your own personal interest, then archive it yourself, I say.

>>2379
Actually just had a thought. Something like ipfs, where a person can pin an individual page to keep it archived. This would make sure you're not relying on an archivefag to keep hosting something, and would give a rough idea of how good a thread is based on how many pins it has. The dirt would quickly be flushed away as no one bothers remembering it.

Nanonymous No.2442 [D] >>2447

>>2441
if only ipfs was anonymous it would be great not only for imageboards, but pretty much everything

Nanonymous No.2446 [D] >>2449

>>2434
Should've let the router warm up.

Nanonymous No.2447 [D] >>2500

>>2442
>anonymous ipfs
Freenet.

Nanonymous No.2449 [D]

>>2446
Maybe, for how long though... I only "warmed it up" for 8 hours or so

Nanonymous No.2500 [D] >>2501

>>2447
>freenet
Shit for archiving, because old files are automatically deleted after some time. Their recommended solution is to regularly reupload the file, which doesn't account for verifying integrity/addressing/etc. Also the anonymity guarantees don't exist when you first upload, so regular reuploading basically guarantees poz. The ipfs people are slowly adding tor support, which is probably much more viable.

Nanonymous No.2501 [D]

>>2500
>ipfs
IPFS is quite a good idea in theory, but it's fucked up from the start because it supports at least two conflicting hashing algorithms (sha256 and blake2b) which will inevitably cause a bunch of backwards compatibility problems later on, as well as content duplication issues.
IPFS is also written in Go, and either this is a shitty programming language or the programmers are incompetent, since the code size, startup time and general operation speeds are very bad. It should absolutely NOT take 10 minutes to hash a 100MB video file which usually hashes in less than 30 seconds with the sha256 command line tool.
Furthermore, dynamic websites using IPFS are forced to use javashit because of design specifics of IPFS. This is cancerous.
What is needed is a generic p2p protocol which runs exclusively over the Tor network. On top of that, we have a remote filesystem protocol, and, in turn, on top of that we can implement imageboards, email, the web, etc. which would fall in line with the "everything is a file" paradigm.
Of course, I personally don't have the time, skill or persistence to do this. I guess I'm an Idea Guy (tm).
Hapase plz implement my idea :3