Saved RDF threads - thank you

[life]

There's no way we can copy the forum 1:1 unless we have database access to the old site.
All I'm doing is downloading the topics including all images up to 1 level of depth, so that includes all avatars as well as all images that where directly displayed in a thread. No user data saved but the nickname, post count and avatar. Anyone could claim to be this and that user, no control there.

[hackenslash]

I'd be interested to hear what nineberry has to say on this topic.

[ED209]

I have a good upload speed, and I'm happy to seed it.

I do know that one kind soul has offered hosting space with massive bandwidth, and another has suggested that a domain may just be available, so it's not beyond the realms of possibility that the forum can simply be migrated, warts and all, to another platform. I'm sure we have enough technical know-how between us to make it happen.

I wonder if that'll be possible really. It wouldn't be the same without everyone, and without giving a fuck about copyright I'd question the ethics of transplanting content written by 1000s of users in one context to another.

Anyway, I like this place and am looking forward to settling in here!

Well, all users would be invited (especially if the archive includes the user database), and the foundation has relinquished all copyright, which means that the copyright is owned by the users. We can certainly find ethical and legal ways to circumvent such issues.

It's not a big problem in the scheme of things, don't get me wrong! But gaining cooperation from the original site or an email list seems unlikely at the moment. And then say someone posted something sensitive three years ago and later left the site, is it really right to put all of that up under a new domain without their permission? It would be the greater good in a way to continue on as before, but this is already to my mind impossible due to people issues and the collective experience of what's happened, before we get onto the inevitable few out of the 85,000 who might be concerned or harmed by such a thing. Torrenting the original data in its original context out of historic affection/nostalgia seems to me the most fitting course of action - you know, since we can't carry on as before, let's grab as much of the old site as we can and move on.

[Valden]

Putting both Nine's and Life's on torrent would be fantastic. (And I'd seed)

[life]

Putting both Nine's and Life's on torrent would be fantastic. (And I'd seed)

I will definately seed once I've downloaded everything.
It's working for almost 20 hours non-stop now and no ending in sight.
I definitely underestimated the size of the forum
If I had to guess it'll probably take another day or so for it to finish.

[macdoc]

We grabbed the entire Climate News thread which was 100 pages and heavy with images.
It was 26 mb.
Used a Firefox add on.

Now looking to host it once I see what it looks like

[ED209]

Putting both Nine's and Life's on torrent would be fantastic. (And I'd seed)

I will definately seed once I've downloaded everything.
It's working for almost 20 hours non-stop now and no ending in sight.
I definitely underestimated the size of the forum
If I had to guess it'll probably take another day or so for it to finish.

Good luck and keep at it

It's great that you are doing this and lots of former members will help with the seeding/burning and mailing of DVDs I'm sure. Speaking personally, I never posted a damn thing worth reading but I'll seed seed seed to prevent all that content being lost forever.

[Gawd]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

[Valden]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

They're not mirroring, they're downloading the text from the forum.

[life]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

Cheers, I tried WinHTTrack first but it crapped out for me as well.
I'm using Offline Explorer.
It sets up a local server at port 800, so basically anything but the search function remains intact locally.
Only 139k of files to go, at this rate I think I'm done in less than 5 hours time. Seems like 10Gig of data is an approx. estimate.
And about 7-zip... I will look into it

[Valden]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

Cheers, I tried WinHTTrack first but it crapped out for me as well.
I'm using Offline Explorer.
It sets up a local server at port 800, so basically anything but the search function remains intact locally.
Only 139k of files to go, at this rate I think I'm done in less than 5 hours time. Seems like 10Gig of data is an approx. estimate.
And about 7-zip... I will look into it

Are you downloading the entire forum now, as Nine, or just those of us who were deleted?

[life]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

Cheers, I tried WinHTTrack first but it crapped out for me as well.
I'm using Offline Explorer.
It sets up a local server at port 800, so basically anything but the search function remains intact locally.
Only 139k of files to go, at this rate I think I'm done in less than 5 hours time. Seems like 10Gig of data is an approx. estimate.
And about 7-zip... I will look into it

Are you downloading the entire forum now, as Nine, or just those of us who were deleted?

42

[davo]

Hi folks,

I have a full copy on one of my servers, ~11gig of fully browsable forum, not sure if I should grab a specific domain to set it up on or put it on an existing domain.

The complete forum is browsable exactly like it is atm. ie: it's a full archive.

So question, do I purchase a domain such as dawkinsforum.org or rdfarchive.org or something? or just throw it up on one of my existing spare domains, something like dawkins.nomasters.org?

[life]

Hi folks,

I have a full copy on one of my servers, ~11gig of fully browsable forum, not sure if I should grab a specific domain to set it up on or put it on an existing domain.

The complete forum is browsable exactly like it is atm. ie: it's a full archive.

So question, do I purchase a domain such as dawkinsforum.org or rdfarchive.org or something? or just throw it up on one of my existing spare domains, something like dawkins.nomasters.org?

Awesome news!
Careful with the legal stuff though (logo and the likes...), definitely copyrighted!

[Valden]

What software are you guys using to mirror RDF? I was using WinHTTrack, http://www.httrack.com/, but it craps out after about 4 gigs. Also, when you finish the mirroring, use 7zip to compress it down, http://www.7-zip.org/. Use the LZMA format set for Ultra in the menu to get the best compression. It's offers better compression than what ZIP or RAR offers. Since it is all text, you should be able to compress the 10gigs down to about 1-2gigs if my test on my 4gigs are anything to go by. At that size, you can split it into two parts using 7zip when you compress it and upload the two parts to http://megaupload.com/ where you can host files up to 1gig each for free. Megaupload offers very fast downloads and has the largest file sizes that you can upload for free.

Cheers, I tried WinHTTrack first but it crapped out for me as well.
I'm using Offline Explorer.
It sets up a local server at port 800, so basically anything but the search function remains intact locally.
Only 139k of files to go, at this rate I think I'm done in less than 5 hours time. Seems like 10Gig of data is an approx. estimate.
And about 7-zip... I will look into it

Are you downloading the entire forum now, as Nine, or just those of us who were deleted?

42

Okay then!