Saved RDF threads - thank you

A forum to talk about other sites and things you've found in the jungle that is the internet.

Please take a moment to read the rationalia guidelines: http://rationalia.com/forum/viewtopic.php?f=3&t=3449
Post Reply
User avatar
Blondie
Forum Desperado
Posts: 1196
Joined: Fri Feb 26, 2010 3:26 pm
Contact:

Re: Saved RDF threads - thank you!

Post by Blondie » Sat Feb 27, 2010 8:20 pm

NineBerry wrote:Status update:

All downloads are finished. I have:

79,507 threads with
2,274,472 posts by
26,581 different users

All the data is kept on the partition, where I also keep my porn (because this is my biggest partition). So, you can be certain, the data is safe :demon:

Here is an automatically generated list of the threads I have: http://www.neunbeere.de/rd.net/Threads.html (5.5 MB)
Here is an automatically generated list of the users I know: http://www.neunbeere.de/rd.net/Users.html (1.5 MB)

The links to User pages already work(soon, are now being uploaded), so you can see what threads every user has participated in, but the actual threads are not there yet.

Here is what I will do next:

1. Adapt the actual forum files to fix internal links and remove content that is not needed. The code for that already exists, now it just needs to process all data. This might take up to a day or a bit more.
2. Then package everything together and give to anyone who wants it.

I will offer all data together as one big download archive. I estimate, this will be at 400-600 MB.

The unpacked version will be at approx. 4 to 6 GB. I don't have a server with enough space or bandwidth to host the unpacked version. But whoever has the capacities can take my archive, unpack it and host it on his server. This way, people could browse the forum to read threads and also link to threads. Also, this way, google could then index the content and one could use Google Search to search the site.

Awesome NineBerry! Creating torrents is really easy, your torrent program should do it for you.. it creates the torrent file which you then upload to piratebay or whatever.

Thanks for doing this :)
In this world there's two kinds of people: Those with loaded guns and those who dig. You dig.

When you have to shoot, shoot, don't talk.

Happy Trails. :)

User avatar
NineBerry
Tame Wolf
Posts: 8950
Joined: Thu Feb 26, 2009 1:35 pm
Location: nSk
Contact:

Re: Saved RDF threads - thank you!

Post by NineBerry » Sat Feb 27, 2010 11:56 pm

Here's another update.

As you have read above, I have all the data on my hard disk. This is now being processed so that internal links work and the interface looks nice and clean and there are no unneeded elements.

My computer is processing the data but will need maybe another 24 hours before it is finished. To give you an impression how the end result looks, I have uploaded some of the finished threads, so that you can get an impression what the end result looks like:

http://www.neunbeere.de/rd.net/Pages/1531-1.html
http://www.neunbeere.de/rd.net/Pages/109237-1.html
http://www.neunbeere.de/rd.net/Pages/110247-1.html

I don't have enough space on my server to host the complete forum, so I will instead (when all processing is finished) offer the website in a packed format for download over a third party (Dropbox). Maybe, someone with enough webspace will then take the data and set-up a live browsable version on some public server.

User avatar
Hermit
Posts: 25806
Joined: Thu Feb 26, 2009 12:44 am
About me: Cantankerous grump
Location: Ignore lithpt
Contact:

Re: Saved RDF threads - thank you!

Post by Hermit » Sun Feb 28, 2010 12:02 am

I've had a look. You're doing a fantastic job there, Neunbeere. :tup:
I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops. - Stephen J. Gould

User avatar
lordpasternack
Divine Knob Twiddler
Posts: 6459
Joined: Thu Feb 26, 2009 10:05 am
About me: I have remarkable elbows.
Contact:

Re: Saved RDF threads - thank you!

Post by lordpasternack » Sun Feb 28, 2010 12:07 am

I cannot express in words my debt of gratitude to the hard work of all you guys who are working to archive that forum. I think we should club together and maybe get some small gifts for those concerned. You really are gems. :cheers: :hugs:
Then they for sudden joy did weep,
And I for sorrow sung,
That such a king should play bo-peep,
And go the fools among.
Prithee, nuncle, keep a schoolmaster that can teach
thy fool to lie: I would fain learn to lie.

WhostolemyID
Posts: 21
Joined: Wed Feb 24, 2010 4:49 pm
Contact:

Re: Saved RDF threads - thank you!

Post by WhostolemyID » Sun Feb 28, 2010 12:10 am

Wow - list of users works, well laid out, it just... works!

Can we wait 9 months for you (not) to bring in a search function?

Again, thank you for rescuing that data! :flowers:

Gifts? you can have my soul... well, you could've, but I gave it to :mob:

User avatar
hackenslash
Fundie Baiter...errr. Fun Debater
Posts: 1380
Joined: Sun Jul 19, 2009 5:05 am
About me: I've got a little black book with my poems in...
Location: Between the cutoff and the resonance
Contact:

Re: Saved RDF threads - thank you!

Post by hackenslash » Sun Feb 28, 2010 2:13 am

Dude! That's brilliant!

Image

A well-deserved Orson. In fact, have another!

Image
Dogma is the death of the intellect

User avatar
fuck you & fuck your god
Posts: 103
Joined: Mon Feb 22, 2010 9:41 pm
About me: aka mechtheist
Off my meds again
Location: No where near Uganda
Contact:

Re: Saved RDF threads - thank you!

Post by fuck you & fuck your god » Sun Feb 28, 2010 4:09 am

NineBerry wrote:Here's another update.

As you have read above, I have all the data on my hard disk. This is now being processed so that internal links work and the interface looks nice and clean and there are no unneeded elements.

My computer is processing the data but will need maybe another 24 hours before it is finished. To give you an impression how the end result looks, I have uploaded some of the finished threads, so that you can get an impression what the end result looks like:

http://www.neunbeere.de/rd.net/Pages/1531-1.html
http://www.neunbeere.de/rd.net/Pages/109237-1.html
http://www.neunbeere.de/rd.net/Pages/110247-1.html

I don't have enough space on my server to host the complete forum, so I will instead (when all processing is finished) offer the website in a packed format for download over a third party (Dropbox). Maybe, someone with enough webspace will then take the data and set-up a live browsable version on some public server.
I will plan on doing this, no problem, have enough space to hold 10 copies, and with unlimited bandwidth, and we can do the torrent approach, that should be able to satisfy most. Tell me what to do when you are ready and I will do my best to have it running ASAP. My assumption as to what you are saying is that you will have available in the sharable area of your Dropbox space a compressed file that when unzipped, and copied to my web space, will be ready to go?

For you torrent guys, I have muTorrent (can't get a 'mu' ), shows this result in setup test:
Image

This should be a decent connection, but have no real idea how it compares, what do you guys say? I am looking into how to create the torrent files from what NineBerry uploads. If I am missing something, displaying extreme ignorance of what needs doing, please tell me, I'm afraid I might be one of those with just enough knowledge to be dangerous. :oops: Thanks for any input.
Image Image Image
Is life not a hundred times too short for us to stifle ourselves--Nietzsche

anthonzi
Posts: 67
Joined: Wed Feb 24, 2010 7:09 am
Contact:

Re: Saved RDF threads - thank you!

Post by anthonzi » Sun Feb 28, 2010 7:31 am

hackenslash wrote:If several people have the torrent seeded, then users can download from multiple locations, each location seeding a portion, meaning quicker download and less strain on the seed.
Of course, but they still need to acquire it beforehand. And the cheapest way for them to do that is usually through p2p. And I don't think there will be hundreds of people starting it up as soon as it's released. It really just depends on how well the direct download server (dropbox) performs. Also, is dropbox free for downloaders? (https://www.dropbox.com/pricing)
(editphail)
fuck you & fuck your god wrote:I have muTorrent (can't get a 'mu' )
Just call it utorrent lol. :D
fuck you & fuck your god wrote: This should be a decent connection, but have no real idea how it compares, what do you guys say? I am looking into how to create the torrent files from what NineBerry uploads. If I am missing something, displaying extreme ignorance of what needs doing, please tell me, I'm afraid I might be one of those with just enough knowledge to be dangerous. :oops: Thanks for any input.
Your connection looks pretty average for an American cable connection, so it should be fine for a 1GB file (he did say it was only going to be like 700MB compressed right?) :cheers: . But as I mentioned above, if our DDL services are fast we might not need to bother.

Image

User avatar
NineBerry
Tame Wolf
Posts: 8950
Joined: Thu Feb 26, 2009 1:35 pm
Location: nSk
Contact:

Re: Saved RDF threads - thank you!

Post by NineBerry » Sun Feb 28, 2010 7:50 am

fuck you & fuck your god wrote:My assumption as to what you are saying is that you will have available in the sharable area of your Dropbox space a compressed file that when unzipped, and copied to my web space, will be ready to go?
Yes :) It will be a .rar file. So, you need to unrar it :) But the rest is true.

User avatar
fuck you & fuck your god
Posts: 103
Joined: Mon Feb 22, 2010 9:41 pm
About me: aka mechtheist
Off my meds again
Location: No where near Uganda
Contact:

Re: Saved RDF threads - thank you!

Post by fuck you & fuck your god » Sun Feb 28, 2010 8:09 am

NineBerry wrote:
fuck you & fuck your god wrote:My assumption as to what you are saying is that you will have available in the sharable area of your Dropbox space a compressed file that when unzipped, and copied to my web space, will be ready to go?
Yes :) It will be a .rar file. So, you need to unrar it :) But the rest is true.
:td: Great! I use 7-zip which handles un-raring, I've taken to using 'zip' as generic usually.
Image Image Image
Is life not a hundred times too short for us to stifle ourselves--Nietzsche

johnawlock
Posts: 11
Joined: Sun Feb 28, 2010 11:20 am
Contact:

Re: Saved RDF threads - thank you!

Post by johnawlock » Sun Feb 28, 2010 2:34 pm

NineBerry wrote:Here's another update.

Maybe, someone with enough webspace will then take the data and set-up a live browsable version on some public server.
I certainly hope this happens. I posted on RDF between December 2008 and March 2009 and I only found out about recent developments last week courtesy of the Times newspaper (non-digital version). I tried to recover some of my posts but it's not easy without a functioning search facility and the google site search option only produced c. 25% and not necessarily the ones which I was after. I only posted a few hundred times so I don't really need to take delivery of 3 DVDs! It'ds good to see that people who know what they're doing are on the job :clap:

I'm a little concerned about the integrity side of things - I think the wholesale retrospective deletion of posts and threads was inexcusable. It's bizarre that the collected outpourings of the most polemical and outrageous posters are going to be preserved for posterity while meaningful contributions have been consigned to the digital abyss. Shades of certain religious compilations, no? (The analogy with the flood myth hasn't escaped me either!) Hopefully the postings of the likes of Mazille which formed the backbone to many threads and who now exists only in palimpsest form can be recovered by negotiation with the perpetrators? I also note that some people used their privileged access rights to delete all their posts - an activity which in the light of this thread seems short-sighted. Hopefully they made personal backups and these too can be re-interpolated into the read-only web version though I doubt the likes of Nineberry will need the work. :shock:

Lastly, I posted mainly in the Faith & Religion forum (I don't need to be convinced of the fact of evolution). This forum was 'hidden' in that it could not be accessed at all unless you were logged in to RDF. I presume in the read-only archived version this will no longer be the case?

anthonzi
Posts: 67
Joined: Wed Feb 24, 2010 7:09 am
Contact:

Re: Saved RDF threads - thank you!

Post by anthonzi » Sun Feb 28, 2010 3:06 pm

johnawlock wrote: I certainly hope this happens. I posted on RDF between December 2008 and March 2009 and I only found out about recent developments last week courtesy of the Times newspaper (non-digital version).
The incident made it to the ink press in Britain did it?

also I am 12 what is this

epepke
Posts: 128
Joined: Thu Feb 25, 2010 4:30 am
Contact:

Re: Saved RDF threads - thank you!

Post by epepke » Sun Feb 28, 2010 6:54 pm

Thanks. Here are some mus you can use: μμμμμμμμ

User avatar
fuck you & fuck your god
Posts: 103
Joined: Mon Feb 22, 2010 9:41 pm
About me: aka mechtheist
Off my meds again
Location: No where near Uganda
Contact:

Re: Saved RDF threads - thank you!

Post by fuck you & fuck your god » Sun Feb 28, 2010 7:07 pm

epepke wrote:Thanks. Here are some mus you can use: μμμμμμμμ
Why, thanks, that was mighty kind of μ.
Image Image Image
Is life not a hundred times too short for us to stifle ourselves--Nietzsche

User avatar
fuck you & fuck your god
Posts: 103
Joined: Mon Feb 22, 2010 9:41 pm
About me: aka mechtheist
Off my meds again
Location: No where near Uganda
Contact:

Re: Saved RDF threads - thank you!

Post by fuck you & fuck your god » Sun Feb 28, 2010 7:22 pm

As I am planning to put the archive on line, anyone have suggestions on the domain name?
Image Image Image
Is life not a hundred times too short for us to stifle ourselves--Nietzsche

Post Reply

Who is online

Users browsing this forum: No registered users and 12 guests