The HD-DVD saga: stickin' it to the man

If there's anything that can be said about the infamous HD-DVD Nacht and the past 48 hours, let's say they've been hectic. I've documented the entire case from this magazine's point of view -- here's an illustrated account of what happened here and around the Web.

Before I start, I would like to make only one request. If, after reading the story, you like it, please submit it to Digg/Reddit/Slashdot or your favorite site. Remember that I can't (at least on Digg) because I'm persona non grata over there.

Now, let's start.

How the story came to be

So there I was, minding my own business, catching up on my daily feed; then it happened, a new post on Reddit was publicizing the HD-DVD processing code. Yes, Reddit is actually where the leak was first made public.

The fact that the security of HD-DVD's AACS had been compromised progressively was already known -- make no mistake. Volume keys had already been retrieved and were shared samizdat in peer-to-peer networks, along with the software on Doom9.

But this was different. A cease-and-desist letter had been issued to Google, regarding search results that included the key. Not any key, mind you, but the processing key. In theory, with this key, one could decrypt all HD-DVD volumes sold so far, so it was a much bigger deal (and a much more publicized censorship attempt) than just volume keys. A blog (Spooky action at a distance) had been completely nuked because of this.

Many sites were already carrying the key; therefore, never in my wildest dreams I imagined what would happen later on. At that moment, I wrote the (now kind of famous) story named Spread this number, partly because I was angry at Big Media's continuing attempt to screw with our lives, and partly because I knew that the story warranted more attention than just a cease-and-desist.

Yeah, Digg provided the initial push...

And I submitted it to Digg, under my account (RuddO). Or at least I think I did. The submission was named Spread this number. Now.

And I forgot about it.

Usually, stories submitted by oneself don't get very far -- you see, in order for stories not to disappear in Digg's machinery, they need to acquire momentum almost right when they are submitted. My only other story that got somewhere was The coolness factor of Linux.

Then, (if I recall correctly) after half an hour of submitting the story, I couldn't access my site. I just couldn't. Why? It turns out the Digg community had started to vote for the story, at a dramatic pace -- a Digg every three seconds, or so. I found out about this by using apachetop and the Digg This plugin (which incidentally had a bug I had to fix -- thanks, Subversion, for letting me track code changes so cleanly!).

My site was dead, and the story was rising, fast! As a temporary measure (or so I thought) I enabled WP-Cache, and the server returned to normal.

...then Digg messed up. Big time.

I suddenly noticed that the server was running fine, not because of WP-Cache (though it helped) but because I suddenly wasn't getting any more hits from Digg. No more hits (except those from Coral Cache and duggmirror). I visited the story on this site, and I saw an error message on the Digg button. Hell, click on the button, nothing happened. Went over to Digg -- the story couldn't be found. Tried to login... my account was disabled due to abuse!

I immediately emailed the support staff at Digg.

While I was doing that, Rudd-O.com died again. Damnit! Right then, I wanted a bigger server, fast (hey, I still do now!). Checking apachetop yielded a different referrer, and many, many more hits per second -- a new story had been submitted, linked to another Web page; the first or so comment had the address of my story, and dozens of other comments pointed out the fact that my submission was killed.

Exactly then's when the shit started to hit the fan. Mounds and mounds of shit, if I may say. Probably due to a combination of being angry at Digg, wanting to "stick it to the MPAA man", and out of sheer luck, the new submission Spread this number. Again. skyrocketed:

You could literally blink, hit F5 to refresh, and see 30 more votes in the yellow button. I'm not kidding, that's how fast the story was rising.

And that's how fast incoming hits to this site were increasing. I quickly adjusted my WordPress plugin list to exclude several active plugins -- I had to resort to removing read permissions from the files, because I couldn't log in to the WordPress administration panel.

The first traffic wave. Huge.

In a couple of minutes, tens of comments started pouring in my article, and some of them were mentioning the fact that I had been banned (evidently, people were aware of that fact):

Just minutes after that, Digg died a horrible 404 death. All of their pages were nowhere to be found!

Was it an administrative fuckup, or a serious multisystem failure? I don't know, but I do know that, at 21:35, a thousand more votes had been cast:

The resubmitted story breaks all kinds of records on Digg

Damn, damn, damn! Too much blinking, too little time. Note the 4300+ Diggs here, and also note that the story wasn't on Reddit... yet:

At 21:54, my site was already serving an average of 7 requests per second. Oh, yeah, I was watching the Simpsons, and getting ready to watch Shark. Those aren't overlays done in Photoshop or The GIMP -- that's Beryl managing my windows:

At this point, I had two questions:

1. How long is the story going to last before they pull it?
2. Am I dreaming?

The traffic explosion looks beautiful on video

Just to make sure I wasn't, I "videotaped" the event using xvidcap:

See that big 'cell-like' organism growing? Well, yeah, that's the second Digg submission we're talking about -- it wasn't censored, yet. Check the same event out, but using text only -- notice how large the story title appears:

Stats time. I was flabbergasted. And the site was faltering.

At this point, I decided to take a look at my statistics. The last couple of days had been fairly good, with 3000 or so visits to my interview with Beryl author Quinn Storm. But nothing could prepare me for what I was about to see:

See the chat window named Andrés Santos? He owns Blogsticker (quite the rage in social media a few weeks ago) -- he's a friend of mine, and we were college classmates. Give him a visit.

Meanwhile, Apache was clogged with keepalive requests. It was only ten thirty, and the story was clocking in at 6500+ diggs:

Minutes after, it was at nearly 7000 diggs. The story sure didn't seem like running out of steam. If it was an Unreal character (to steal a commenter's expression) the off voice would have gone Muh muh muh muh muhnster kill, kill, kill...:

Why would they kill the story, without a cease-and-desist? Why was my account missing?

Why was Digg killing the story, when the cease-and-desist already had included the code? More importantly, who could have fucked up so majorly in drafting that cease-and-desist? If I may, the guys who devised the URL trick to embed the code... those are the real geniuses behind the story.

Of course, we now know that HD-DVD sponsored some of Digg's work. Meanwhile, my account was still AWOL (absent without logic):

And the story had climbed to position 1 in the 30-day Top list. Minutes later, magic number 7707 was on the yellow box, and then, 8038. So far, I'm confident we had broken the acceleration record:

Statistics for Rudd-O.com soar sky-high

At this point, it was time to check my stats again. Yeah, I was bored with TV:

Just for kicks, I also watched BloGalaxia, the Latin American blog directory. #20, not bad. How much money had I made with AdSense? Here it is as well:

The Digg story breaks all kinds of records -- for real!

In the meantime, the story was pushing ten thousand diggs, and had floated to page two of the yearly Top stories. Minutes after, it hit the proverbial page one:

Hooray! In under seven hours, from submission time to page one? Wow! Now I was almost positive the story couldn't be killed -- at least not without a cease-and-desist, right? Especially since the swarm had gotten so big:

At this point, I'm speechless. People started sending me screenshots. 10.000 diggs. 11.111 diggs. The digg count reached 12.555:

At this point, I hit the sack.

The second disappearance, the second (and most catastrophic) Digg fuckup.

Ten AM, Tuesday. I move the mouse, hit F5, and discover something extremely odd. Can you figure out what I figured out?

In a couple of hours, the community had propelled the story beyond any established records, yet Digg still managed to get it killed. At the same time, in contrast, Reddit was embracing the story:

Boing Boing picks the story up

While Digg was turning into a shitstorm of highly voted, tongue-in-cheek submissions, related to the famous illegal code... several high-profile, more mainstream sites started picking the story up:

The money was good, but the stats had dropped sharply:

Meanwhile, my site was dying, spiraling out of control. The few requests that managed to get through, couldn't even grab the stylesheet. Even Greg House's wicked behavior couldn't put a smile on my face. Actually, I think he was "mocking my fail":

Del.icio.us to the rescue!

With Digg out of the picture, someone else came to the "rescue" of my ailing server: del.icio.us! Bear in mind, this was four hours after I woke up:

I started to suspect WP-Cache. True enough, for some reason, WP-Cache wasn't using its cache, but generating a new page every time. It might have been due to the volume of the comments in my story -- every time a comment is posted, the page is regenerated again to be served, fresh. Note that I had disabled a couple of plugins already, but basic functionality of the site was there. Slow, but there.

While that stabilized, I wrote up a quick blurb and submitted it to Slashdot.

Slashdotted!

I tend to keep a terminal window running apachetop and check it frequently. Unbelievably, this let me in almost immediately on to the fact that I had been published in Slashdot, at 2:50 PM:

Rudd-O.com was now maxed-out:

It was time to resort to the last weapon in the arsenal of Web serving (beyond, you know, spending more money, which doesn't work for cheapskates like me). Squid!

(Sorry. I'm such a sucker for lolcats!)

I immediately googled for Squid, used smart to install it, and set up a front proxy, then moved Apache to port 81. Don't worry, evil hackers, I firewalled port 81 too. This immediately bumped my requests per second to more than twenty, and freed Apache to serve dynamic pages only:

And it showed up in the stats!

Arr, mates! The Pirate Bay offers their kind assistance!

Three hours after, the Squid referrer log let me in on a "small" new development: The Pirate Bay had linked to my story, smack in the middle of their home page. With a little bit of humor, I might add:

Traffic was now so high that Squid was starting to actually consume more than 1% of CPU. Requests per second? Between 50 and 60. Apache was no longer coping with the demand for dynamic pages, and I was scrambling with the MaxClients setting, between swapping more, or serving fewer clients:

Meanwhile, del.icio.us was showing the story as #1 in their hotlist, and a few hours later, I had climbed to number 6 in Latin America's BloGalaxia:

At this point, all major sites are carrying the story, from one angle or another. There's no stopping now.

But, wait a second... what happened to Digg in the meantime?

Poor Digg. Their "proactive" enforcement of the DMCA turned against them.

Believe it or not, the shit continued to hit the fan all day long over there. In fact, Kevin Rose himself backed down after he could witness the major PR fuckup that had been the killing of both my submission and the subsequent resubmission by the Digg community.

Most people are already calling May 1st. 2007 HD-DVD Night. If you ask me, they're right. Witness for yourself how the Digg front page looked like just a few hours ago. Every single story had something to do, either tongue-in-cheek or directly, with the AACS processing key censorship deal. Every single story (yes, even the "color schemes"):

One more thing left to say: thanks, immense thanks to everyone who participated

And this is where I sign off. But, before hitting the sack, let me show you a beautiful Beryl scale-powered screenshot, showing all the major sites that carried this story in their front pages:

Thanks, guys, for sticking together and showing the world how wrong and greedy the movie studios were. Thanks for linking to my site. Thanks for defending moral values over profit. Thanks for pitching in and trying to change the world, even if it's only one processing key at a time.

DRM is wrong and is immoral. Censorship in the name of DRM is even more wrong.

But, most importantly, yesterday we stuck it to the man