Highly technical explanation inbound.
As is the case with Denial of Services attacks, you can't really stop the traffic from coming down the pipe -- and it's rather trivial these days to generate attacks in excess of 10Gbit/sec.
But most transit providers have a mechanism for blackholing traffic destined to your own IP addresses. It works by allowing you to announce /32s (single IP addresses) using a specific BGP community tag. Routes announced with that specific tag are then filtered at the provider's edge instead of yours. You're basically telling your upstream providers not to route you traffic destined for a particular IP address.
It's a last ditch effort in the world of DDoS mitigation. It effectively shuts down a single service on your network to allow other service to continue to function.
At some point during the 24 hours that we were seeing the traffic we blackholed the web server. Not necessarily because we were receiving too much traffic -- but because it looked as if the attacker was exploiting extremely specific vulnerabilities in PHPBB to shut the service down.
There were also problems with the consistency of the data loaded on the server. Things were no longer being indexed correctly. So there's no way to tell for sure if it was coincidence or a targeted attack vector.
When we figured out that rolling the database back was going to fix the problem we immediately switched to more progressive (and automated) filtering techniques and dropped the BGP blackhole announcements. But we forgot to drop the advertisement to Level3. Anyone who has an ISP that uses Level3 as its best path to Mafiascum was affected.
