It happened on an ordinary Friday afternoon at around 6 pm. Suddenly, our traffic at autoplenum.de quadrupeled. Just like flipping a switch. Boom! There it was, tons of new users out of the clear blue sky.
My first concern was for our servers and their ability to deal with that peak load.
Luckily, I had finished my migration to newer servers with enough spare capacity only a few weeks earlier. Phew! They were acting as if nothing had happened.
After relaxing about the server capacity, my second look was at the type of traffic which was coming in.
As nice as a ton of traffic can be, if it’s of low quality (or only bots!), it just costs bandwith and doesn’t help you meet your overall goals. Interestingly enough, all those additional users seemed to follow a very uncommon pattern:
- They all hit the homepage. As most of our regular users come from the search engines, they usually reach our site on pages one or two levels down the navigation hierarchy. Only a small fraction of our regular users ever reach the home page.
- None of them visited any other page on our site. This was very uncommon, too, as usually our visitors hang out for a couple of pages. Our visit/PI ratio dropped to half what it usually was.
- There was no referrer. Whenever a user comes from another site (like a search engine or any other web site) the browser usually sends the referrer. If the referrer is empty, that usually means that a) the user typed in the URL directly or b) he comes from e.g. a desktop application like a twitter client, clicking on a link in a tweet. None of our additional users had a referrer set. That indicated that something strange was going on here…
The Hunt For The Agressor
Taking the factors into account that a) the traffic spiked within one hour, b) the visitors seemed completely uninterested in our site and c) none of them had a referrer told us that there was something bad going on. Was it a unsuccessful DDoS driven by a Bot network? Or what else could exhibit such strange patterns? And how much more traffic would that cause? Would our servers survive in the long run? Was it only a test of our capabilities to withstand traffic spikes with more to come to finally bring us down? Seeing such strange effects makes you a little paranoid 😉
To find out whether there was any way to block that bad traffic I started to analyze the traffic sources: IP Adresses, browsers, OSs, etc. Unfortunately, there was no single source. All traffice exhibited the exact same patterns, no matter whether it was “good” or “bad”. That left us with no way to block anything. We could only sit on our hands and hope that it didn’t get worse!
The Affiliate Spammer
Luckily, our online marketing manager did his routine check of our affilinet account. There he also saw a strange effect. One affiliate, which signed up recently, had produced a vast amount of leads – cookies valid a couple of months with the potential to earn him a couple bucks per lead – if one of the users having such a cookie would sign up for our service. That was insane. Double-checking what that guy offered we found that they market a desktop application for ripping Internet radio streams into mp3. Bingo. Desktop software. No referrer. Aha! Obviously, that software loaded our site in the user’s browser without the user knowing or wanting to have anything to do with a car related site. But that one page load generated a lead for the affiliate. That’s what he wanted. It took us only five minutes to cancel his account and surprisingly our traffic was back to normal a couple of hours later. The weekend was saved!