One very false positive: McAfee in full damage control mode

Many instances of malware on Windows-based systems masquerade themselves as system services -- the various independent processes that respond to requests from both the operating system and applications with functions that users typically need. Network connectivity and printing are among the more common Windows services; and if you've ever perused the processes list of Task Manager (or, better yet, Process Explorer), you'll find these processes are represented by the single .EXE file that hosts them, svchost.exe.

Any anti-virus database looking for a rogue system service will probably have to refer to svchost.exe as the process that launches it, even though that process is clearly part of Windows itself. On Wednesday, McAfee distributed a .DAT file to many of its enterprise customers that may have had a single faulty character. As a result, their anti-virus systems successfully quarantined not the service launched by svchost.exe, but svchost.exe itself.

"Having talked to literally hundreds of my colleagues around the world and e-mailed thousands to try and find the best way to correct these issues, let me say this has not been my favorite day. Not for me, or for McAfee. Not by a long shot," wrote the company's executive vice president for customer support, Barry McPherson, late Wednesday. "Mistakes happen. No excuses. The nearly 7,000 employees of McAfee are focused right now on two things, in this order. First, help our customers who have been affected by this issue get back to business as usual. And second, once that is done, make sure we put the processes in place so this never happens again."

Wednesday's single, solitary false positive triggered a wave of collective, true negatives from the user community. "Using PR fuzzies like 'protect our customers from a seemingly endlessly multiplying variety and volume of attacks' and 'we wanted to protect our customers, as we have done successfully thousands and thousands of times before,' you sound less concerned with your company's screw up and more concerned about image. I pay your company to protect my computers," wrote Mike O., "and I don't care what you've done in the past. That is your job. When I am staring at a fleet of rebooting machines your past performance is meaningless. Here's some unsolicited advice: Just stick to facts and contrition. As a customer my only concern is the answer to the question, 'What have you done for me lately?'"

Other users raised the question of whether the fault in the .DAT file constitutes legal negligence -- something that could be prosecuted. Others questioned how they were supposed to access McAfee technical support through their non-working computers (where one of the first messages of response they see, even as of now, advises customers not to reboot their systems).

And one message from a self-titled security geek began by sarcastically thanking McPherson for his support, before going on: "I'm angry because you wasted a day's work for me and my colleagues. I'm angry because your 'support' achieved nothing. I'm angry because your apologies are meaningless because this will happen again as it has happened before. Most of all, I'm angry because there's nothing I can do because as a geek I must merely work around the flaws in products from whichever vendor spent most time sucking up to our purchasing manager. Maybe a decision-maker will read this. Maybe not."

Early Friday morning, McAfee posted what it calls a "SuperDAT Remediation Tool," that promises to suppress the driver that triggers the false positive and that forces svchost.exe into quarantine. Acknowledging the fact that computers whose svchost.exe files are already in quarantine can't access the Internet anyway (since that's a service hosted by that very file), McAfee advises customers to use a working computer to download the file, save it to "portable media," then take it to the impacted computer and run it from there.

One problem, in this era after the death of the floppy diskette, is that for many systems, "portable media" means "USB thumb drive." And that's another service that's hosted by svchost.exe.

Well before McAfee published its Remediation Tool, an independent IT specialist named David Blankenship -- whose own company was affected by the false positive on Wednesday -- devised his own solution and posted it to his Windows Live space. The solution involves extracting the original svchost.exe file from the Windows installation disc, copying them to a CD or USB (again, the latter may not work), rebooting the affected system (something McAfee advises customers not to do), log in as an administrator under safe mode, then restore that file to its rightful location. The procedure also involves manually restoring the corrected version of McAfee's .DAT file.

To be complete, Blankenship included a Step 12, which apparently many users obediently followed: "Send hate-mail to McAfee for ruining your day."

The damage to users' systems is unlikely to be permanent; unless desperate users tried drastic steps without considering the ramifications, their file systems will likely remain intact. What will have been lost could be measured in a few days' work time, but probably not the work itself.

However, this incident does expose the reality of a single point of failure introduced into users' systems everywhere through the introduction of what's supposed to be a security program. The one false character in McAfee's .DAT file (assuming that's the cause) provided customers with more of a nuisance than most malware is capable of achieving on its own. While McAfee concerns itself, as it should, with restoring its reputation, it might also want to take the time to consider another "r" word: rearchitecture.