RSSMeet Microsoft's Ultimate Beta Tester
By Nate Mook and Tim Conneally | Published July 17, 2006, 4:45 AM
With tens of thousands of external beta testers examining the latest Microsoft software, little is said about the company's largest guinea pig: itself. Long before a final release, Microsoft has put the beta bits into production -- a process it calls "eating its own dog food."
BetaNews recently sat down with Microsoft Chief Information Officer Ron Markezich, often referred to as "Microsoft's ultimate beta tester," to get an in-depth look at how dog-fooding helps shape the software that hundreds of millions of people use each day.
As CIO, Markezich heads up Microsoft's IT department, which is responsible for managing the massive infrastructure that supports over 63,000 employees around the world. Perhaps more importantly, his team also plays a vital role in product development by actively using Microsoft software before it's complete, filing bugs and providing feedback.
Finding individual consumers to beta test products like Windows Vista is easy; even Google offers much of its software as betas. But businesses are a different story. Corporations rely on mission-critical systems to keep things running smoothly, and any hiccup due to buggy software could have major consequences.
Microsoft has attempted to alleviate this dilemma by encouraging businesses to install its enterprise software during the beta phase through the use of "go live" licenses. However, the company has learned the only way to firmly stand behind a release when talking to partners is to first deploy it internally.
That issue came to a head with the release of Exchange Server 2000. Markezich says the product was not running optimally inside Microsoft, but it was shipped to customers nonetheless. The result was a veritable nightmare: customers experienced the same problems and Microsoft had to rush out numerous hotfixes.
"From that day forward we said we will not ship a product that we sell to the enterprise until we run our business on it," Markezich explained. Since then, the dog-fooding program has extended across a variety of Microsoft products and now serves as a quality checkpoint on the road to a final release.
"Not only do we want to get involved in the beta testing, but we also want to give feedback in the product planning, product designs, requirements, and then all the way through the entire lifecycle," Markezich says.
Unlike a typical beta tester, Markezich's team holds considerable sway over the release of a new product. In fact, he has the power to delay a release to manufacturing (RTM) if a software product hasn't hit goals and expectations the development team had previously established as part of the dog-fooding process.
"If we don't meet these shared goals for a particular milestone, we don't allow the product group to move on to the next milestone, all the way to RTM. If we don't meet the goals that we have laid out for RTM, sometimes even a year and a half or two years prior, we don't allow the product to ship," Markezich said.
For Windows Vista, Markezich has shared goals for BitLocker, enterprise search, strong user authentication, network access protection, and a number of other business-oriented features.
"This organization is an extension of the product team; we have sign-off authority on those products before they ship," he adds.
"eating its own dog food"
vista beta is now called "dog food"
yum yum
funny how you can't hide the taste of "dog food"
no matter how pretty you make it look....
please pass the "dog food"
(I will pass on that).....
Score: 0
|CyberDoc999,
You've got to stop eating dog food. Hasn't anyone told you it's not fit for human consumption! ;-)
Score: 0
|They are the dog and also the food.
Score: 0
|They are the hound and the hare at the same time. Who may be the judge and the defendant at the same time and expect a fair trial?
Score: 0
|Brave man. To put his picture and name out there tagged to such a bold initiative as that of saving the face of Microsoft's ailing public perception. He's either got massive gonads or no brain, or both. Or maybe they're holding his family hostage until Vista ships.
Score: 0
|thanks, you made me smile ;)
Score: 0
|Monster Megabytes, monster HD, monster OS...for minor tasks? Too many backdoors. Maybe the best would be to simplify things. The biggest the OS, the hardest to control it.
Score: 0
|Slip in a DOS Diskette.
Simple enough for ya?
Score: 0
|lol. dont think you can get much smaller that that.
Score: 0
|How about this?
Score: 0
|Sorry, I didn't mean that. In my opinion the size of the system since Windows 98 SE until Vista has grown unproportionally to the new advantages. In my opinion a high percentage of dirt and unuseful drivers could be avoided or cleaned unharmlessly.
Score: 0
|I just think it is horrible, inhumanne, appaling, having poeple test their own systems inhouse, and shock of all shock finding out it doesn't work quite as well as they thought. I knew Microsoft was mean but that is just down-right...mean!
Score: 0
|Hi Nick,
Dog-fooding does suck while you're working there and it can have a significant effect on productivity. For example if you're supposed to get 8 hours of testing done, but first you have to download over a slow network (1-3 hours) and then upgrade to the latest beta of Windows. However, now that I don't work there, I can really appreciate how significant of a test that is. I think it's one of their best practices. Other companies will adopt this policy soon enough. It's true a company shouldn't be selling anything to others that it isn't yet willing to use itself.
Cheers,
Christian
Score: 0
|Yes, I find this a very good policy and hope other companies follow, as well.
Score: 0
|I wish we had a CIO that actually knew something.
Our CIO is a freaking idiot and doesn't even know what a T1 is or what a switch or router does. Some days he has more problems with his desktop than the average user here.
He needs fired.
Score: 0
|The cannot can't fire him. Your company probably probably spent tons on hired him and gave him one of those contact that guarantee millions as long as he shows up to work. If they do fire him, they have to buy out his contract, which probably cost more. I am in a similary situation, my director here know jack-sh1t, and occasionally when he does "work", and that mean trouble for the team. So we are better off when he does nothing, and let us run on our own.
Score: 0
|That sucks. My last CIO seemed to know everything. I didn't expect him to know technical details, but he could talk to the programmers in detail like he was one, and was an awesome manager as well. He could then talk to every other department like he worked for them too. I was very impressed by his knowledge, but doubt that I'll find too many people like him in my future opportunities.
Score: 0
|Is it just me or does this guy look a little nutts in that picture?
Score: 0
|yep. he looks nuts to me too.
Score: 0
|It begs to have a mustache and glasses drawn on it. Doesn't it? :)
Score: 0
|Indeed it does. :)
Score: 0
|No matter how much dog-fooding, hackers are waiting out there. We shall have security patches again, for sure. In my opinion only open code would make possible for experts to know what the system is really doing and prevent leaks by personalizing their own.
Score: 0
|I'm usually out bashing Microsoft, but I can't really agree with you. Open and closed software alike have security vulnerabilities. Open source has more people to fix the code, but also more to mess it up. The only real way to get rid of vulnerabilities is have companies rethink the way they write code, whether it be open or closed source. They need to have coders that know what they are doing and many people auditing. Sure, it would make code take several times longer to complete, but it would be a lot more secure.
Score: 0
|The other option is to simplify the language to the point that errors are obvious. A 1200 line javascript app is much easier to debug than a 50000 line C++ app. The disadvantage there though is speed.
Score: 0
|Speed is not a problem any more. It means a faster processor and memory upgrade. Your option is a very clever one. Congratulations.
Score: 0
|Maybe the best: back to Windows 98 SE and Opera.
Score: 0
|...
Microsoft has ~got~ to find some way of getting
a new OS out the door in less than six years.
Obviously, what they're currently doing ain't
working.
Instead of self-congratulatory Puff Pieces like
the one above, Microsoft needs to seriously
re-think it's development model.
...
The Computer Rodent
...
Score: 0
|Agreed
Score: 0
|You will see a faster OS release cycle from MS after Vista. Keep in mind that between XP and Vista, MS rolled out Server 2003, R1, and XPSP2. These are just OS releases and don't include complex systems like Exchange and SQLServer.
Score: 0
|...
"You will see a faster OS
release cycle from MS after
Vista"
...
Yeah, that's what Steve Ballmer said last week.
But talk has always been cheap at Microsoft, and
promises easier than performance.
It remains to be seen if Ballmer can actually
decrease the cycle time for it's consumer OS.
It'll take ~more~ than optimistic announcements
and calls to the PR Dept. !
...
The Computer Rodent
...
Score: -1
|...
"Keep in mind that between
XP and Vista, MS rolled out
Server 2003, R1, and XPSP2"
...
That might be an excuse if Microsoft was a little
mom-and-pop organization instead of a huge international
corporation staffed with 61,000 employees (2005).
What's the deal ? Microsoft can't chew gum ~and~ walk
both at the same time ?
...
The Computer Rodent
...
Score: 0
|Don't.
Feed.
The.
Trolls.
Score: 0
|right on
Score: 0
|I wonder how many times his Mom tells him to take out the trash?
Score: 0
|Only one more time a week than your's does.
Score: 0
|Too funny...
Score: 0
|genius!
Score: 0
|