RSSMeet Microsoft's Ultimate Beta Tester
By Nate Mook and Tim Conneally | Published July 17, 2006, 4:45 AM
(continued from previous page)
Dog-fooding begins with the development team, which runs a product during its alpha stage. Beta testers are then invited from within Microsoft in "chunks" using internal advertising and awareness campaigns. Over time, the number of employees running pre-release software can encompass much of Microsoft.
Over 60,000 machines were running Windows XP Service Pack 2 during its beta phase, and Markezich expects at least that many for Vista before it ships. By mid-June, Microsoft had 16,000 Windows Vista Beta 2 users and 25,000 on Office 2007 Beta 2.
"We have a group for each product we're dog-fooding that manages the testing process," Markezich said. "As we get to Beta 2 we say: 'Woohoo! People, get on Beta 2!' Part of the role is evangelizing, and part of the role is getting feedback to the product teams -- then we'll determine what build to make available to all the teams."
Typically, there are around eight deployments of the various builds spanning from the alpha stages to RTM. Another full deployment is made across the company at RTM.
Although Markezich is currently overseeing the largest number of Microsoft products ever to be dog-fooded simultaneously, he has no trouble rattling off the numbers. Along with Vista and Office, over 6,000 mailboxes are now on Exchange 12, and more than 30 Longhorn Servers are already in deployment.
1,000 Microsoft employees are running the next version of Windows Mobile, code-named "Crossbow," and 7,000 Systems Management Server v4 -- renamed to System Center Configuration Manager 2007-- installations are live. Over 3,000 machines running the next Microsoft Operations Manager release are also in production.
Markezich's eyes light up when he's asked to expound the benefits Microsoft has received through dog-fooding. Not only does his team play a key part in making sure a software product is of the highest quality, but he says Microsoft also gets to take advantage of new functionality right away.
"Once we rolled Exchange 2003 out, I went from 74 sites with Exchange servers, to 7 sites with Exchange servers throughout dog-food testing. Right now, I'm down to 4 sites with Exchange servers," Markezich explained. "So the fact that I'm dog-fooding means I'm going to get more value out of it very soon."
"My highest scores for employee satisfaction and user satisfaction are years that we have big dog-fooding waves like this," he notes. "Honestly, when an employee finds a problem, we get excited -- if it's before RTM."
Based on results from dog-fooding Exchange, Office and Vista, Microsoft is embarking on what it calls the "7x24 initiative." The company is pledging that employees will get back 7 million hours of productivity after using these products for 24 months. Markezich says Microsoft will "blow that away" with over 10 million hours of extra productivity.
But Markezich admits it's not always smooth sailing when you're rolling out potentially buggy software to employees who have their regular jobs to accomplish.
"Sometimes, as a CIO, you'll get an irate employee who says: 'You guys screwed me, I had a customer presentation and I couldn't do it because of this!' But we tell them: 'You just provided more value to the company because you found that problem than you could have provided otherwise, and now our customers won't have the same issue.'"
Moving forward, Markezich sees more and more products being dog-fooded within Microsoft, and says the extra overhead of running mission-critical systems on beta software is well worth the benefits.
"I'm waiting for the day when we start dog-fooding Xboxes," he jokes, "but we haven't gotten there yet."
"eating its own dog food"
vista beta is now called "dog food"
yum yum
funny how you can't hide the taste of "dog food"
no matter how pretty you make it look....
please pass the "dog food"
(I will pass on that).....
Score: 0
|CyberDoc999,
You've got to stop eating dog food. Hasn't anyone told you it's not fit for human consumption! ;-)
Score: 0
|They are the dog and also the food.
Score: 0
|They are the hound and the hare at the same time. Who may be the judge and the defendant at the same time and expect a fair trial?
Score: 0
|Brave man. To put his picture and name out there tagged to such a bold initiative as that of saving the face of Microsoft's ailing public perception. He's either got massive gonads or no brain, or both. Or maybe they're holding his family hostage until Vista ships.
Score: 0
|thanks, you made me smile ;)
Score: 0
|Monster Megabytes, monster HD, monster OS...for minor tasks? Too many backdoors. Maybe the best would be to simplify things. The biggest the OS, the hardest to control it.
Score: 0
|Slip in a DOS Diskette.
Simple enough for ya?
Score: 0
|lol. dont think you can get much smaller that that.
Score: 0
|How about this?
Score: 0
|Sorry, I didn't mean that. In my opinion the size of the system since Windows 98 SE until Vista has grown unproportionally to the new advantages. In my opinion a high percentage of dirt and unuseful drivers could be avoided or cleaned unharmlessly.
Score: 0
|I just think it is horrible, inhumanne, appaling, having poeple test their own systems inhouse, and shock of all shock finding out it doesn't work quite as well as they thought. I knew Microsoft was mean but that is just down-right...mean!
Score: 0
|Hi Nick,
Dog-fooding does suck while you're working there and it can have a significant effect on productivity. For example if you're supposed to get 8 hours of testing done, but first you have to download over a slow network (1-3 hours) and then upgrade to the latest beta of Windows. However, now that I don't work there, I can really appreciate how significant of a test that is. I think it's one of their best practices. Other companies will adopt this policy soon enough. It's true a company shouldn't be selling anything to others that it isn't yet willing to use itself.
Cheers,
Christian
Score: 0
|Yes, I find this a very good policy and hope other companies follow, as well.
Score: 0
|I wish we had a CIO that actually knew something.
Our CIO is a freaking idiot and doesn't even know what a T1 is or what a switch or router does. Some days he has more problems with his desktop than the average user here.
He needs fired.
Score: 0
|The cannot can't fire him. Your company probably probably spent tons on hired him and gave him one of those contact that guarantee millions as long as he shows up to work. If they do fire him, they have to buy out his contract, which probably cost more. I am in a similary situation, my director here know jack-sh1t, and occasionally when he does "work", and that mean trouble for the team. So we are better off when he does nothing, and let us run on our own.
Score: 0
|That sucks. My last CIO seemed to know everything. I didn't expect him to know technical details, but he could talk to the programmers in detail like he was one, and was an awesome manager as well. He could then talk to every other department like he worked for them too. I was very impressed by his knowledge, but doubt that I'll find too many people like him in my future opportunities.
Score: 0
|Is it just me or does this guy look a little nutts in that picture?
Score: 0
|yep. he looks nuts to me too.
Score: 0
|It begs to have a mustache and glasses drawn on it. Doesn't it? :)
Score: 0
|Indeed it does. :)
Score: 0
|No matter how much dog-fooding, hackers are waiting out there. We shall have security patches again, for sure. In my opinion only open code would make possible for experts to know what the system is really doing and prevent leaks by personalizing their own.
Score: 0
|I'm usually out bashing Microsoft, but I can't really agree with you. Open and closed software alike have security vulnerabilities. Open source has more people to fix the code, but also more to mess it up. The only real way to get rid of vulnerabilities is have companies rethink the way they write code, whether it be open or closed source. They need to have coders that know what they are doing and many people auditing. Sure, it would make code take several times longer to complete, but it would be a lot more secure.
Score: 0
|The other option is to simplify the language to the point that errors are obvious. A 1200 line javascript app is much easier to debug than a 50000 line C++ app. The disadvantage there though is speed.
Score: 0
|Speed is not a problem any more. It means a faster processor and memory upgrade. Your option is a very clever one. Congratulations.
Score: 0
|Maybe the best: back to Windows 98 SE and Opera.
Score: 0
|...
Microsoft has ~got~ to find some way of getting
a new OS out the door in less than six years.
Obviously, what they're currently doing ain't
working.
Instead of self-congratulatory Puff Pieces like
the one above, Microsoft needs to seriously
re-think it's development model.
...
The Computer Rodent
...
Score: 0
|Agreed
Score: 0
|You will see a faster OS release cycle from MS after Vista. Keep in mind that between XP and Vista, MS rolled out Server 2003, R1, and XPSP2. These are just OS releases and don't include complex systems like Exchange and SQLServer.
Score: 0
|...
"You will see a faster OS
release cycle from MS after
Vista"
...
Yeah, that's what Steve Ballmer said last week.
But talk has always been cheap at Microsoft, and
promises easier than performance.
It remains to be seen if Ballmer can actually
decrease the cycle time for it's consumer OS.
It'll take ~more~ than optimistic announcements
and calls to the PR Dept. !
...
The Computer Rodent
...
Score: -1
|...
"Keep in mind that between
XP and Vista, MS rolled out
Server 2003, R1, and XPSP2"
...
That might be an excuse if Microsoft was a little
mom-and-pop organization instead of a huge international
corporation staffed with 61,000 employees (2005).
What's the deal ? Microsoft can't chew gum ~and~ walk
both at the same time ?
...
The Computer Rodent
...
Score: 0
|Don't.
Feed.
The.
Trolls.
Score: 0
|right on
Score: 0
|I wonder how many times his Mom tells him to take out the trash?
Score: 0
|Only one more time a week than your's does.
Score: 0
|Too funny...
Score: 0
|genius!
Score: 0
|