Login:
Password:

Meet Microsoft's Ultimate Beta Tester

By Nate Mook and Tim Conneally, BetaNews

July 17, 2006, 4:45 AM

(continued from previous page)

Markezich BigDog-fooding begins with the development team, which runs a product during its alpha stage. Beta testers are then invited from within Microsoft in "chunks" using internal advertising and awareness campaigns. Over time, the number of employees running pre-release software can encompass much of Microsoft.

Over 60,000 machines were running Windows XP Service Pack 2 during its beta phase, and Markezich expects at least that many for Vista before it ships. By mid-June, Microsoft had 16,000 Windows Vista Beta 2 users and 25,000 on Office 2007 Beta 2.

"We have a group for each product we're dog-fooding that manages the testing process," Markezich said. "As we get to Beta 2 we say: 'Woohoo! People, get on Beta 2!' Part of the role is evangelizing, and part of the role is getting feedback to the product teams -- then we'll determine what build to make available to all the teams."

Typically, there are around eight deployments of the various builds spanning from the alpha stages to RTM. Another full deployment is made across the company at RTM.

Although Markezich is currently overseeing the largest number of Microsoft products ever to be dog-fooded simultaneously, he has no trouble rattling off the numbers. Along with Vista and Office, over 6,000 mailboxes are now on Exchange 12, and more than 30 Longhorn Servers are already in deployment.

1,000 Microsoft employees are running the next version of Windows Mobile, code-named "Crossbow," and 7,000 Systems Management Server v4 -- renamed to System Center Configuration Manager 2007-- installations are live. Over 3,000 machines running the next Microsoft Operations Manager release are also in production.

Markezich's eyes light up when he's asked to expound the benefits Microsoft has received through dog-fooding. Not only does his team play a key part in making sure a software product is of the highest quality, but he says Microsoft also gets to take advantage of new functionality right away.

"Once we rolled Exchange 2003 out, I went from 74 sites with Exchange servers, to 7 sites with Exchange servers throughout dog-food testing. Right now, I'm down to 4 sites with Exchange servers," Markezich explained. "So the fact that I'm dog-fooding means I'm going to get more value out of it very soon."

"My highest scores for employee satisfaction and user satisfaction are years that we have big dog-fooding waves like this," he notes. "Honestly, when an employee finds a problem, we get excited -- if it's before RTM."

Based on results from dog-fooding Exchange, Office and Vista, Microsoft is embarking on what it calls the "7x24 initiative." The company is pledging that employees will get back 7 million hours of productivity after using these products for 24 months. Markezich says Microsoft will "blow that away" with over 10 million hours of extra productivity.

But Markezich admits it's not always smooth sailing when you're rolling out potentially buggy software to employees who have their regular jobs to accomplish.

"Sometimes, as a CIO, you'll get an irate employee who says: 'You guys screwed me, I had a customer presentation and I couldn't do it because of this!' But we tell them: 'You just provided more value to the company because you found that problem than you could have provided otherwise, and now our customers won't have the same issue.'"

Moving forward, Markezich sees more and more products being dog-fooded within Microsoft, and says the extra overhead of running mission-critical systems on beta software is well worth the benefits.

"I'm waiting for the day when we start dog-fooding Xboxes," he jokes, "but we haven't gotten there yet."

<< Prev | 1 | 2

Add a Comment (39 Comments)

BetaNews reserves the right to remove any comment at any time for any reason. Please keep your responses appropriate and on topic. Foul language and personal attacks will not be tolerated.

Name (required):

E-mail (required):

Enter Your Comment:

By Malshadow

posted Apr 1, 2007 - 9:52 PM

Sometimes I wish people would just leave Microsoft to die. After all, they're all at least millionaires off of products that work half as good as their free counterparts. Exchange server isn't the only thing to ever be shipped out with problems, almost every one of their products have bad problems, and when they release updates to fix them they often include spyware.

Score: 0

By ramesh_pilani4

edited Aug 2, 2006 - 10:43 PM

How will I become a beta tester...?
Guide me...

Ramesh with hope

Score: 0

By CyberDoc999

posted Jul 18, 2006 - 7:37 AM

"eating its own dog food"
vista beta is now called "dog food"
yum yum
funny how you can't hide the taste of "dog food"
no matter how pretty you make it look....
please pass the "dog food"
(I will pass on that).....

Score: 0

By paulm

posted Jul 18, 2006 - 5:57 PM

CyberDoc999,
You've got to stop eating dog food. Hasn't anyone told you it's not fit for human consumption! ;-)

Score: 0

By mesiex

posted Jul 19, 2006 - 8:29 AM

They are the dog and also the food.

Score: 0

By mesiex

posted Jul 18, 2006 - 1:01 AM

They are the hound and the hare at the same time. Who may be the judge and the defendant at the same time and expect a fair trial?

Score: 0

By ds0934

posted Jul 17, 2006 - 11:52 PM

Brave man. To put his picture and name out there tagged to such a bold initiative as that of saving the face of Microsoft's ailing public perception. He's either got massive gonads or no brain, or both. Or maybe they're holding his family hostage until Vista ships.

Score: 0

By Ciprian.Dobrea

posted Jul 18, 2006 - 4:45 AM

thanks, you made me smile ;)

Score: 0

By mesiex

posted Jul 17, 2006 - 3:40 PM

Monster Megabytes, monster HD, monster OS...for minor tasks? Too many backdoors. Maybe the best would be to simplify things. The biggest the OS, the hardest to control it.

Score: 0

By PC_Tool

posted Jul 17, 2006 - 5:00 PM

Slip in a DOS Diskette.

Simple enough for ya?

Score: 0

By dvferret

posted Jul 17, 2006 - 8:01 PM

lol. dont think you can get much smaller that that.

Score: 0

By Kramy

posted Jul 18, 2006 - 12:12 AM

How about this?

Score: 0

By mesiex

posted Jul 21, 2006 - 6:37 PM

Sorry, I didn't mean that. In my opinion the size of the system since Windows 98 SE until Vista has grown unproportionally to the new advantages. In my opinion a high percentage of dirt and unuseful drivers could be avoided or cleaned unharmlessly.

Score: 0

By 33Nick

posted Jul 17, 2006 - 11:45 AM

I just think it is horrible, inhumanne, appaling, having poeple test their own systems inhouse, and shock of all shock finding out it doesn't work quite as well as they thought. I knew Microsoft was mean but that is just down-right...mean!

Score: 0

By seier

posted Jul 17, 2006 - 5:47 PM

Hi Nick,

Dog-fooding does suck while you're working there and it can have a significant effect on productivity. For example if you're supposed to get 8 hours of testing done, but first you have to download over a slow network (1-3 hours) and then upgrade to the latest beta of Windows. However, now that I don't work there, I can really appreciate how significant of a test that is. I think it's one of their best practices. Other companies will adopt this policy soon enough. It's true a company shouldn't be selling anything to others that it isn't yet willing to use itself.

Cheers,
Christian

Score: 0

By dvferret

posted Jul 17, 2006 - 8:02 PM

Yes, I find this a very good policy and hope other companies follow, as well.

Score: 0

By Spyderloco

posted Jul 17, 2006 - 10:26 AM

I wish we had a CIO that actually knew something.

Our CIO is a freaking idiot and doesn't even know what a T1 is or what a switch or router does. Some days he has more problems with his desktop than the average user here.

He needs fired.

Score: 0

By xyzcb1

edited Jul 17, 2006 - 11:33 AM

The cannot can't fire him. Your company probably probably spent tons on hired him and gave him one of those contact that guarantee millions as long as he shows up to work. If they do fire him, they have to buy out his contract, which probably cost more. I am in a similary situation, my director here know jack-sh1t, and occasionally when he does "work", and that mean trouble for the team. So we are better off when he does nothing, and let us run on our own.

Score: 0

By jshurst

posted Jul 17, 2006 - 12:00 PM

That sucks. My last CIO seemed to know everything. I didn't expect him to know technical details, but he could talk to the programmers in detail like he was one, and was an awesome manager as well. He could then talk to every other department like he worked for them too. I was very impressed by his knowledge, but doubt that I'll find too many people like him in my future opportunities.

Score: 0

By crashoverride

posted Jul 17, 2006 - 9:17 AM

Is it just me or does this guy look a little nutts in that picture?

Score: 0

By ds0934

posted Jul 17, 2006 - 11:54 PM

It begs to have a mustache and glasses drawn on it. Doesn't it? :)

Score: 0

By crashoverride

posted Jul 18, 2006 - 10:18 AM

Indeed it does. :)

Score: 0

By otto888

posted Jul 17, 2006 - 10:09 AM

yep. he looks nuts to me too.

Score: 0

By cannie

posted Jul 17, 2006 - 7:18 AM

No matter how much dog-fooding, hackers are waiting out there. We shall have security patches again, for sure. In my opinion only open code would make possible for experts to know what the system is really doing and prevent leaks by personalizing their own.

Score: 0

By compm375

posted Jul 17, 2006 - 8:41 AM

I'm usually out bashing Microsoft, but I can't really agree with you. Open and closed software alike have security vulnerabilities. Open source has more people to fix the code, but also more to mess it up. The only real way to get rid of vulnerabilities is have companies rethink the way they write code, whether it be open or closed source. They need to have coders that know what they are doing and many people auditing. Sure, it would make code take several times longer to complete, but it would be a lot more secure.

Score: 0

By Kramy

posted Jul 18, 2006 - 12:15 AM

The other option is to simplify the language to the point that errors are obvious. A 1200 line javascript app is much easier to debug than a 50000 line C++ app. The disadvantage there though is speed.

Score: 0

By cannie

posted Jul 18, 2006 - 7:00 AM

Speed is not a problem any more. It means a faster processor and memory upgrade. Your option is a very clever one. Congratulations.

Score: 0

By cannie

posted Jul 26, 2006 - 4:51 PM

Maybe the best: back to Windows 98 SE and Opera.

Score: 0

By PC Rat

posted Jul 17, 2006 - 7:12 AM

...

Microsoft has ~got~ to find some way of getting
a new OS out the door in less than six years.

Obviously, what they're currently doing ain't
working.

Instead of self-congratulatory Puff Pieces like
the one above, Microsoft needs to seriously
re-think it's development model.

...

The Computer Rodent

...

Score: 0

By frankwick

posted Jul 17, 2006 - 11:39 AM

You will see a faster OS release cycle from MS after Vista. Keep in mind that between XP and Vista, MS rolled out Server 2003, R1, and XPSP2. These are just OS releases and don't include complex systems like Exchange and SQLServer.

Score: 0

By PC_Tool

posted Jul 17, 2006 - 1:56 PM

Don't.

Feed.

The.

Trolls.

Score: 0

By Intrusive_Rogue

posted Jul 17, 2006 - 2:07 PM

right on

Score: 0

By PC Rat

edited Jul 17, 2006 - 1:48 PM

...

"Keep in mind that between
XP and Vista, MS rolled out
Server 2003, R1, and XPSP2"

...

That might be an excuse if Microsoft was a little
mom-and-pop organization instead of a huge international
corporation staffed with 61,000 employees (2005).

What's the deal ? Microsoft can't chew gum ~and~ walk
both at the same time ?

...

The Computer Rodent

...

Score: 0

By The Man

posted Jul 17, 2006 - 10:11 AM

Agreed

Score: 0

By donpacman

posted Jul 17, 2006 - 7:04 AM

I wonder how many times his Mom tells him to take out the trash?

Score: 0

By Intrusive_Rogue

posted Jul 17, 2006 - 10:07 AM

Only one more time a week than your's does.

Score: 0

By donpacman

edited Jul 21, 2006 - 12:02 AM

genius!

Score: 0

By jshurst

posted Jul 17, 2006 - 12:02 PM

Too funny...

Score: 0