RSSMicrosoft Responds to Re-discovery of Vista Network Slowdowns
By Scott M. Fulton, III | Published August 27, 2007, 3:51 PM
A curious network performance reduction noticed by many Windows Vista users of the 2CPU forum that became the talk of Slashdot last week has been identified as having been caused not by DRM, as Slashdot users expected, but by a curious prioritization "feature" of Vista that's intentionally biased toward Media Player at the expense of network and system resources.
The effects of this feature were first revealed last June, as BetaNews reported, by Microsoft security engineer Mark Russinovich.
In a blog post to SysInternals this afternoon, Russinovich once again provides a clear view of what some prominent bloggers have called a bug, but which Microsoft maintains is an intentional choice: favoring multimedia playback when the CPU is stressed, even if it means slowing down file and network transfers and even the mouse pointer.
The culprit, Russinovich writes, is the Multimedia Class Scheduler Service (MMCSS), one of the many services run within Vista's service host. When the CPU is under stress, Windows Media Player 11 places a call to MMCSS, which in turn boosts the player's own priority relative to other processes. Many Vista users found out about this through the use of Vista's Performance Monitor.
"When a multimedia application begins playback, the multimedia APIs it uses call the MMCSS service to boost the priority of the playback thread into the realtime range, which covers priorities 16-31, for up to 8 ms of every 10 ms interval of the time, depending on how much CPU the playback thread requires. Because other threads run at priorities in the dynamic priority range below 15, even very CPU intensive applications won't interfere with the playback."
At least, that's the intention. Internet traffic shouldn't be affected by this performance throttling, he continues, due to multiple connections that fragment Internet traffic even in the best conditions anyway. Conceivably, the lesser the bandwidth, the fewer the fragments.
 |
| From Mark Russinovich's demonstration at TechEd 2007 last June, this histogram from Vista's Process Explorer clearly shows the task priority for Windows Media Player 11 (the cyan-colored line) boost dramatically when his stress test is applied to the CPU. |
But the amount of throttling Microsoft engineers intentionally chose to employ was based on their assessment of 100 Mbps networks, where fragmentation is great enough that packet receive rates are high. "The hard-coded limit was short-sighted with respect to today's systems that have faster CPUs, multiple cores and Gigabit networks," Russinovich went on, "and in addition to fixing the bug that affects throttling on multi-adapter systems, the networking team is actively working with the MMCSS team on a fix that allows for not so dramatically penalizing network traffic, while still delivering a glitch-resistant experience."
That's where Mark leaves us hanging for now. He did use the word "bug" to describe the behavior of this choice on multi-adaptor systems, and he did characterize Microsoft's forthcoming solution as a "fix." That flies in the face of an official Microsoft response to ZDNet blogger Adrian Kingsley-Hughes, who had written earlier that he was going to turn most Vista sounds off until Microsoft worked out a solution.
"Please note that some of what we are seeing is expected behavior," the spokesperson told Kingsley-Hughes, "and some of it is not. In certain circumstances, Windows Vista will trade off network performance in order to improve multimedia playback." Then reviving an old Microsoft phrase that was so often used in the 1990s to officially explain away behavior that couldn't otherwise be explained, Microsoft wrote, "This is by design."
Another citation from Microsoft's response says users typically hate having to hear sputters and clicks from their speakers when their network or CPU is stressed, and it's that hatred which provoked the company to make the trade-off in the first place.
Russinovich's characterization of the throttling trigger as "hard-coded" may be an indication that the company's fix may involve giving the user a choice of which services should be pared down when crunch time comes, system services or multimedia. But with the sheer number of system services Vista now runs, crunch time may be coming too often anyway.
Why would anyone use that useless OS, unless they are a complete idiot...
Score: 0
Well, I'm happy nothing like playing a HD video file and NOT having my video skip while all my torrentz run in the background :P
Score: 0
Hey this is something I already knew, vista blows!
Vista has plenty of killer features, otherwise known as bugs. Stop trying to cover it up Microsoft.
Score: 0
This actually makes sense from the user's point of view. Too bad Microsoft didn't test on Gigabit networks. That was their mistake. A mac works the same way.
Score: 0
Too bad Microsoft didn't test on Gigabit networks.
Too right. I expect that when testing network connectivity, they tried to limit all possible variables, thus negating the use of any media players. The purpose of testing, however, especially with Microsoft, should be focusing more *on* the variables, and with the new prioritization of media, media should *definitely* have been part of the testing process.
MS did a number on this one. I'm just glad I haven't upgraded to gigabit at home yet. The effects on 10/100 are barely noticeable (in some cases) to not at all noticeable (in my case), or I'd be heading back to XP.
Score: 0
MP3 decoding should take a fraction of one percent of the CPU cycles on a modern PC. The MP3 player in your pocket probably runs on 200MHz ARM or less. Modern Ethernet cards and Gigabit cards in particular use DMA to put packets directly into RAM. That uses no CPU. Maybe another fraction of one percent to set up DMA to move the data onto a hard disk.
I can add and I know what goes on inside a computer and something else is going on here other than just prioritizing.
Any computer with 512 MB and a 1 GHz processor should be able to play MP3 and receive a full speed Gigabit network file copy without using more than 1% of it's CPU cycles.
There's something fishy in Redmond.
Score: 0
I can add and I know what goes on inside a computer and something else is going on here other than just prioritizing.
There's something fishy in Redmond.
Never attribute to malice that which can be adequately explained by incompetence.
Score: 0
Now I've read Mark's blog and it is simple incompetence. The group that was responsible for glitch free multimedia got their hooks into the network stack and inserted a hard coded value to limit the number of packets received. They made sure the number was small enough to guarantee they would meet their objective of glitch free multimedia playback. As I understand it they are now looking to formulate a packet limit based on the capabilities of the machine.
I guess the group at Microsoft that was responsible for high-speed network throughput doesn't listen to music.
And to clarify my comment above, receiving Ethernet packets is cheap, but processing a TCP/IP stack will chew up some cycles.
Score: 0
Yessir. They really screwed up on this one.
I am hoping a fix is released quickly on one hand, and on the other, I hope they *really* work to get it right.
What I would *love* to see is the ability for users to completely disable the MMCSS prioritization, but I won't be holding my breath.
Score: 0
Why was Vista released at all? Microsoft has a history of releasing finals that are actually betas, but Vista took 4 years. Someone needs to eliminate about a billion lines of code.
You'd have thought that they would have learned from their browser that lean and mean is the way to go. If XP wasn't available, everyone would be using 'nix. When you have more money than most countries, there's no excuse for this.
Score: 0
Microsoft is well known for:
"It's not a bug, it's a feature!"
Score: 0
Really, I can't believe they're using that one again. Jeesh, that is one company totally bereft of originality.
Score: 0
How does this whole scenario apply to multimedia content streamed from the Internet? If the media player gets a priority boost, what good is it going to do if the streamed content is not available at the expected rate due to the hit on network performance? Isn't this kind of a performance deadlock?. In this case wouldn't it make sense to boost both the multimedia playback and the network service?
Score: 0
If i just could uninstall Windows Media completely...i never use this poor player, there are better and free ones !
But i hope Microsoft will end this poor behaviour by Vista.
Score: 0
Interesting read, I keep hearing a lot of misinformation about the problem (notably from Linux users).
Mark Russinovich is a great talent, I often wonder where he would lead Microsoft in OS directions... but different fields I suppose. He's just always coming out with clear information, useful tools people need, and excellent research into things. Long may it continue.
Score: 0
I've said it before, Babylon2x, and it bears repeating: Mark Russinovich was the smartest single hire Microsoft has made in perhaps the last three years. And yes, that's counting some of the big hires during that period.
-SF3
Score: 0
Couldn't agree more :)
Score: 0
A few things Scott left out:
This affects gigbit networking only. Not 10/100. Most home users will be using 10/100 fast ethernet.
This will really only affect business users who's businesses have migrated to full gigabit 8and* Vista (A rarity for sure), as well as enthusiasts and home users who have heavily invested in an HTPC running Vista getting their multimedia from a home file-server.
Again, while a a *few* people have reported this with Fast Ethernet, it's more likely those incidences are unrelated, or due to other issues.
The folks on 2CPU have pretty much narrowed it down.
the portion Scott included from MS's response may lead some to believe MS intended the gigabit slowdowns reported.
"Please note that some of what we are seeing is expected behavior," the spokesperson told Kingsley-Hughes, "and some of it is not. In certain circumstances, Windows Vista will trade off network performance in order to improve multimedia playback." Then reviving an old Microsoft phrase that was so often used in the 1990s to officially explain away behavior that couldn't otherwise be explained, Microsoft wrote, "This is by design."
Don't let this quote mislead you to think MS intended this exact outcome. Further down, they specify:
Of course some users, especially ones on Gigabit based networks, are seeing a much greater decrease than is expected and that is clearly a problem that we need to address.”
So they will be fixing this, and it was not an effect they had intended.
Score: 0
I didn't leave it out. The priority switch takes place on all Vistas, not just those with gigabit networks. I have seen it with my own eyes impact systems with 10/100 networks, although the argument is that the slower users won't feel so much of the sting by percentage.
This is not a bug, or an isolated problem. This is the way Vista was intentionally programmed to behave. It was a calculated tradeoff. What was not intended was the disappointment people have in reduced performance.
-SF "Paid Professional Journalist" 3
Score: 0
I didn't leave it out. The priority switch takes place on all Vistas
i stated as much below.
I have seen it with my own eyes impact systems with 10/100 networks
Your eye-witness accounts seem to contradict much of the testing that has gone on over at the 2CPU forums, not to mention my own.
The loss on 10/100 seems to be, according to them, around 10%, and according to my own tests, nil. This is pretty negligible, either way.
I am using 10/100 at my house (Realtek 8139, both ends). My Vista system averages about 30MB/Sec on large file transfers over the network to my server. This does *not* change when running any media app to play Mp3 files. (It does change quite a bit on video, but video takes quite a few more reads/sec than a simple MP3, attributable to a hard-drive bottleneck, not the network)
This is the way Vista was intentionally programmed to behave. It was a calculated tradeoff. What was not intended was the disappointment people have in reduced performance.
Let me paste the relevant MS response once more:
Of course some users, especially ones on Gigabit based networks, are seeing a much greater decrease than is expected
Do you *seriously* believe Microsoft *intended* a 90% drop in performance on gigabit based networks???
What, in your mind they just figured no-one would notice?
So much for lack of bias...
"Paid Professional Journalist"
Paid != Correct, good, honest, or unbiased if that is the implication you are trying to make there, and no, this is not an accusation (that would be the bias comment above).
it seems we're in agreement on everything except the effects on 10/100 and on whether a 90% drop over GigE was intentional.
I can understand (though not agree with) some drop in network performance. I don't believe it is necessary, but whatever. I do *not* believe they intended it to be a 90% drop. Your implication above is that it was, and they just figured we were all extremely stupid and thus wouldn't notice. I find that *very* hard to believe.
{Edited. Put in HD transfer rates instead of network transfer rates. My bad. Fixed.}
Score: 0
You're being a troll again, PC_Tool. Luckily, my first thought was immediately put to rest by Scott himself--that he hadn't left out what you said he had. Do you realize how many words you've used to say nothing?
Learn to play nice, PC_Tool. Didn't your mother teach you that? Regardless of your apparent beliefs, you do not own these boards. (Yea, I know that comes as a shocker to you.)
Score: 0
You can pick up GigaBit routers for less than $20 these days. GigaBit add-on cards are cheap. Microsoft is releasing a Home Server. Consumers are hardly able to buy a new machine without Vista these days. Most cheapo consumer network storage has GigaBit ports. My whole house is GigaBit for a few dollars. It's a question of very short time until it becomes the standard. Vista should look forward and not backwards, especially when it comes to network where everything is moving (they didn't for graphics...). It's the type of bug a team of clever visionary people should not have missed... maybe... 80% of the time ;-)
Score: 0
Thanks for your input.
/sarcasm
Do you actually have anything to add, or are you just here to start name-calling?
The irony of your post is loud and clear.
Score: 0
Never said it wasn't a problem, and perhaps my estimates of who has GigE are understated, but walking into any of our local computer shops, I am hard-pressed to find GigE prominently displayed amongst the enormous selection of 10.100 equipment.
Score: 0
enalposi: Gigabit routers for $20 since when ? Maybe you were thinking about Gigabit Switches there? And anyway they cost a bit more than $20.
Score: 0
Yes, sorry - switches. With delight I notice wireless N routers with GigaBit starting to show up below the $100 mark.
Score: 0
I guess it all depends on how well your audio driver is written. I know the Creative driver for my Audigy 2 is horrible. I can't even listen to iTunes without the whole system stuttering a bit.
Score: 0
Audio drivers, audio hardware, etc have no effect.
This is a glitch in the prioritization of Windows Audio on systems that are using Gigabit networks.
Score: 0
It's refreshing to see Microsoft handle this so well and Russinovich does a great job at it. My only question is why would a "professional" os favor a multimedia media player over network under stress? What is it, a boy or a girl? If it is intentional, than can users needing the other way around switch priorities?
Score: 0
i have vista running WMP all day everyday, it runs just fine, vary rarely dose it affect my other apps and network, but most of that has to do with hardware.
people runing vista with an older cpu and chipset, and minimal amount of RAM would have issues with this alot. witch alot of vista upgraders have.
Score: 0
still the fastest microsoft os to date, my machine is flying at home i don't use media player though so i haven't noticed any problems the priority situation is causing
Score: 0
No, DOS was much faster. :P
And for completely the same tasks, XP is still faster on equivalent hardware. Would you like to try again?
Score: 0
Gigabit, or 10/100?
This issue only affects gigabit, it seems.
Scott leave that tidbit of info out?
Score: 0
Hopefully Microsoft continues to use Mark Russinovich in this fashion......
...to provide a no-nonsense, technical explanation for things without any "spin".
Of course they arent ALWAYS going to be so forthcoming (nor do I think they should), but he plays a great role with the community by talking about this stuff, and I hope they dont try to get him to MS-Speak. The community respects him a lot.
Score: 0
Yet another reason not to upgrade to vista :p
In the past, the user was God and the system did only what it was told. Now MS has decided the system is God and the user must pray to it; making sacrifices of time to the UAC.
Score: 0
Give the user too much control and the OS is susceptable to attack. Give them too little and people complain about anything they don't understand or want. It is a tight rope that must be walked by everyone.
Who do you think there are more of tho? People who have, and utilize, gigabit network interfaces or people that want to hear stutter free music and barely know how to turn on their computer?
As a solution just don't use WMP. It's not all multimedia services that change their prioritization just WMP11. So if you have a problem with the reduction in network speed simply use one of the many media players freely available on the internet.
Score: 0
In the past the user didn't have to worry about software modifying system crucial files without their permission either. Those days are gone hence UAC. Also UAC has nothing to do with this article. In many people's minds network threads should be more important then multimedia threads. Yet if MS didn't increase priority of multimedia, users would complain that their movies are choppy while they're trying to IM, Post to their blog, watch YouTube, read a friends myspace account with 20 dancing gif files, all while playing the latest flash game from yahoo and talking over skype, while downloading the latest firefox.
Score: 0
well they made it so people dont have to step up and play god, because i'd say and its a guesstimite.. but about 95% of the home pc users didnt even have a clue that they were "god" on their OS, hence all the zombie PC's out there that are hooked up to the net.
atleast it gives you the option, and if you cant figure out how to be "god " on vista maybe thats a good thing and not bad
Score: 0
I guess the real question is the professional version of Vista favoring network stacks and the home user multimedia?
Score: 0
Nope.
All versions of Vista contain this nifty little oversight.
Again, near as anyone can tell with anything resembling reliability, this really only affects systems on Gigabit networks. Not 10/100 fast ethernet.
Score: 0
I guess one could argue that in the office environment there should not be too much multimedia playing going on. Likewise if Admins want to, they use group policies to make it so that the user can't even run Windows Media Player.
Score: 0
Darn near everyone here has MP3's stored on flash drives and playing throughout the day.
Score: 0
An old 1997 PC with pentium 200Mhz used to play mp3 without a glitch.
It took a decent 2000 500MHz to playback DVD.
Now : what is the mediaplayer doing to need that much CPU power today ?? (same question goes for any other microsoft product, and even many *nix softwares )
Something else now : what if the media player plays high resolution online medias ? It has top priority to have no glitch, but glitches will come from the data transfer itself. Well thought redmond :-)
Score: 0
This is such a wonderful point.
Score: 0
WMP probably changes it's priority regardless of what it is playing. So while it would be needless if it was playing a cd or mp3 (depending on system) it would be almost required for dvds (once again depending on specs) or HD-DVD/Blu-Ray.
Perhaps WMP should make the differentiation between what it is playing but it doesn't seem to.
No matter what they do some people won't like it. The most vocal in this case are the minority of users and those most able to install and use something else.
WMP is just one of many free media players. If you don't like it use a different one!
Score: 0