RSSAMD Claims of Intel Benchmarks 'Not Ethical' Scrutinized
By Scott M. Fulton, III | Published March 2, 2007, 5:33 PM
8:45 pm March 2, 2007 - After BetaNews' publication late this afternoon of our original story on ZDNet blogger George Ou calling into question the validity of recent AMD benchmark claims -- after AMD had called Intel's claims into question -- AMD changed its mind and decided to comment in full. Spokesperson John Taylor's response appears in full after our original story:
5:33 pm March 2, 2007 - After a press conference on Wednesday in which AMD Executive Vice President Henri Richard told reporters he's tired of being pushed around by Intel with regard to the accuracy of its recent performance superiority claims, ZDNet blogger George Ou turned the interrogation light back onto AMD. As Ou confirmed, data AMD used to debunk some of Intel's performance claims had since been superseded by new data that substantiates Intel.
"I think we've been too quiet," Richard told reporters, "and I think part of that is because we're trained to be very honest, grounded in reality, truthful with our benchmarks, and I'm sick and tired of being pushed around by a competitor that doesn't respect the rules of fair and open competition."
Later, in a video interview with ZDNet reporter David Berlind, Richard expanded on those comments, elevating them into a serious charge: "I think using five-year-old benchmarks that nobody uses any more to pretend to have a better performance, is not ethical."
But one of the slides from a presentation Berlind gave during the conference cited official SPECint_rate2006 results dated last January 29, showing the best peak performance of systems tested for the Standard Performance Evaluation Corporation using comparable dual-core Opteron and Xeon processors. In that slide - which AMD also provided to BetaNews last February 7 - an AMD Opteron 2220 SE-based server outperformed an Intel Xeon 5160-based system by a narrow margin of three percentage points, in a test in whose category Intel had earlier claimed superiority. Intel's numbers, cited by AMD, show SPECint_rate2000 results for systems with the same two processors, in which the Xeon system performed 57% better than the Opteron system. AMD has pointed out Intel's numbers come from an outdated benchmark.
 | A composite of two slides from an AMD presentation last February 7, showing SPECint_rate performance figures for Intel Xeon 5160 and AMD Opteron 2220 SE processors, as cited by both companies. In the original presentation, AMD has one slide with the right-hand "reality" side partly obscured; in the second slide where "reality" is revealed, the "perception" side is obscured. |
BetaNews cited these figures in a February 7 story that also examined the efficacy of AMD's claim. The numbers AMD chose to cite, we confirmed then, represented peak performance of the two systems tested, whereas the base performance figures for the same two systems showed the Xeon server with 6% better base performance.
The day after BetaNews' initial report, Dell submitted a new test result to SPEC, for a Precision 690 server with a Xeon 5160 processor clocked at 3 GHz. The new figures give the 5160 a best peak performance score of 54.9 - 6% better than AMD's best peak performance for Opteron 2220 SE - and a best base performance score of 53.2 - 17.6% better than the AMD.
In a scathing blog post this morning, George Ou invoked the dreaded "H" word in questioning whether AMD should have known the SPEC figures were updated during its press conference on February 28, and pulled the slide from its deck - on the same day Richard planned to accuse Intel of being "not ethical."
"It is a well-known fact that all companies hardware or software will present their best foot forward and favor the benchmarks that makes them look good," Ou wrote. "But leaving out your competitors up-to-date data to make it look like you're ahead in the same benchmark is very disturbing and there is a world of difference between cherry picking benchmarks versus cherry picking data. The only thing worse than that is hypocrisy."
AMD thought long and hard today about how to respond to Ou's claims, before electing to decline comment to BetaNews.
In an interview last month with AMD's senior client performance analyst Mark Welker - who incidentally is a member of SPEC committees - he acknowledged that both AMD and Intel may tend to highlight the benchmarks which most favor their processors at any one point in time. But he also suggested a remedy to the confusion to which such habits can give rise.
"From a testing standpoint, we need to look more at what a real user does," Welker told BetaNews. "There are users out there who do one thing at a time; there are also crazy users out there who have fifteen, twenty things open at once, [all of it] legitimate...We're trying to go out, talk to people and the press, survey people, and try to get end-user scenarios so that we can ask the end users, what would make sense to them?
"I want a real-world-style benchmark that...won't be a CPU-specific benchmark," Welker continued. "I can bet you dollars to doughnuts that I can write one that makes mine oh-so-much better than Intel's, and Intel can write one that looks better than us. It's going to happen. But I think if we write stuff that's for systems and stuff that will generally benefit the end user, then if the benchmark shows up that we might be weak in some spots but strong in others, that gives us something to work on, and we can evolve our processors and our systems, because we're even more of a systems house than we used to be. We can evolve our systems to help the end user more, to go out there and say, 'This is what's going to make a difference to you,' because it's not just about frequency any more. It's about the whole system."
Tonight, AMD spokesperson John Taylor provided the following response to BetaNews, which we present in its entirety:
As for responding to Mr. Ou’s blog, we clearly showed the date of the benchmark and link to the Web site we were pulling from for this very reason – benchmarks can get updated ahead of the corporate legal review process. This was a matter of weeks/days, not months/years as in the Intel presentation from last week. That same Intel presentation made some comparisons to single-core AMD Opteron processor performance from several years ago, and less than half the time made comparisons to current-generation highest-performing AMD Opteron processors.
We also clearly marked the names and model numbers of the processor models we were comparing.
The immediately previous slide that Mario Rivas presented – that Mr. Ou does not show but BetaNews thankfully does – spoke to Intel portraying a 57% advantage over AMD Opteron-based systems. We were addressing the misleading perception of gross performance advantages that are at the center of Intel’s months-long marketing campaign which reached new lows during a presentation to Wall Street analysts last week.
Even if Intel eked out a slightly higher score in recent days, the basic point remains unchanged; roughly 50 percentage points of Intel’s claims get lopped off. And certainly, we may again top the new Intel scores in the weeks/months to come. There is a significant difference between Intel endeavoring to foster perceptions of a 57% performance advantage, and going back and forth on a 2-3% difference. And AMD Opteron remains the standard-bearer for performance-per-watt, an important direct comparison that Intel continues to avoid.
There may continue to be updates of benchmarks scores that amount to back-and-forth until the introduction of AMD “Barcelona” server processors. “Barcelona” is the most significant architectural enhancement from AMD since the introduction of the AMD64 architecture and AMD Opteron processors in April 2003.
Come on Intel and AMD. Stop looking at performance for once. Please put your attention on environment. I meant reduce heat generation and power consumption. Now I am finding ways to reduce my 2GHz core2duo for reducing heat and noise from the fan.
Score: 0
|Agreed!!!
Score: 0
|I think this is a red herring.
As others have posted, there will be a continuing see saw competition between the two major players. Just as it should be!
And as far as benchmarks, who cares what either Intel or AMD claim! As far as I am concerned, no one should take either of their claims at face value!
Each company is going to slant their tests and subsequent results in a way to optimize and stress the advantages of their strategic difference anyway.
There are plenty of independent 3rd party testing labs available, and I would rather read what they have to say.
I just want the competition to continue.
Score: 0
|The wheel will turn, it always does.
Not so long ago it was AMD totally out of sight and Intel that couldn't compete.
Right now it's Intel's Core 2 Duo that is out of sight.
It'll change about, it always does.
But ethical (or rather non-ethical and particularly illegal) business practices are another matter.
If Intel get found guilty of playing that game and breaking the law then no amount of a technical lead could - or should - help them.
Score: 0
|Richard was hoping that we wouldn't know the difference between an old benchmark and an old test. He might have been right.
Score: 0
|Well, filter processing taking 2 minutes instead 1 minute (or even 1,5 vs 1 minute) is quiet a lot if you have to deal with several filters on multiple images your whole work time... Framearates are also very important for all gamers. So in the end, the only leftovers are those who use quad core CPU's to browse the web and send joke emails...
Score: 0
|Does it really matter that one proceesor will finish rendering a filter in Photoshop 2 milliseconds faster than the other? 90% of Americans use thier PC only for web surfing and sending joke emails.
RAM and HD RPM speed is everything in the real world unless you are one of those tools that uses your PC only for playing WoW.
Score: 0
|I am not American, but I do agree in that obvious sense that most users do not pay enough attention when they think about their PCs. I do not even try to convince my friends that the bottlenecks of their system is not in the place they think, e.g. that buying a faster HDD (neither RAM not CPU) would be better for decompressing large RAR files or that the chipset of the mobo can be an issue when someone uses a lot of USB devices.
Most people do care of isolated synthetic benchmarks, which accuracy and impact on the whole system is almost negligible.
What is interesting the cult of benchmarks is widespread even among pretty advanced users.
Score: 0
|I'm baffled - what does not being American have to do with your opinion?
Score: 0
|read the post he's replying to...makes sense if you do that...and if you have a broad grasp of understanding...wink wink
Score: 0
|blah blah blah!
Here's the reality. Ever since the mid 80s, when Intel decided to ignore government stipulations that said it (Intel) had to release licensing and patent information on it's 486 processors that were being used in ballistic missiles for the U.S. military (you know the government likes to have a backup plan), and AMD ended up winning a multimillion dollar lawsuit against Intel, Intel has been on the offensive and has been bullying their way around in the I.T. industry. The reality is, it doesn't really matter whether Intel processors outperform AMD processors using benchmarking specifics from 2000 or 2006. What matters is, Intel continues to try to limit AMD's visibility. Henri Richard is right, AMD's been the nice, quiet, conservative company that has gained shares of the market due to the quality of their products. Mind you, they did this with absolutely no advertising campaign other than word of mouth. That in itself is a magnificent feat.
As I'm sure you're aware, the only company advertising AMD based computers on the national media markets is Dell, and very rarely do they do that. Most of the time, Dell is advertising computers with Intel processors in them, and it wasn't until they acquired Alienware that they even began to offer servers, workstations, PCs, and laptops with AMD processors in them. HP has yet to advertise an AMD based computer, and Gateway tries to stay out of the limelight.
Thus, the two major computer manufactures in the world (HP and Dell) are more focused on promoting Intel based computers.
Why? Because Intel pays them to advertise Intel based computers. That is, Intel gives these companies major price breaks to stay Intel. And how can they afford to do this? Because they are a multibillion dollar company!
The reality is, the AMD vs. Intel battle that continues to rage is close to an end. Too many people are becoming aware of Intel's unfair business practices. So even though Intel's newest processor line may have a few point lead over AMD's newest processor line on the benchmarks, and even though Intel is spending billions to prepare manufacturing of the 45 nm lithography technology which will be implemented into their newest processors, AMD's strong, moral business ethics will always keep them ahead of Intel in my eyes. If Intel were to stop being so greedy and corrupt, and stop trying to limit AMD's visibility, then I might be inclined to support them more.
It's like the AT&T vs Verizon battle continuing to rage on. The reality there is, both of these corporations were a part of the original AT&T before the 1984 divesture. Now although they continue to compete with each other, they are not using unfair business practices to do so.
Thus, if Intel was to realize that fair competition creates good business, and good business creates better profits, then I wouldn't be so outspoken against Intel.
Just to let you know, I've been building servers, workstations, and PCs, and testing the benchmarks, for 13 years professionally, and more than 20 years on my own.
Score: 0
|From an end user I have two smilar processors for my gaming boxes built almost identical and Intel blows my AMD box away and has less issues.
I am using amd 3400+ as my main box but INTEL IS my Server box.
Score: 0
|My server is an AMD box, and my PC is an Intel. :P
Server: 70 days uptime.
PC: Is shutdown regularly.
Only reason your PC would be unstable is if it's incorrectly configured or using cheap parts. It's a 1 in 1000 chance it's a CPU issue. CPUs rarely fail.
Score: 0
|"Only reason your PC would be unstable is if it's incorrectly configured or using cheap parts."
exactly!
Score: 0
|I am inclined to believe AMD's allegation that Intel is an unethical company. They have long had a reputation as a company who takes great liberties with inventor's work. They are a founding member of the Coalition for Patent Piracy, while they call it the Coalition for Patent Fairness the truth of the matter is that their idea of fair is getting other's intellectual property for nothing. The Coalition is well known by both independent inventors and large companies for an unprecedented level of arrogance and lack of morals or ethics. This is easy to verify, just review their litigation histories.
For a great example of the real Intel see Ray Niro's expose' at http://www.piausa.org/pa...ymond_p_niro_08_04_2005.
Now that is just one example of why Intel is a washed up tech company.
Ronald J. Riley,
President - www.PIAUSA.org - RJR at PIAUSA.org
Executive Director - www.InventorEd.org - RJR at InvEd.org
Washington, DC
Direct (202) 318-1595 - 9 am to 9 pm EST.
Score: 0
|Intel a "washed up tech company"? Yeah, ok, don't forget the straws for your nose so the sand doesn't get in. I have no doubt to them being unethical after I was in an IT position as a contractor in their server group awhile back. However, to equate ethics with success(or lack thereof) is stupid, and to then broadcast that as fact, is negligent stupidity. For that fact, show me any large organization that isn't ethically challenged. Even the ACLU has been known to use it's tactics abusively. Granted, the end results with them are a more positive result, but a wrong does not justify a further wrong in ethics.
Score: 0
|What about all the independent test from reviewers around the web? AMD should have researched more carefully before they opened their mouth. If Intel had pulled something shady, people around the web would have said something already.
Score: 0
|If Intel had pulled something shady, people around the web would have said something already.
Yeah, and then Intel fanboys would claim the dissenters were AMD fanboys, thus decreasing the chances of any really meaningful discussion. (Not saying AMD fanboys wouldn't do the same were the position were reversed, just that fanboys will be fanboys.)
Score: 0
|I think AMD has the right point of view on the matter, and ive been hearing some things about a new AMD technology that would make it much faster than intel-Although core2 is rivaled by nothing at the moment in my mind.
Score: 0
|I like AMD and all, but this is stupid. They just can't stand it that Core technology is superior to anything they can offer at the moment.
Score: 0
|Aren't we talking Xeon and Opteron? :P
Score: 0
|Let's wait for AMD to get the leadership again.
Score: 0
|