Is core i7 optimized for benchmark codes?

[zane]

Is core i7 optimized for benchmark codes, or the OS didn't use all the potentials of the new CPU?

I read a lot reviews and forums about the new Intel CPU core i7. The new Intel CPU looks great. For example, according to spec cfp2006,
viewtopic.php?f=52&t=135802, Core i7 is about 100% faster than core 2 for float point performance. Although someone suggested that Intel might optimize the CPU for the benchmark codes, I still decided to upgrade my computer from core 2(Q6600) to core i7(920). Because I thought the better bandwidth of core i7 would help the multithread computation. Well, I was wrong. My testing results made me very disappoint to the new core i7 CPU.

My new computer
CPU: core i7 920 at 2.66G
MB: asus p6t
memory: 6G DDR3 1600
OS: fedora 10 64bit

My old computer
CPU: core 2 Q6600 at 2.4G
MB: gigabyte p35
memory: 4G DDR2 800
OS: fedora 9 64bit

compiler: Inter ifort 11

The testing code is a openmp CFD(computational fluid dynamics) code. The code used about 500MB memory. The testing results:

thread no. core i7 920, core 2 Q6600, core i7 advantage

1 thread : 202.7s , 213.1s , 5.13%

2 threads : 109.0s , 109.3s , 0.27%

4 threads : 96.1s , 68.3s , -28.93%

(core i7 HT is off, it is slower if HT is on. the clock of core i7 920 is about 10% higher than core 2 Q6600)

The scaling of core i7 for 4 threads is really bad. Considering the bandwidth of core i7 is about twice of core 2, this result is really strange. I don't know why, maybe the OS didn't use all the potentials of the new CPU? I guess I should sell my new computer and try the Shanghai CPU next year.

[JulianL]

That is very strange. Even on 1 thread it's effectively slower because the 2.4 -> 2.66 GHz clock jump is a 10.8% increase so, even if performance doesn't scale linearly with clock speed increment, +5.13% still seems a bit low. It certainly seems to show no IPC improvement on your tests.

I notice that you have Fedora 10 on the old system and Fedora 9 on the new one. Could that be affecting anything? It's also interesting that you put an older release of Fedora on the newer system. Did you have trouble with Fedora 10 supporting the new system?

- Julian

[AussieFX]

Is that version of Linux NUMA aware?
I find it funny that all the intel crowd are now touting NUMA. When C2D was around they used to bag NUMA probably because it made AMD run better.

Thanks zane for the info and welcome to the zone.

BTW how hot is it running?

[xPliZit]

Maybe it has something to do with the compiler.
Nehalem has only 256KB L2 cache whereas all previous modells like Q6600 has like 8MB?
If the compiler generates code with greater than 256KB cache usage in mind all that data would goto L3 instead of L2 and L3 is somewhat slower than L2.
This would explain that some software is slower on Nehalem. Most intel optimized software out there expects L2 to be greater than 256KB.

Otherwise dont know.

[abinstein]

That is very strange. Even on 1 thread it's effectively slower because the 2.4 -> 2.66 GHz clock jump is a 10.8% increase so, even if performance doesn't scale linearly with clock speed increment, +5.13% still seems a bit low. It certainly seems to show no IPC improvement on your tests.

Why is it strange? Just because some Intel-bought review sites can show better performance, doesn't mean average users easily can. The big question is, well Corei7 seems good on benchmarks, so if I buy one off-the-shelf, do I get the benefit? I think from zane's experience so far the answer is negative.

I notice that you have Fedora 10 on the old system and Fedora 9 on the new one. Could that be affecting anything?

That (and pretty much everything else) is lame excuse. Shouldn't I use a new OS with my new processor? That's the problem of "optimized" benchmarketing, that they are distorting the facts that actual buyers will see. That's the problem of on-line reviewers being nothing more than a company's marketing arms.

[abinstein]

Is that version of Linux NUMA aware?
I find it funny that all the intel crowd are now touting NUMA. When C2D was around they used to bag NUMA probably because it made AMD run better.

He's running a single-processor machine, so it should have nothing to do with NUMA.

[maduroutmb]

I notice that you have Fedora 10 on the old system and Fedora 9 on the new one. Could that be affecting anything?

Given Linux Torvalds' tireless efforts to make Intel chips run faster, I'm sure that it crippled the Core i7 system.

[zane]

That is very strange. Even on 1 thread it's effectively slower because the 2.4 -> 2.66 GHz clock jump is a 10.8% increase so, even if performance doesn't scale linearly with clock speed increment, +5.13% still seems a bit low. It certainly seems to show no IPC improvement on your tests.

I notice that you have Fedora 10 on the old system and Fedora 9 on the new one. Could that be affecting anything? It's also interesting that you put an older release of Fedora on the newer system. Did you have trouble with Fedora 10 supporting the new system?

- Julian

Sorry, it is a typo. the old system is Fedora 9 and the new computer is Fedora 10. Because Fedora 10 is so new, I will try other OS(ubuntu and XP).
I have my test on other CPUs, including Phenom 9850,Opteron 8346 and Xeon E5410. Core i7 920 is not the slowest one, but the scaling is the worst one.

For core i7 920, the temperature is about 56~60C at 100% loading.

[carniver]

Is core i7 optimized for benchmark codes, or the OS didn't use all the potentials of the new CPU?

I read a lot reviews and forums about the new Intel CPU core i7. The new Intel CPU looks great. For example, according to spec cfp2006,
viewtopic.php?f=52&t=135802, Core i7 is about 100% faster than core 2 for float point performance. Although someone suggested that Intel might optimize the CPU for the benchmark codes, I still decided to upgrade my computer from core 2(Q6600) to core i7(920). Because I thought the better bandwidth of core i7 would help the multithread computation. Well, I was wrong. My testing results made me very disappoint to the new core i7 CPU.

My new computer
CPU: core i7 920 at 2.66G
MB: asus p6t
memory: 6G DDR3 1600
OS: fedora 10 64bit

My old computer
CPU: core 2 Q6600 at 2.4G
MB: gigabyte p35
memory: 4G DDR2 800
OS: fedora 9 64bit

compiler: Inter ifort 11

The testing code is a openmp CFD(computational fluid dynamics) code. The code used about 500MB memory. The testing results:

thread no. core i7 920, core 2 Q6600, core i7 advantage

1 thread : 202.7s , 213.1s , 5.13%

2 threads : 109.0s , 109.3s , 0.27%

4 threads : 96.1s , 68.3s , -28.93%

(core i7 HT is off, it is slower if HT is on. the clock of core i7 920 is about 10% higher than core 2 Q6600)

The scaling of core i7 for 4 threads is really bad. Considering the bandwidth of core i7 is about twice of core 2, this result is really strange. I don't know why, maybe the OS didn't use all the potentials of the new CPU? I guess I should sell my new computer and try the Shanghai CPU next year.

What compiler flags did you use for each? Can you try turning on HT and opening 8 threads on the i7?

[AussieFX]

To quote zane

(core i7 HT is off, it is slower if HT is on. the clock of core i7 920 is about 10% higher than core 2 Q6600)

[AussieFX]

That is very strange. Even on 1 thread it's effectively slower because the 2.4 -> 2.66 GHz clock jump is a 10.8% increase so, even if performance doesn't scale linearly with clock speed increment, +5.13% still seems a bit low. It certainly seems to show no IPC improvement on your tests.

Why is it strange? Just because some Intel-bought review sites can show better performance, doesn't mean average users easily can. The big question is, well Corei7 seems good on benchmarks, so if I buy one off-the-shelf, do I get the benefit? I think from zane's experience so far the answer is negative.

I notice that you have Fedora 10 on the old system and Fedora 9 on the new one. Could that be affecting anything?

That (and pretty much everything else) is lame excuse. Shouldn't I use a new OS with my new processor? That's the problem of "optimized" benchmarketing, that they are distorting the facts that actual buyers will see. That's the problem of on-line reviewers being nothing more than a company's marketing arms.

How true....and prophetic.

[mmarq]

Is core i7 optimized for benchmark codes, or the OS didn't use all the potentials of the new CPU?

I read a lot reviews and forums about the new Intel CPU core i7. The new Intel CPU looks great. For example, according to spec cfp2006,
viewtopic.php?f=52&t=135802, Core i7 is about 100% faster than core 2 for float point performance. Although someone suggested that Intel might optimize the CPU for the benchmark codes, I still decided to upgrade my computer from core 2(Q6600) to core i7(920). Because I thought the better bandwidth of core i7 would help the multithread computation. Well, I was wrong. My testing results made me very disappoint to the new core i7 CPU.

My new computer
CPU: core i7 920 at 2.66G
MB: asus p6t
memory: 6G DDR3 1600
OS: fedora 10 64bit

My old computer
CPU: core 2 Q6600 at 2.4G
MB: gigabyte p35
memory: 4G DDR2 800
OS: fedora 9 64bit

compiler: Inter ifort 11

The testing code is a openmp CFD(computational fluid dynamics) code. The code used about 500MB memory. The testing results:

thread no. core i7 920, core 2 Q6600, core i7 advantage

1 thread : 202.7s , 213.1s , 5.13%

2 threads : 109.0s , 109.3s , 0.27%

4 threads : 96.1s , 68.3s , -28.93%

(core i7 HT is off, it is slower if HT is on. the clock of core i7 920 is about 10% higher than core 2 Q6600)

The scaling of core i7 for 4 threads is really bad. Considering the bandwidth of core i7 is about twice of core 2, this result is really strange. I don't know why, maybe the OS didn't use all the potentials of the new CPU? I guess I should sell my new computer and try the Shanghai CPU next year.

There are very simple explanations.

* You didn't apply the most CORRECT compiler options, most probably very few ppl knows them because they might involve some little debugging and or tuning of the source code involved... as example that SPEC suit source code...

* Your compiler used is totally different from the one used by intel in their SPEC tests

* you didn't use or don't know the *tricks* ( i believe very few ppl know), because that programmable microcontroller that is responsible for the Turbo boost must be very clever indeed... and the tests we see with those 100% improvement where made with a chip at 3.6 GHz or close... if not higher.

[zane]

There are very simple explanations.

* You didn't apply the most CORRECT compiler options, most probably very few ppl knows them because they might involve some little debugging and or tuning of the source code involved... as example that SPEC suit source code...

* Your compiler used is totally different from the one used by intel in their SPEC tests

* you didn't use or don't know the *tricks* ( i believe very few ppl know), because that programmable microcontroller that is responsible for the Turbo boost must be very clever indeed... and the tests we see with those 100% improvement where made with a chip at 3.6 GHz or close... if not higher.

OK. I redid my tests on windows XP, with/without HT. I used similar compiler flags as SPEC 2006.
I also overclock Core i7 from 2.66G to 3.5G. Below are the setup and comparison.

Core i7 computer
CPU: core i7 920 at 2.66G
MB: ASUS P6T
memory: 6G DDR3 1600
OS: Fedora 10 64bit
Compiler: Intel Fortran Compiler Professional 11.0.069
Compiler flags: -r8 -xSSE4.2 -ip -openmp -opt-prefetch

OS: Window XP 64bit
Compiler: Intel Visual Fortran Compiler Professional 11.0.066
Compiler flags: /Qautodouble /QxSSE4.2 /Qip /Qopenmp /Qopt-prefetch /F1000000000

Core 2 Quad computer
CPU: Core 2 Q6600 at 2.4G
MB: gigabyte p35
memory: 4G DDR2 800
OS: fedora 9 64bit
Compiler: Intel Fortran Compiler Professional 11.0.069
Compiler flags: -r8 -ip -openmp

Dual Xeon computer
CPU: dual Xeon E5410 at 2.33G
MB: Tyan S5396A2NRF
memory: 8G DDR2 667 ECC Fully Buffered
OS: fedora 9 64bit
Compiler: Intel Fortran Compiler Professional 11.0.069
Compiler flags: -r8 -ip -openmp

i7t.png (16.05 KiB) Viewed 4377 times

*For Core i7, Turbo mode is OFF.

For four threads, Core i7 920@3.55G is slower than Q6600@2.4G!
The scaling of Xeon E5410 is the best.
I also did the test on Phenom 9850, the scaling is very good. I don't want to post the result here. Some people on other forum already called me fanboy.

[abinstein]

The reason of poor Corei7 seems due to the use of OpenMP library. In SPEC rates, multiple processes are running side-by-side without inter-process communication. This emulates server workloads. In OpenMP, multiple threads need to communicate to each other. It seems Core i7 ironically do not improve upon Core2 with respect to such workloads, even though HT seems perfect design for it.

I think this echos two points that I'm been trying to make before:

1. Different programs have different performance features. Some get big improvement from Core2 to Corei7, some gets none. Based your CPU purchase by a set of (biased) programs is pointless if you don't use those programs day in and day out.

2. There is some problem with Hyperthreading, and that is we don't know when it will be beneficial until we try it. If enabling/disabling HT requires a reboot, then having HT really becomes a nuisance rather than a feature.

[xPliZit]

I would like to see the phenom score.

[carniver]

The reason of poor Corei7 seems due to the use of OpenMP library. In SPEC rates, multiple processes are running side-by-side without inter-process communication. This emulates server workloads. In OpenMP, multiple threads need to communicate to each other. It seems Core i7 ironically do not improve upon Core2 with respect to such workloads, even though HT seems perfect design for it.

Obviously i7 not only didn't improve upon C2, they took a great step backwards in this case. I suppose the next version of ifort shall fix this though.

[mmarq]

"zane" has being doing quite a remarkable job for which we must thank him.

"zane" you are not a fanboy, because those can't do anything you can... and i can't see nothing particularly wrong with your approach, matter of fact looking at the FLAGS it seems that you tried to favor i7...

From big numbers from 65%+ to 101%+ of advantage of a i7, to zane test where actually the i7 performs worst on 4 threads and miserably on 8, is too much to be just a simple little thing ! Perhaps more testers can confirm either of the positions( amdzone?).

hmmm... could it be that Intel hand tuned the SPEC source code specifically for that test ??... and at the same time crippled the Xeon systems in order to show big percentage better ??...

Well, its an idea that i think that nobody will be chocked with in this forum .

[The_Ghost]

zane

Welcome to the Zone

we hope that you can come to some conclusion with your problem, and keep us updated on it

[scientia]

I know what the allegations have been but I have no way of knowing if they are true since I have not inspected Intel compiled code. It was sugested that Intel created hand tuned assembler code for key benchmarks and that when the compiler recognized these benchmarks it put the assembler inline to make critical sections run at top speed. If this were true then it would be a complete sham since no other source code would get the same big boost. A faster library is fair (since any code could benefit) but targeting specific benchmarks would not be.

BTW, carniver were you agreeing with Das Tipitz that Intel couldn't have a TLB bug 15 months after tapeout? And, are you saying that Intel hasn't bothered to optimize its fortran compiler during the same time frame?

[carniver]

BTW, carniver were you agreeing with Das Tipitz that Intel couldn't have a TLB bug 15 months after tapeout? And, are you saying that Intel hasn't bothered to optimize its fortran compiler during the same time frame?

I have no idea if Intel can fix a silicon revision within 15 months, all I know is that a lot of errata can be fixed in BIOS.

As for the fortran compiler update, that bit has to do with how I think OP's application isn't open source, so Intel has to wait till the profile data gets sent back to their headquarters before they can do something to it

[scientia]

Obviously i7 not only didn't improve upon C2, they took a great step backwards in this case. I suppose the next version of ifort shall fix this though.

Okay here is the thing. Nehalem's instruction ordering should be nearly identical to Penryn's. Even if the new compiler did not use all the SSE4 and better FP division that Nehalem has (which it does) that still wouldn't make the code go slower. That really only leaves the cache profile as the slowing factor and that can't be fixed with a newer compiler. In other words, if Intel was getting a boost off the big L2 (which many have suspected for quite some time) then that is gone with Nehalem and there is nothing you can do about it. You might be able to get part back by redoing the some of the cache profiles. It is also possible that the newer compiler is not yet fully utilizing the improved address handling with Nehalem but that would only be a small bump. The bad thing about both of these is that they work well with Shanghai as well.

[carniver]

In other words, if Intel was getting a boost off the big L2 (which many have suspected for quite some time) then that is gone with Nehalem and there is nothing you can do about it.

Also it may not be aware of the presence of L3.

The bad thing about both of these is that they work well with Shanghai as well.

It may be bad for Intel, but in general it is not a bad thing at all.

[scientia]

Also it may not be aware of the presence of L3.

Possibly but if I recall correctly, PGI only uses total cache rather than specific sizes for L1, L2, and L3.