a 32 Core systems hehe from a friend of mine.

Well @ 2CPU.com Slash a friend of mine from that place is building something out of this world really it's an 8-Way Quad Core AMD OpteronMP 8347 HE Barcelona with 16x 1GB ram, he has not everything on houuse 4 of the 8 CPU's and the mobo with the PSU and the case , here a quote from him on how he proceeded

Zitat von Slash;770415

PIC TIME!!!
Captions are above the pic they go with.

Case after removing the PCI slots bracket thing. I actually found a computer part today that I don't know what it's called. Also marked up the slot where the tabs on the bottom of the cards will go. Have to turn those notches into a channel.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0001.jpg]

Motherboard with Opterons in it. Didn't need to put them in there, but I couldn't not do it. 16" x 13"
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0005.jpg]

Expansion board with HTx connectors. This will stay in the box 'til I get my heatsinks.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0007.jpg]

Marked up top rail for notching. Had to do this to fit PSU's. Literally going to have 2mm between motherboard and PSU's.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0009.jpg]

Top rail after notching. It looks a lot closer in the pics than it is. I have about 1/8" all the way around all components there.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0016.jpg]

Shot of the 750W Silencer and another old PSU I used in place of the 910W since it's the same width and height. One reason I went with the PC P&C's is that the vents are all on the one side and on the back. Having them on both sides or the top/bottom would have made this placement impossible.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0014.jpg]

Still need to cut the back plate for the 2 PSU's.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0013.jpg]

Here you can see 2 expansion cards I installed to test the fit. The video card is in the #1 PCIe slot, and the ethernet NIC is in the #5 PCI slot. You can also see the I/O ports line up good. Also, you can see the gap where I had to move the bracket assembly down. The old rivet holes are also visible next to the new rivets. Talk about a headache making sure I had the motherboard standoffs lined up with the motherboard, tapping all the standoffs, measuring and matching up the I/O bracket for up and down spacing and getting it right with less than 1mm to play with.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0012.jpg]

Another shot of the I/O bracket I relocated. It's off by a red hair, but all the ports are perfectly accessible.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0010.jpg]

Another angle of the notches for the toroids and caps. Thought about cutting the U-Rail down to a 90° instead by removing the whole entire side, but decided the case needs all the structure I can keep in it considering the weight it will be carrying.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0018.jpg]

This is how much room I have between the PSU and the motherboard with the PSU flush against he bottom of the case. Once I add in the shaved down bolt heads for the casters, it'll be about 1mm.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0020.jpg]

Couldn't resist. Attempted power-on with the 750W, with old, random heatsinks sitting on top of the CPU's. Left the NIC and the video card plugged in.

Press the power button, PSU fan starts, BIOS code LED readout on motherboard flashes a few boot numbers, then gives the 4 beeps in a row and hangs at code 28, which is size/detect RAM. No RAM yet, so it should hang there. Everything looks functional.

In that state with 4 CPU's and a 6600GT it draws a peak of 178 watts.

16 cores. Halfway there.

Alles anzeigen

I'll keep this up to date on how he completes this hefty task.

Zitat von Chosen_One

Professional hardware in consumer-case. Well...nice bicolage but I prefer the professional hardware in a professional case. This is what I see in our computer centers.

Let me guess. He will install Windows XP on the system

Sorry, for being not so enthusiastic...

he no and no and no

Hi is going to install Win 2008 Enterprise Edition, this is his workstation and the coolest thing he made the impossible possible he modded an expensive Lian Li server case of around 300 eur to fit such a setup this I find much more interesting and clever minded than waisting money on a Tyan VC50 case of about 3000 eur or more rofl

Innovative people like Slash, they have the spirit to interest me alot more than rich people who have money as power to buy everything and try to make others look as if they are nothing... And really all the moeny he used was no more that 2500 USD, he played it very wise for this project, here he writes on how he got this gear, Slash is nnot to be underestimated, he is a smart guy and he knows what to take on and what not.

Zitat von Slash;769919

Thx for the confirmation that you can OC these.

CPU's were $165 shipped, total for 8.

Picked up a Lian-LI PC-71B for $90 shipped. Plan to remove a couple brackets, etc for clearance, but the 4" space above the motherboard backplate should fit the board to the top of the case with about 1/4" to spare. Black aluminum is (in theory) the best case material for radiating and conducting heat out. If I plan to dump 800W-1500W inside it, I'll take anything I can get to work in my favor.

I like my old PC-70. That thing's outlasted 4-6 total system upgrades since I bought it in '01.

Contacted PC P&C about customizing a 1200W. Would have been $600+. Plus it's 4-6db louder. Back to the 750's. The Silencer 910 is the same size with an extra 14A at 12V. If the 750's come up short, I can swap for 910's without modding my mods.

TYAN S4985G3NR-SI Quad 1207 with 6-core support for later...............$900
TYAN M4985-SI 4 socket expansion board.........................................$575
8 pairs of 2x1GB DDR2-800 ECC REG(16 sticks 1GB each)....................$300(hoping to eBay these for $150-ish)
2 PC Power & Cooling 750W Silencer PSU's........................................$240
8 ARCTIC COOLING Freezer 64 Low Profile's......................................$200
8 Opteron 8347's(Had to cheap out someplace. Will upgrade later.).......$165
LIan-LI PC-A70B...........................................................................$90
2 PCIe 8-pin to SSI 12V 8-pin adapters.............................................$20
TOTAL Guesstimate.......................................................................$2500-ish

That's less than half what I see the Tyan systems go for. I'll watercool it if the fans are too loud or I see my OC temps going too high.

Alles anzeigen

he thought this and yeaps it was correct , heck he even got the parts he seeked cheaper than he thought

Zitat von Slash;770246

Yeh, johnny, the non "-SI" parts do not support newer CPU's. Some quad cores are also not supported, but most are.

Everything's ordered. Final price list:
TYAN S4985G3NR-SI Quad 1207 with 6-core support for later...............$855
TYAN M4985-SI 4 socket expansion board.........................................$580
8 pairs of 2x1GB DDR2-667 ECC REG(16 sticks 1GB each)....................$112
PC P&C 750W Silencer for expansion board and a drive or 2 .................$90
PC P&C 910W Silencer for motherboard + 1-2 video cards...................$182
WD Caviar Green 500GB.................................................................$40
DVD drive.....................................................................................$15
8 Dynatron A86G coolers...............................................................$189
8 Opteron 8347's(Had to cheap out someplace. Will upgrade later.).......$165
LIan-LI PC-A70B...........................................................................$90
2 PCIe 8-pin to SSI 12V 4+4-pin adapters.........................................$20
WD Caviar Green 500GB.................................................................$40
Aluminum 6061 T-6 Bar Stock.........................................................$32
Aluminum 6061 T-6 1mm sheet.......................................................$21
Stainless 6-32 50x 2.5" bolts 300x nuts, and 100x lock nuts..................$35
Nylon washers #6 ID 5/16" OD 50x..................................................$6
TOTAL ......................................................................................$2428

Came in $78 under my original guess.

I bought a lot more hardware, stock, and sheet than I needed for this job, but I'll use it on other things.

The Dynatron coolers are the ones Tyan uses on the VX-50, so I know they will fit. I'll find something quieter later.

Will set the sheet (expansion board plate) at one height, then use stopped-off nuts to float the board at exactly the right height for the card connections using the length of the 2.5" screws. Will cut off the excess.

Bar stock will be used to support the plate in two parallel runs.

Bar stock plus 2 2.5" screws will be used to make backplates for the heatsinks.

Went with the 750W plus the 910W since that's enough to trip my 15A circuit. I basically have a system now with 2 12V rails, a 60A and a 74A. 134 amps at 12V is more than most car's alternators supply. That's 1608 watts. With 88% efficiency, if I max out both 12v rails and nothing else, that's 1800 watts, or 15.7A at 115V. Figuring 115V, since that's typical drop for 125V at full load.

Alles anzeigen

Just awesome to build a home built super computer with such a budget.

And some more progress on that 32 core system heh finally some news again

Zitat von Slash;770862

Quick update. Been moving slowly lately since I've been busy with "real" work.

Got both boards mounted now, but need some tweaks to level expansion board and make sure I don't have any stress on the HTx cards. PSU mounts are completely done. Can fit 4 PCIe x16 cards plus 1 PCI card. Used a card with a backplate on it to test worst-case fit.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0031.jpg]

Top bracket fit testing. This will be riveted to the top of the case, and will have 3 small stud posts to hang the expansion board on.
[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0034.jpg]

More to come soon

Zitat von Chosen_One

What will he do with the system? Windows Server 2008 EE is no workstation-OS. And I guess 16GB RAM is not enough. 512MB per core? Sounds not impressive...

btw: Innovative people like Slash? Innovative? Explain, please...

Yes it's very innovative to buy a 32 core system what normally would cost you over the 15000 USD for under the 2500 USD...and it's also innovative because it's something you don't just see every day, @ 2CPU.com a 2CPU system is avery normal thing, and we talk 2 CPU's so Dual Quad Cores for example.
The system will be used for Folding @ Home and for hydro simulation app for designing boat/ship/offshore supply vessel hulls and drives.

16GB of ram is indeed somewhere on the tight edge, though he was planning to upgrade this someda in the future, but for now it's fine.

does not mean others have to go with your opinions, they still remain your options @ 2CPU we thing different.

He is looking for more ram btw , here some updates and some benchmark scores, as the OS goe's it's not that abd as it seems.

Zitat von Slash;770907

Not really for my use. Looking like 24GB right now.

Got the motherboard running with 16 cores until the 50mm fans come in.

Running Windows Enterprise Server 2008, I get a passmark CPU score of 8138.
SiSoft Sandra arithmetic 'mark gives 85.7 GFLOPS and 113.8 GIPS.
SiSoft Sandra multi-media 'mark gives a Float score of 208.5 Mpixels/sec and an Integer score of 347 MPixels/sec.
SiSoft Sandra Cyrpto 'mark gives an aggregate score of 1.23GB/s.
Cinebench will not load. Guess it won't run on this OS?

Highest power draw at the wall was 385W. As a worst-case test, I have the 750W supply currently running the motherboard. The 910W will power it once all is final.

Think some of the super-cheap 1GB sticks I bought are flaky. Not too surprised...

btw this was dobnne with only 4 quad cores heh 16 cores total this is just the half of the total system, more tests soon with all 32 cores.

Zitat von Chosen_One

Of course. Think whatever you want.

btw: Maybe your friend should get a copy of the VMware ESX Server. If the system is compatible it's a good basement for such a system.

Hmm good idea I'll tell him that hehe thx man.

Well here's the rest of his build all 8 CPU's thus 32Cores and all the ram has arrived

Zitat von Slash;771243

Up and running. 32 cores, 16 memory channels, 20GB RAM, Total 60.8GHz CPU, 84.8GB/s theoretical memory throughput.

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0036.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0037.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0040.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0042.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0043.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0045.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0049.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0050.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0052.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0053.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0055.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0057.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0059.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0061.jpg]

[Blockierte Grafik: http://i291.photobucket.com/albums/ll283/SlashG4/SANY0063.jpg]

32 core results:
Passmark CPU score of 15353.
SiSoft Sandra arithmetic 'mark gives 170.68 GFLOPS and 226.2 GIPS.
SiSoft Sandra multi-media 'mark gives a Float score of 550.5 Mpixels/sec and an Integer score of 688.3 MPixels/sec.
SiSoft Sandra Cyrpto 'mark gives an aggregate score of 2.44GB/s.

Highest power draw at wall on the 910W PSU is 396W.
Highest power draw at wall on the 750W PSU is 406W.

910W PSU is at about 40% load, with about 45A on 12V to spare.
750W PSU is at about 50% load, with about 30A on 12V to spare.

Alles anzeigen

As you can see she's all but complete 8 CPU's 32 cores and tons of ram :cool: