Computational machine

[cedev]

I am looking to build a machine for computational purposes with the following specs.

Motherboard:
* ASUS Z10PE-D8 WS
* SUPERMICRO X10D

CPU:
2 x Intel Xeon E5-2660 v3

RAM:
8 x 16Gb

Scratch storage:
1TB Effective (RAID 0 or RAID 5 or RAID 6)
* SATA SSD
* PCIe SSD (preferably)

Longer term storage:
4TB Effective (RAID 0 or RAID 5 or RAID 6)
7200RPM HDD

Chassis:
Tower format
Any suggestions?

PSU:
Any suggestions?


Is the integrated graphic card enough to run Linux on it?
Can the PCIe SSDs be arranged in RAID?
Is there anything else missing?


Thank you for your help,
Cédric

 

[PP Mguire]

What kind of computational purposes?
Do you plan to add any GPU power later for double precision purposes?

I'm 90% sure you can't RAID PCI-E with NVMe, which is what you'd want to go with if you need very quick storage. Something like the 1.2TB Intel 750.

 

[cedev]

Mechanical and/or fluid flow computation for engineering.

At the moment I am limiting to CPU computation because of cost (TESLA is slightly too expensive).

Do you think using this intel PCIe drive for scratch space and normal SSDs for system would not slow down the system?

Thank you for your suggestions

 

[PP Mguire]

Mechanical and/or fluid flow computation for engineering.

Got it.

At the moment I am limiting to CPU computation because of cost (TESLA is slightly too expensive).

Don't need to do Tesla. Check your program and see what Quadro or Firepro cards they can utilize.

Do you think using this intel PCIe drive for scratch space and normal SSDs for system would not slow down the system?

Well, how much normal system stuff are we talking? You could do a 400GB 750 for system drive and the 1.2TB for scratch and keep all data on fast flash storage. I guess it just depends on what is being used on standard SSDs and how much speed the "normal" usage drive needs.

Thank you for your suggestions

To add that, if you plan on no GPU (well, would need a cheap GPU at least for video on the Asus board) you could easily get away with a good 650w PSU which will leave room for at least one computational GPU and wouldn't need such a large UPS to stick in front of it.

 

[cedev]

I will write the code for the GPU cards even though I have never done it before...

The normal system stuff was referring to the system and software, i.e. basically read only stuff. Your solution sounds good to me.

For the chassis, I found these.
Supermicro | Products | SuperWorkstation | Mid-Tower | 7038A-i with Part List
Supermicro | Products | SuperWorkstation | 4U | 7048A-T

In one of their manuals, I found this diagram:
http://ecco-qua.tk/?fi=xkdg0mj3eQ&fn=System_Block_Diagram.png

Would it make sense to have the following?
* 1 x 400Gb 750 on slot 6 for system and software
* 1 x 400Gb 750 on SLOT1 or SLOT3 for scratch of CPU1
* 1 x 400Gb 750 on SLOT5 or SLOT2 for scratch of CPU2
Would it be possible to enforce the CPU uses a specific SLOT?

If I decide to add some GPUs later on, would it be possible to do so, for instance on SLOT1 and SLOT5?

Thank you for your help,
Cédric

 

[cedev]

Maybe I should be looking at this to accommodate future improvements:
Supermicro | Products | SuperWorkstation | 4U | 7048GR-TR

 

[PP Mguire]

Maybe I should be looking at this to accommodate future improvements:
Supermicro | Products | SuperWorkstation | 4U | 7048GR-TR

Honestly, any chasis/board combo that provides a redundant PSU setup IMO would be best.

I will write the code for the GPU cards even though I have never done it before...

The normal system stuff was referring to the system and software, i.e. basically read only stuff. Your solution sounds good to me.

For the chassis, I found these.
Supermicro | Products | SuperWorkstation | Mid-Tower | 7038A-i with Part List
Supermicro | Products | SuperWorkstation | 4U | 7048A-T

In one of their manuals, I found this diagram:
http://ecco-qua.tk/?fi=xkdg0mj3eQ&fn=System_Block_Diagram.png

Would it make sense to have the following?
* 1 x 400Gb 750 on slot 6 for system and software
* 1 x 400Gb 750 on SLOT1 or SLOT3 for scratch of CPU1
* 1 x 400Gb 750 on SLOT5 or SLOT2 for scratch of CPU2
Would it be possible to enforce the CPU uses a specific SLOT?

If I decide to add some GPUs later on, would it be possible to do so, for instance on SLOT1 and SLOT5?

Thank you for your help,
Cédric

Every server board is different but I THINK you can designate slots to processor. Although QPI really makes that a bit pointless unless you run V1 chip and specifically define processors (no communication between the two). Not the best setup again IMO.

Yes, if you have enough wattage to cover some GPUs later I'd just put all your SSD storage on the bottom slots and leave the 16x slots near the top available for GPUs in the future.
For the money and for what's being done I personally feel a single 1.2TB 750 plus a 500GB SATA SSD (or 400GB 750 budget allowing) would be best unless you are wanting to run multiple instances of the software. Then it would make your above questions make a bit more sense.

 

[cedev]

That was exactly the idea to have several piece of software running at the same time.

What are V1 chips?

I'll start to get some pricing.

Thank you for your help.

 

[PP Mguire]

Xeons come in different flavors, but I derped when I said V1. I meant the 1*** series chips.
V3 denotes the 3rd generation, which is the 2011-3 chip.

The first number in the Xeon shows how many chips it can run in tandem on a single board. For instance your 2660 is mean for a dual CPU system. A 4669 is made for a quad CPU system. It represents how many QPI links are in the chip to communicate with other chips. A 1660 is meant for a single socket board. I'm not 100% on it, but placing two of those in a dual socket board should mean they run independently but I don't think you can actually do that. Meaning for dual CPU you need the 2*** chip and it shares all 80 PCI-E lanes and all cores act as one chip so to speak.

The 1.2TB 750 should independently have enough bandwidth to feed multiple instances of your software as a scratch disk anyways, as I'm thinking you'll still be CPU limited in terms of speed as it is. That is of course unless you can target separate scratch disks for each instance of the software. Since you're running the software it's up to you to figure that out.

 

[cedev]

Thank you for your explanation on CPU denomination.

Would you have any suggestions on where to buy the parts for shipment to the UK?