Big data on a shoestring: New, refurbished, or custom build

[ClimateWonk]

Hi Folks –

I am trying to identify a good desktop primarily for use doing statistical computing with the R language. Not sure if I should be posting here or in custom builds or somewhere else. I have some very large files (Census & BLS stuff, mainly – as much as 14 gig in a file) and R likes to hold what it is working on in RAM, so I need to max out the RAM – I am thinking 16 gig to start, but with the expansion slots to go up to 64 if that proves to be necessary -- & plenty of hard disk space, at least 2 TB. Maybe a hybrid drive, or a small SSD as well? Though I think this is more of a nice-to-have than a have-to-have. & of course I need a motherboard that can handle that much RAM .

This project (I’m writing open source software to study income inequality) is a budgetless labor of love. I work in the nonprofit sector and am between jobs right now, so I want a least-cost system given my needs and reasonable reliability. It does not need to be blazingly fast, and I will not be doing gaming or CAD. I am guessing that an I5 system or the equivalent AMD is probably more than adequate. I would prefer Windows 7 to Windows 8. It is my understanding that I need Windows 7 Professional or better to handle that much RAM. Is this correct?

Since I am not doing gaming or CAD, I don't think I need anything special by way of graphics capability. However, I do have a question. My workstyle often has me with two or three hundred windows or browser tabs open at a time. This can slow my current machines considerably. If I want to eliminate this problem, is that a matter of CPU speed, RAM, number of processors, graphics capability, or what?

I'd be happy to take a pre-built if I could find one with my specs and price range, as I know I lack the expertise to competently evaluate components. But I am willing or even eager to go for a custom build, or a refurbished system, if I can get some extra warranty and ideally local service (Oakland/Berkeley or BART-accessible). Can you even get warranties for custom builds, other than the component warranties? I am completely ignorant about this.

Am I crazy to hope to bring this in under $1K? If I could meet these specs for $600 to $800 I’d snap it up immediately.

Any advice or recommendations greatly appreciated.

Warmest regards, Beanxx

For purposes of budgeting, I am trying to get a better handle on what all I need in a custom build. Here is my current list, with some initial thoughts on roughly the level of component I need. I would be very grateful if anyone could tell me if there is anything obvious I am missing. For instance, I do not know what normally comes with a motherboard and case. Do I need to separately purchase USB and other ports? WiFi capacity? Graphics processing?

1. CPU i5 or comparable AMD
2. Motherboard (should have expansion slots for 64GB of RAM)
3. RAM (16-32GB)
4. Hard drive (3 GB)
5. SSD? not sure I need this.
6. OS (Win7 Prof)
7. Power supply
8. Optical drive
9. Case

[rubberman]

Don't bother with the expense of building your own gear. Rent time on a large Amazon cloud system to do the analysis. When you are don't, you shut it down and aren't being charged for resources you don't need. If this is an occasional need, then it can be a LOT cheaper than building your own gear! Example, an m2.4xlarge image costs about $1.35USD / hour - and gives you 8 virtual cores, 68.4GB RAM, and about 1.7TB of local data. To build something like you want you would be looking at $3000-5000USD - a lot of time on the cloud! $1K? I doubt it. The RAM and OS are the expensive bits. I have a 3GHz, 8 Core, 8GB RAM, 2.3TB disc system that I built 6 years ago. It cost me $5000. It is still my primary home workstation. Now, I would get a decent laptop, and when I need extra computing power, I would push it to the cloud.

[ClimateWonk]

Hi rubberman! Thanks for your thoughts!

I have actually identified a number of systems -- gaming machines, mostly -- on Amazon or Newegg that meet all my requirements in the $1300 range. And I have found a few that almost meet them in the $7-800 range. Plus my current computer is six years old, and it is time to replace it. The question is, can I get the price down to the $1000 level for which I was hoping.

Whether I can or not, I'll take another look at Amazon. But I think it is unwieldy to be uploading a 16 gig file from my home every time I want to do work. It's not like I have a T2 line.

Warmest regards, Andrew