Disk is the New RAM?

By Clay Breshears (Intel) (80 posts) on April 18, 2008 at 2:19 pm

In a past post, I've questioned how we are expecting to "feed" the ever-growing number of cores with data.  In the April 2008 issue of Communications of the ACM, Daniel Kunkle and Gene Copperman (yes, those Northeastern University researchers that have recently been computing Rubik's Cube solutions) may have come up with part of the solution. The article, "Solving Rubik's Cube: Disk Is the New RAM," explains how their realization that 50 disks can provide an aggregate bandwidth of 5GB/s, almost the bandwidth of commodity RAM, and that this could allow a method to hold the massive data sets needed for their computations with the ability to quickly access that data within the nodes of the cluster. 

But, can you really treat 50 disks as if it were RAM?  Kunkle and Coopperman point out that the disks were distributed across their cluster, so access should tend toward accessing the local disk rather than disks across the network. Also, because of their physical restrictions and associated latencies, disks aren't really random-access devices. These two points need to be taken into account when developing the parallel algorithms that will attempt to make the most efficient use of multiple disks as if they were a very large RAM.

It's a pretty keen idea and published none too soon if the April 2008 issue of IEEE Computer is any indication. The focus of that publication is "Data-Intensive Computing."

Using multiple disks on a cluster, as Kunkle and Coopperman did, is all well and good. I'm still not sure how or if this idea might be able to translate down to a single manycore chip. The cluster's multiple nodes all have separate connections for disks; a single chip only has so many paths onto it. We might be able to fill up memory from multiple disks, but we are still going to need some way to push that data into the chip for all the cores to compute on.

Categories: Multicore

Comments (3) Comments RSS Feed

By Michael Shadle (Intel) on April 23rd, 2008 at 1:58 am
And on the flipside, RAM is the new disk:
http://www.hyperossystems.co.uk/07042003/hardware.htm :)

By Clay Breshears (Intel) on April 23rd, 2008 at 12:22 pm
I looked at the devices being described on the above URL. I couldn't figure out how they were thinking to charge $2700 for 16GB. I mean, I picked up a 4GB thumb-drive for $45 at Fry's in Wilsonvile, OR, last time I was there. They had such drives upto 32GB and 64GB for a few hundred dollars.

And then I realized that the webpage was from back in 2003. In five years, we've seen the price slashed from $150/GB to $10/GB and the size has come down from a 5.25" disk bay to my thumb. Of course, my drive isn't configured for RAID.

By Michael Shadle (Intel) on April 23rd, 2008 at 1:20 pm
Yeah, I know. I'm sure there's some other (more updated) products out there, I found that randomly and thought it would be interesting to share.


What do you think?

Name (required)

Email (required; will not be displayed on this page)

Your URL (optional)

Comments (required)