Functional Languages versus threading

By Robert Chesebrough (Intel) (3 posts) on April 22, 2008 at 8:26 am

I had responded to some questions in my other post (view from 22nd annual IPDPS - http://softwareblogs.intel.com/2008/04/15/some-views-from-the-22nd-international-parallel-amp-distributed-processing-symposium/) about functional languages.  It was suggested by Clay B that I should make a seperate post along the functional language topic - so here goes. 

I asked Dr Dennis (Prf Emeritus CS at MIT) his thoughts on what should be taught to undergrads to prepare them for a many core future. This was a lunchtable conversation and not a true curriculum planning session - but none the less - Dr Denis did not hesitate to mention functional languages as one of the first things that should be taught. I don't think he was implying that a functional language is the magic bullet or one size fits all solution to bringing new developers to the discipline.

The reasons he articulted for teaching a functional language are that:
1) it is a model for parallel prgramming that avoids the sunchroonization hazards associated with threading - data races etc.
2) version of parallel languages are already being taught in many schools already so there s a low barrier to entry
3) Examples of functional languages can already be found in industry

I would argue that there will still be a need to teach students about threading (but not with as high a priority). If you teach them about threading, you have to teach them about the associated hazards. This does not contradict that we should also teach them a functional language.

On other fronts here at IPDPS I talked to Keshav Pingali from the University of Texas, and Vivek Sarkar from Rice University who were panelists here Wednesday night. These panelists discussed how the current parallel computation community could pass their learnings on to a new generation of develpoers just starting out in multicore. After the panel discussion, Michael Wrinn & I were able to get their take on essentially the same question I asked of Dr Dennis. They indicated that there still is some value in teaching functional programming to the new crop of developers. Keshav in particular indicated that his early exposure to functional languages has influenced his coding style in positive ways - years later. They acknowledged that some implentations of functional languages have been poor performers on sequential machines in the past and so they have not really caught on in industry (Erlang is the excetion to this statement) and also they cited some difficulties with cache locality & dealing with multidmensional arrays from a developers perspective. However, they also indicated that several groups have been adding extensions to these lanuages, I think Haskell was mentioned, to allow for improved handling of arrays and to address locaility issues.

So here are my musings after havng these discussions. Warning controvesial road ahead - use caution when entering - Here's a controversial perspective on appropriateness of functional languages as we move into many core land. Maybe the locaility issues that caused functional languages to be discounted wont be so significant in a time when we are utilizing hundreds of cores. Why? In sequential programming days and in early days of multicore - 2 cores could possibily give you a 2X speedup. But the tax we pay for poor cache locality might be hypothetically ~10X - the number thrown around here in our conversations with these disinguished professors from Texas. So any language that could gain you 2X by parallelism - but lose you 10X thru poor locality probably would die a natural death. BUT - improve cache locality fo these languages and apply to many more cores. Suppose in future we have 100 cores and that compilers for the functional languages get a second look and get needed optimizations so that we reduce cache locality issues to be a 50% penalty rather than 10X (50% penalty means we run at 1.5X speedup). Now assume that we approach 100X due to parallelism but pay 1.5X tax. Thats a net gain ~66X speedup. Compiler writers might be persuaded to optimize a functional language if there was a decent pay off. 66X might do it. If the industry had frozen in a past instant of time at only 2 cores and if compiler writers at that time thought that they might be able to reduce locality penalty to 50% would it have been a worthwhile endeavor? 2X gain/1.5X loss 1.3X net - that a 30% gain in performance - not so clear that compiler writers would be enthusiastic to do all the work required because the potential pay off would not be that large. But maybe for 66X gain thay may consider it.

Caveat - I know I am assuming no overhead here and we likley wont scale perfectly or anywhere near it at 100 cores etc. But if there are no synchronization hazards - and we reduce the locality penalties maybe these things will scale better than we imagine - does anyone have a quantitaive way to address this? Quick - can someone whip out an equation from Hannesy & Patterson?

My point is that perhaps we should keep an open mind ot the possiblity of functional languages and hope that the "economy scale" and clever compiler writers might be able to bring a solution. The payoff ultimately would be fundamentally easier approaches to parallelism.

Who knows?

bobc

Categories: Academia, Events, Multicore, Open Source, Software Engineering

Comments (0) Comments RSS Feed

No comments have been posted for this entry yet.


What do you think?

Name (required)

Email (required; will not be displayed on this page)

Your URL (optional)

Comments (required)