PGAS versus MPI and what should we teach undergraduates??

By Robert Chesebrough (Intel) (3 posts) on April 22, 2008 at 8:34 am

One of the highlights of this 22nd annual IPDPS conference was the Wednesday night panelist discussion. The discussion probed the general topic of what the current parallel programming experts (eps IPDPS faculty & researchers) can teach to a new generation who will just now be cutting their teeth on MC processors and growing up in a non sequential programming landscape. The discussion was recorded for IEEE TV but I have not seen it posted yet on either the IPDPS site or the IEEE TV site. So for lack of being able to review the panelists discussion again in detail I have relied on my cryptic notes, scribbled furiously during the discussion. The panel consisted of several of the "earth movers" - those prominent professors or researchers in parallel computing - who have participated in massively parallel computing or more appropriately, shaped massively parallel computing landscape. I want to cover each of the panelists opinions about the future, but must start somewhere - so let me begin with Katherine Yelick's presentation.Kathy Yelick is NERSC Division Director, Lawrence Berkeley National Lab and EECS Department, University of California at Berkeley. She received her Bachelors (1985), Masters (1985), and PhD (1991) degrees in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology. Her research interests include parallel computing, memory hierarchy optimizations, programming languages and compilers. In this panel discussion, Kathy laid out a path whereby CS students would likely be split into two types of specialties: 1) an efficiency layer - Performance oriented parallel programmers who understand cache locality and lower level parallelism mechanics and who write abstraction layers & libraries upon which the rest of developer community stands
2) a productivity layer - who use the lower level parallel abstractions and libraries to and who focus on application domain
In Katherine's model - she would hope that ~10% of the developers who have to be deeply involved in the efficiency layer, while the majority (90%) of developers would be trained as application domain folksKatherine stated that it would be a disaster if the split between efficiency programmers and application programmers urns out to be more like 50/50% rather than the 10/90% split proposed and believes universities should begin prepping CS students along these two disciplines and hopefully - the numeric proportion will fall to the 10/90.She laid a case for tackling large many core application going forward with Parallel Global Addressable Systems (PGAS) languages - like Unified parallel C (UPC), Titanium, CAF etc as opposed to continued efforts in MPI. They case she laid out was similar or better performance with PGAS with substantially fewer lines of source code. It turns out UPC was discussed frequently in the sessions I attended and I am considering whether Intel Software College should add a module on UPC or Titanium etc for our University program - Thoughts?She said there are gaps in current training at the undergraduate level that need correction. Essentially these gaps are 1) there is currently a lack of deep understanding of performance & algorithmic complexity, 2) there is currently a lack of understanding of algorithms
3) there is a lack of sophisticated understanding of concurrency, synchronization, non determinism, load balance etc - presumably - these gaps are less of an issue for the application domain folks but are of critical importance ot efficiency programmers of the future. This is an area where many of our Intel Software college already has some material in the pipeline and its was good to get implicit validation that our plans align with the Berkeley vision.Katherine roughly laid out the Berkley approach, where they have begun teaching parallelism to more entry level programming students. They have incorporated cluster computing fundamentals into the introductory computer science curriculum at UC Berkeley. In that course, they have developed coursework and programming problems centered around Google's MapReduce. They used a language called Scheme to write and run MapReduce programs and can tackle parallel problems on a cluster using a special interface. Students can begin addressing data-parallel problems about two thirds of the way into this course. They are targeting more concurrency in their systems course and now have a capstone course for seniors who will be moving off into the efficiency programming type of endeavors. At the higher level, they will be focusing on 13 Berkley "motif" or what has been called the Berkeley 13 dwarfs. These are 13 categories of applications that have similar target kernels underlying their operation. Efficiently parallelize the kernels and you solve a whole class of applications in that share that kernel or motif.This Berkley approach is one that Intel Software College is pursuing to help seed our faculty training. We are working with a couple of the Berkley faculty teaching the Berkley motifs exploration course together with design patterns. We plan to incorporate findings and lablets and instructor materials from this course to our wiki in the June time frame.As part of those postings we will also be posting materials we have created or materials we have received on other parallel techniques: Design Patterns, MPI, functional language examples (Erlang), Software Transactional Memory, etc.So next time I'll recap a few more of the panelist discussions and how they may impact our curriculum postings.If you have examples of what YOU are teaching in YOUR undergraduate courses to teach parallelism - I'd love to hear from you - please respond to my blog or better yet - post your materials for use by many universities to our wiki!bye for now  

Categories: Academia, Events, Multicore, Software Engineering

Comments (0) Comments RSS Feed

No comments have been posted for this entry yet.


What do you think?

Name (required)

Email (required; will not be displayed on this page)

Your URL (optional)

Comments (required)