I am working with some of the brightest folks here at Intel, our subject matter experts and architects, such as Clay Breshears, Michael Wrinn, Bob Chesebrough and Tim Mattson (amongst others).  I will also be working closely with the indomitable Wolfgang Rosenberg, manager of the Intel Academic Community. 

My job is to reach out to educators and researchers around the world, to connect them with Intel experts and to help foster development of a curriculum to educate the next generation of programmers and engineers on the newest compute platforms.

Hopefully, this blog will go a long way to opening up channels of communication 

We have a number of events and initiatives planned for this year. 

We have already started our monthly Academic Community Curriculum Webinar Series.  During these webinars, we discuss the newest curriculum topics.  It is a great way to speak directly with our course architects.  I moderate the series and I very much look forward to speaking with you there soon.

The next in the series is on May 15 on multi-core design patterns.  Please Register below.

 Register or view past event here.

------------------------------------------

We are creating quite a few short videos supporting our academic efforts.

 I'm in the process now of filming a series on threading topics with an emphasis on game development and visual computing.  So far only the first title on Optimizing for DirectX is posted, but the rest will be available soon.

Is this type of content useful?  Are there better ways to scale out our knowledge and build conversation?  I'd like to hear that from you.

 I've asked around internally as to how folks like to consume information.  As you might imagine, there were a wide-range of responses.  Tim Mattson just rolled his eyes when I started to talk about videos and webinars.  While he is a great presenter, his own preference is to just download the PowerPoint or code and have done with it.

Others, myself included, prefer a richer content set.  For me, nothing beats the immediacy of a live event.  That is one reason we have our monthly webinars.  I am also quite interested in convening smaller conversations, perhaps using something like Communicator or Live Meeting, to discuss specific topics or curriculum ideas.  Let me know by responding to this blog.

 ---------------------------------------------

Finally, I've become very interested in different forms of new media.  I'm often available on Twitter -find me as @psteinb.

I am the owner of the Intel Software Second Life Island. 

IM me on Second Life as Peretz Stine.

Check out our launch video.

Over the last year, we ran an event series on our Second Life island dedicated to engaging engineers and professionals around the world in conversation on this unique environment.  That program, sadly, is ended, but you can still view much of it here:

Are you interested in meeting on Second Life or other virtual worlds?  It can be arranged.

 Well that's enough for now -you have you orders -tell me how best to foster dialogue.  I'll be working as hard as I can, but you are the whole point.  Let's start the conversation.

Dave Sekowski: We recently announcement with Sun Microsystems that we have made TBB available on Sun Solaris* using Sun Studio* compilers. What exactly went into the patches we made to TBB?
Chris Huson: One set of changes was the addition of using statements in the test system, to accommodate slight differences in the header files; most of these were incorporated as-is after some discussion. Sun also disabled the "warning is error" switch in the build. We use it to be pedantic about the code, but the Sun compiler found different things to warn about. A change disabling the switch for Sun was put into mainline. The other major change was to add SunOS to the preprocessor statements which were for Linux.

Dave Sekowski: In porting TBB to Sun Solaris with Sun Studio what needed to be changed and why?
Chris Huson: The change needing review on the Sun side was support for the stricter Sun support for standard library functions. Some of these functions are in the global namespace on some platforms, and in the std:: namespace on others (including Sun). Vladimir Polin incorporated Sun's modifications in a way that also supported older platforms, and Sun reviewed and approved those changes.

Dave Sekowski: Can you tell me a little about building it with Sun Studio compilers and working it into our regular build, test and release flow?
Chris Huson: The process was virtually identical to the Linux build, especially after we started using the Sun Studio Express compiler. The system is now being incorporated into our nightly build and test system, with no major problems so far.

Dave Sekowski: Do you have any additional comments on how it went?
Chris Huson: The changes were small, and incorporating them into the current version of TBB was pretty painless.

Dave Sekowski: Finally, what did you think about the collaboration with Sun to make this port happen?
Chris Huson: The Sun developers did a great job in porting TBB to their platform. Because they adhered to the spirit of the design of TBB, incorporating those changes was an easy job.

From OpenMP to TBB - some general tips

Basically, OpenMP applications contain pragma directives to specify parallelism. They are required to parallelize loops, provide locks and protect data. To port to TBB, we needed to match OpenMP pragmas to TBB classes and templates. Let's consider this very simple loop from 320.equake test:

#pragma omp parallel for private(i)
    for (i = 0; i < nodes; i++) {
      w2[j][i] = 0;
    }
  }

OpenMP executes the loop in parallel; each thread has its own instance of variable "i". The analogous TBB code is shown below - I encapsulated the loop body in a function object that is passed to the template function tbb::parallel_for. The "i" variable is used as the loop variable in the internal loop in operator().

class SVMPLoop1Class
{
    int m_j;
public:
    SVMPLoop1Class (int j):m_j(j) {};
    void operator () ( const tbb::blocked_range<int>& range) const{
        for ( int i = range.begin(); i != range.end(); ++i )
            w2[m_j][i] = 0;
    }
};

SVMPLoop1Class svmp1loop(j);
tbb::parallel_for(tbb::blocked_range<int>(0, nodes), svmp1loop, tbb::auto_partitioner());

TBB adds one more class to the code so it looks more complicated than the original code.  However the tbb::parallel_for call itself looks similar to the original loop. Note that I used tbb::auto_partitioner here to allow TBB to choose the grain size automatically.
The scheme here is common for  #pragma omp parallel for directives. If you have several variables in private directive, just move them to local variables inside the loop function. Shared variables can be copied into data members, like j variable in the example above. The SPEC applications excessively use global variables; I found replacing them by local ones works fine in many cases.
Explicit OpenMP locks can be replaced by TBB mutexes, for example tbb::spin_mutex. Like in this example from 332.ammp:

OpenMP code:

      omp_set_lock(&(a1->lock));
      ...
      omp_unset_lock(&(a1->lock));

TBB code:

{
      // scoped_lock acquires lock on creation
      // and releases it automatically after leaving the scope
      tbb::spin_mutex::scoped_lock lock (a1->spin_mutex);
      ...
}

This construction is quite simple too.

Resolving thread ID dependency - tbb::parallel_reduce instead of tbb::parallel_for.

 While I was able to replace most omp parallel for directives using the approach above, I found the loop in 320.quake that finds the hypocenter and epicenter to be more complicated. It separates the data between threads, finds out the minimum value for each thread and after that finds the global minimum. The thread ID received using omp_get_thread_num() is used to protect the data from races. Using thread IDs doesn't fit well into the higher-level task-based programming model used by TBB. I found tbb::parallel_reduce suits here. An algorithm that finds a global minimum from per-thread minima is a good example of a reduction. Here is the example from 320.quake benchmark (I skipped some code just to simplify things):

OpenMP code:

#pragma omp parallel private(my_cpu_id,d1,d2,c0)
{
   my_cpu_id=omp_get_thread_num();// thread ID
   bigdist1[my_cpu_id]=1000000.0; // minimum per thread
   temp1[my_cpu_id]=-1; // node for which the minimum was found
#pragma omp for
     for (i = 0; i < ARCHnodes; i++) {
        d1 = distance(...);
        if (d1 < bigdist1[my_cpu_id]) {
           bigdist1[my_cpu_id] = d1;
           temp1[my_cpu_id] = i;
        }
    }
}

// finding out the global minimum
numthreads = omp_get_max_threads();
d1=bigdist1[0];
Src.sourcenode=temp1[0];
for (i=0;i<numthreads;i++) // minimum per thread
{
   if (bigdist1[i] < d1) {
      d1=bigdist1[i];
      Src.sourcenode = temp1[i];
   }
}

TBB code:

struct SearchEpicenterAndHypocenter
{
    double m_bigdist1;
    struct source *m_Src;
    int m_temp1;

    SearchEpicenterAndHypocenter(struct source * Src): m_bigdist1(1000000.0), m_temp1(-1), m_Src (Src) {}

    SearchEpicenterAndHypocenter(SearchEpicenterAndHypocenter &search, tbb::split)
    {
        m_Src = search.m_Src;
        m_bigdist1 =1000000.0;
        m_temp1 = -1;
    }

    void operator () ( const tbb::blocked_range<int>& range)
    {
        for (int i = range.begin(); i != range.end(); ++i)
        {
            // The function finds out the local minimum
            SearchCycle (i, m_temp1, m_bigdist1, m_Src);
        }
    }

    void join (SearchEpicenterAndHypocenter &search)
    {
        if (search.m_bigdist1 < m_bigdist1) {
            m_bigdist1 = search.m_bigdist1;
            m_temp1 = search.m_temp1;
        }
    }
};

SearchEpicenterAndHypocenter searchcenters (&Src);
tbb::parallel_reduce(tbb::blocked_range<int>(0, ARCHnodes), searchcenters, tbb::auto_partitioner());
Src.sourcenode = searchcenters.m_temp1;

As usual, the TBB version creates an additional function object that is passed to the template function  tbb::parallel_reduce. Due to the ability to use any user-defined code for reduction, parallel_reduce does not need the thread ID based trick used with OpenMP. So the thread ID dependency problem was naturally resolved.

Resolving thread ID dependency - placing a thread ID dependent buffer inside the tbb::parallel_for and tbb::parallel_reduce

The above approach won't resolve all thread ID dependencies. I found it doesn't work when I ported other SPEC benchmarks: 330.art and 332.ammp. Parallel loops in these benchmarks use some global buffers , which are protected from data races using thread IDs. Here is an example of such a buffer from the 330.art test:

f1_neuron **f1_layer; // The buffer protected by thread ID
int numthreads = omp_get_max_threads();
f1_layer = (f1_neuron **)malloc(numthreads * sizeof (f1_neuron*));

#pragma omp parallel for private (i)
   for (i=0;i<numthreads;i++)
       f1_layer[i] = (f1_neuron*) malloc(numf1s * sizeof(f1_neuron));

// After that each thread works with its own part of f1_layer buffer.
o = omp_get_thread_num();

#pragma omp for private (k,m,n, gPassFlag) schedule(dynamic)
    for (ij = 0; ij < ijmx; ij++)
    { 
       j = ((ij/inum) * gStride) + gStartY;
       i = ((ij%inum) * gStride) +gStartX;
       k=0;
       for (m=j;m<(gLheight+j);m++)
         for (n=i;n<(gLwidth+i);n++)
           f1_layer[o][k++].I[0] = cimage[m][n];
       ...
    }

This approach is very simple but tricky, and doesn't work with TBB because we have no access to thread IDs. So I had to implement another approach - such buffers can be moved inside the parallel_for and parallel_reduce function objects as members. Each task will operate with the buffer from its copy of the body object, and so thread safety won't be violated. Here is an implementation example from 332.ammp benchmark (I skipped some code to shorten the example):

//this class is for reduce version of the main loop
class MMFVUpdateLoopReduceClass
{
  ATOM **m_atomall; //This buffer was threadID dependent in OpenMP code
  public:
  MMFVUpdateLoopReduceClass(...)
    {
        //Allocating memory for the buffer in constructor
        m_atomall = (ATOM**) malloc( m_natoms * sizeof(ATOM *) );
        ...
    }

    //This method is emtpy, it doesn't do the exact reduction
    void join (MMFVUpdateLoopReduceClass &reduceloop) {}

    // Splitting constructor
    MMFVUpdateLoopReduceClass(MMFVUpdateLoopReduceClass &reduceloop, tbb::split)
    {
        //The buffer is allocated in splitting constructor too
        m_atomall = (ATOM**) malloc( m_natoms * sizeof(ATOM *) );
    }

    void operator () ( const tbb::blocked_range<int>& range) const{
        for ( int ii = range.begin(); ii != range.end(); ++ii )
        {
            // do some work using m_atomall
        }
    }

    ~MMFVUpdateLoopReduceClass()
    {
        free(m_atomall);
    }
};

// The parallel loop
  MMFVUpdateLoopReduceClass loop(...);
  tbb::parallel_reduce(tbb::blocked_range<int>(0, jj), loop, tbb::auto_partitioner());

You probably noticed that tbb::parallel_reduce doesn't do any reduction here. Why not just use parallel_for? The reason is due to a performance difference. Note that we placed memory allocation in the constructor. It is heavy operation in terms of performance, so if the constructor is called frequently, you get a performance degradation. I found that TBB does fewer constructor calls in tbb::parallel_reduce compared to tbb::parallel_for, so parallel_reduce with an empty "join" method is more efficient here.

I was able to get a slight performance improvement with TBB over OpenMP - even though that was not the purpose of this exercise.  Finally, I hope my experience will help you to add TBB support to your applications :-).

These are tough problems and I don’t have answers for them yet.  But I have a few ideas and I hope in the next few posts to share them and get community feedback that might inspire some solutions.  So bear with me as I stumble about and maybe we can all learn something.

Anyone who has had much exposure to presentations about Intel TBB has probably heard in one form or another the motto, “process locally, steal globally.”  The Cilk philosophy embodied in the TBB task scheduler is to constrain active threads to their own local region in memory to exploit any memory that may already reside in the caches of the processing element running that thread; meanwhile, idle threads should try to steal work from memory regions as yet untouched by the active thread(s) to avoid interrupting those running threads.

When one of my customers presented an example using a nested pair of parallel_for statements, I realized that all my knowledge on subject of scheduling was theoretical: I hadn’t dug down into the guts of this code.  Now might be the time. 

First, here’s a sketch of the test code.  You’ll note an outer and inner loop with a lock on the outer and some spinning work on the inner.  This is intended to represent the structure of a program operating on a set of objects that block because of some postulated resource contention, and the inner loop represents the work done to compute that shared resource.  You’ll see that the lock is currently commented out.  When running this code on an 8-core machine, it will sometimes lock up.  Some ideas have been bounced around about why, but I’m curious whether I can demonstrate the workings of the problem rather than just speculating about it.  There’s also explicit references to the auto_partitioner, but with the advent of the affinity_partitioner, use of the auto_partitioner has been deprecated.  Still, we’ll start with this one and then see if we can tell the difference when looking under the hood.

So what’s going on here?  As is my wont, I turned to Intel® Thread Profiler for a first look at the processing:

Hrumph!  I can see eight threads cranking at the work, which is good because I’m running on an 8-core processor.  32% of the run time is spent at concurrency level 8.  Most of the time is spent in that serial tail that must represent startup processing, but the whole run is under 0.2 seconds so I suspect that’s a fixed overhead that will be negligible compared to the overall time of real work.  But that’s about all.  Zooming in on the high concurrency zone:

Where the work is done, the test is keeping 8 threads busy 94% of the time. But still there’s no clue about how the task scheduler is dividing the work, though it looks like it gets interesting towards the end of the run. Guess I’ll have to revert to an old standard, inserting print statements to expose what is happening in the code:

Unfortunately, while I can print out the loop bounds that tell me where I am in the nested executions of the array, I can’t print which thread is handling which range.  The TBB task object that might contain that information lies outside the scope of the outer_loop class.  Stuck.

But not for long.  In the evolving library that is Intel Threading Building Blocks, there is a new feature, available in at least the latest open source release (upgraded to Stable as tbb20_20080408oss), called task_scheduler_observer.  I’ve implemented an observer that lets me identify which thread executes which range.  In my next post I will describe it and start exploring the behavior of the scheduler.  Stay tuned!

Share your experiences and feedback with us. Blog and tell us what you think.

Be it your opinion on the need for threading applications, or the benefits of parallel programming or your feedback on Intel® AVX (Intel® Advanced Vector Extensions) or the contests on Intel® Software Network… Are you ready for the Threading Challenge? Or is the Game Demo Contest your calling?

Blog and tell us all about it now!

At what project level are you currently applying TBB?

81 people participated in the poll, making the following selections:

• 75.3% (61 votes) - Just getting started (learning about TBB)
• 6.2% (5 votes) - Developing new software that applies TBB
• 6.2% (5 votes) - Modifying existing software to use TBB
• 4.9% (4 votes) - Designing new software that will apply TBB
• 4.9% (4 votes) - Working at multiple levels on multiple TBB projects
• 2.5% (2 votes) - Maintaining software that uses TBB

While it's clear that most people who participated are investigating TBB, it's also interesting to note the breakout for the developers who are actively using (or planning to use) TBB in actual projects. Over 11% of respondants reported that they are designing or developing new software that applies Threading Building Blocks. Almost 9% of respondants are either modifying existing software to apply TBB or maintaining software that already applies Threading Building Blocks. And another 5% of the responding developers are working at multiple levels on multiple TBB projects.

The poll results show that, eight months after TBB was launched as an open source project, there is a group of developers who are deploying TBB in new and existing applications. Meanwhile, there is a much larger group of developers who are interested in the Threading Building Blocks technology. When these projects that are applying TBB are completed to a degree that they can be made publicly available, they will provide a template that can be studied by other developers, as they design and develop their own projects that apply TBB for multithreading and scaling. It will be interesting to repeat this poll after some of these projects are completed and made public.

New poll: your OS for TBB development

The April Threading Building Blocks poll has been posted. This poll asks:

On what Operating System(s) do you develop your TBB applications?

The response options are:

• FreeBSD
• Linux
• MacOS
• Microsoft Windows
• Unix
• Unix on Windows (Cygwin, MinGW, UWIN)
• Other
• More than one OS

Even if you're not working on an actual TBB-related project yet, feel free to vote. Just select the operating system you're using for experimenting with TBB as you learn about it.

To vote, go to the TBB home page and scroll down a bit; you'll see the poll on the right side of the page.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

• Open Source Operating Systems that Offer TBB Packages
• Open Source Projects that Apply TBB

I consider Wikipedia one of the greatest new informational resources the Web has produced. It provides information to those who want to find out about something, but it also provides an open venue for anyone who wants to contribute information to the world in their area of expertise.

I encourage the Threading Building Blocks community to extend the TBB Wikipedia page by supplementing the information about TBB itself, or adding new information about your own TBB-related projects, and about Linux and other operating systems that are making TBB available through their package manager systems.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Commercial TBB supported Linux distros

The Commercial TBB site includes a System Requirements section that identifies several Linux distributions on which TBB has been tested. These distributions are:

• Red Hat Enterprise Linux* 3, 4, or 5
• Red Hat Fedora Core* 4, 5, or 6 (not supported on Itanium-based systems)
• Asianux* 2.0
• Red Flag DC Server* 5.0
• Haansoft Linux* Server 2006
• Miracle Linux v4.0
• SuSE Linux Enterprise Server* 9 or 10
• SGI Propack* 4.0 (supported on Itanium-based systems only)
• SGI Propack 5.0 (not with IA-32 architecture processors)
• Mandriva/Mandrake Linux 10.1.06 (not with Intel Itanium processors)
  Turbolinux GreatTurbo* Enterprise Server 10 SP1 (not with Intel Itanium processors)

This list tells us that commercial TBB has been installed and tested on these distributions, but it doesn't tell us which of these distributions offers or plans to offer TBB open source packages. In July 2007 (when TBB Open Source was announced), people associated with several of these distributions commented on TBB open source (see the "TBB's status with Operating System Vendors (OSVs)" TBB forum post), suggesting that they planned to make TBB "more easily assessible" on their system (Novell/OpenSUSE), or that they are bundling TBB with their distribution (Asianux and Turbolinux). So, I did some searching to see if I could find out the current status of TBB and these distributions.

Red Hat Enterprise Linux: no open source TBB package information found...

Fedora 8: As I reported in my last post, the tbb20_20070927oss stable release has apparently been packaged into Fedora 8 and is available as an RPM file on the Fedora Download Server.

Asianux, Red Flag, Haansoft Linux, Miracle Linux: the Asianux About page suggests that these are all essentially the same Linux distribution, just distributed by different vendors. I wasn't able to find any references to TBB in the Asianux bug list. I created an account so I could access the technical support site, but there is a new account approval procedure that blocked my access. Searching on the web, I found a description of a purchasable Value Pack of Intel software for Haansoft Linux. So, perhaps the way you get TBB onto Asianux and the related distributions is through purchasing and installing a commercial set of Intel software. I found no information about open source TBB packages for Asianux.

SuSE: no open source TBB package information found...

SGI Propack: this is a purchasable software package designed for SGI Altix computers.

Mandriva/Mandrake: For Mandriva, I searched the forums and mailing list archives, and did general web searches; no open source TBB package information turned up.

TBB in FreeBSD

FreeBSD has embraced open source TBB since soon after its inception. The latest update to the FreeBSD TBB port was made on February 7, 2008. I believe this means that the tbb20_20080207oss development release of TBB is available in FreeBSD. That release is no longer listed on the TBB Downloads site. Because of this, I'm not sure what will happen if you try to install TBB using the FreeBSD package manager. It depends on if there is a dependency in the FreeBSD package on a link to the original tbb20_20080207oss download, and if that the download link still exists (even though we can't navigate to it using a browser any longer).

Conclusion

Threading Building Blocks is used by many developers on Linux and FreeBSD systems. Packaging of TBB is actively under way for Debian Linux, Ubuntu Linux, and the Fedora Project; TBB is also available through the FreeBSD package manager.

It's likely that work is being done to package Threading Building Blocks for other Linux distributions; but, at the moment, there does not appear to be publicly available information about these efforts.

Fortunately, it's relatively easy to install TBB onto any Linux system. You install the source and build it; or you can download one of the prebuilt Linux binary downloads that are delivered with the TBB Commercial Aligned releases. Once you set your environment variables (see the tbbvars.* files), you'll be set to go!

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Threading Building Blocks can be installed on any Linux system by downloading the source and directly building TBB. I did it this way for my Gentoo system (see my "Threading Building Blocks, AMD Athlon 64 X2, and Gentoo Linux" post). But in the eight months since TBB became an open source project, a significant amount of work has also been done by developers in packaging TBB for specific Linux distributions, enabling installation of TBB using each distribution's package manager.

In this series of posts I'll provide an overview of the status of TBB on various Linux distributions. As far as I know, TBB will be included in several upcoming Linux releases, but is not yet available as a package in current stable Linux releases. Intel thoroughly tested TBB on several Linux distributions before TBB became an open source project, so these distributions would seem to be excellent candidates for packaging open source TBB. I'll investigate all of these as well, over the course of these blogs.

Ubuntu and Debian

As I reported in January, Threading Building Blocks is being packaged into Ubuntu Hardy Heron, which is expected to be released in April (the beta can be downloaded now). This came about through a request submitted to the Ubuntu bug list by Sadiq Jaffer, a regular visitor to the #tbb IRC channel. Though the period for syncing new packages from Debian had expired, the Ubuntu team made an exception for TBB. So, TBB will be available to Ubuntu Hardy Heron users using the Ubuntu package manager.

TBB being in Ubuntu Hardy Heron was of course possible only because of the efforts of people at Athena Capital Research who packaged TBB into Debian Sid. For Debian users who are working with the current stable Etch version, Athena's Roberto Sanchez has created a non-official set of Etch TBB packages. I installed TBB onto my Debian Etch machine using Debian's APT package manager and Roberto's Etch TBB packages, and described that process in my "Threading Building Blocks Debian Linux Packages" post.

Fedora

The status of Threading Building Blocks on Fedora is a bit confusing. The Fedora Package Database site indicates that TBB has been approved for inclusion in Fedora 8, but the package has not yet been implemented. The FedoraMD.org blog documented TBB's approval for Fedora 8 on February 13, 2008.

But further searching located a downloadable tbb-2.0-4.20070927.fc8.src.rpm file on the Fedora Download Server. This package is based on the tbb20_20070927oss release, which is now classified as a stable release. You can see an overview of the features that were new in this release in my "Threading Building Blocks Open Source Release Versions Matrix" post. The 20070927 release also includes changes that were initially released in the earlier 20070815 and 20070719 TBB releases.

Conclusion

Debian, Ubuntu, and Fedora are the three of the most widely used Linux distributions. In all three cases, TBB is (or will soon be) available using the respective distribution package managers.

In my next post I'll look into TBB availability on some other popular Linux distributions, and I'll also look at TBB on FreeBSD as well.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

"But wait," someone could complain, "your example so far is unfair because you've stacked the deck. The truth is that, when you find a superlinear speedup, what you've really found is an inefficiency in the sequential algorithm."

When parallelism is simpler

I had the good fortune to be able to speak with Herb Sutter today about these issues. We found ourselves in agreement in most areas. We agree, for example, that memory issues can make an enormous performance difference when you compare an algorithm run in parallel with a serial algorithm that retraces the data processing of the parallel algorithm in an element-by-element manner.

But even when I asked Herb to ignore memory issues (which is where I believe the greatest advantages will lie for some parallel algorithms), he remained quite adamant that superlinearity is possible. How so? Well, Herb mentioned things like algorithm complexity, and the known or unknown characteristics of the data. As I pondered this, Herb asked that I re-look at the last section of his "Going Superlinear" article. Here we find (among other things) a comparison between a simple parallel algorithm and the serial algorithm that retraces the parallel algorithm's data processing element-by-element. The serial algorithm, in this case, is:

more complex. It has to do more bookkeeping than a simple linear traversal. This additional work can be a small additional source of performance overhead ...

This is indeed correct. The "bread-slicing" example in my "Superlinearity Is Impossible" post would involve slightly more complicated programming than the simple incremented loop that could be easily parallelized using something like TBB.

Here's Herb's conclusion about the serial algorithm that mimics the parallel processing and the parallel algorithm:

when we're comparing the proposed algorithm with simple parallel search, we're not really comparing apples with apples. We are comparing:

• A complex sequential algorithm that has been designed to optimize for certain expected data distributions, and
• A simple parallel algorithm that doesn't make assumptions about distributions, works well for a wide range of distributions, and naturally takes advantage of the special ones the optimized one is trying to exploit ...

So, is superlinearity possible?

Have I been proven wrong? I don't think so. When we talk about "superlinear" speedups, I think it all comes down to a matter of definition: what do we each mean by "superlinear performance"? The comments posted to my "Superlinearity Is Impossible" post revealed that different people think about this quite differently.

My background is physics, and mathematical modeling and simulation. I'm accustomed to thinking in terms of an ideal world that doesn't really exist when I think of equations (a world full of infinitely-long wires, which can be approximated as having an infinitesimal thickness, etc.). I view Amdahl's Law as existing in this type of realm. Hence, I asked Herb to pretend memory access is instantaneous as we discussed whether or not superlinearity is possible.

In this "purist" theoretical way of looking at superlinearity and Amdahl's Law, I still consider superlinearity impossible. I view Amdahl's Law as akin to the "law" of gravity. Just because I see an airplane go up into the air doesn't mean the law of gravity has been broken. Other factors were involved.

The fact that duplication of the parallel algorithm's element-by-element processing can require a more complex serial algorithm means you really aren't comparing apples with apples, as Herb says. Nor can you; this situation seems inescapable.

Indeed, I ran several actual tests tonight on my quad-core system, writing some very simple programs where I eliminated advantages due to memory cache as much as possible. I found many cases where the more complex serial algorithm was slower than the simple algorithm that would have been parallelized; but surprisingly, sometimes my compiler's optimization actually made the more complex serial algorithm run faster than it's simpler cousin! If I turned the optimizer off, then the more complex algorithm always took more time to complete.

So, this is a complex realm. Looking at it all through a practical (i.e., non-purist, not rigidly theoretical) lens, I certainly agree that there are software engineering techniques which, when applied to software that is intended for a parallel environment, will produce programs that will complete their tasks in an amount of time that implies superlinear performance when compared with an equivalent serial algorithm. I do not, however, believe that this means that the computational limits represented by Amdahl's Law are being violated.

The implications for programming education

All this complexity implies that parallel programming might be best considered as being its own unique realm when it comes to performance optimization techniques. Even discounting the standard problems involving thread-safety, race conditions, deadlocks, etc., optimized parallel programming is a very different creature from optimized serial programming. To consider parallel programming as being a "next natural stage" that is somehow in sync with the serial programming most developers already understand is quite possibly a mistake, in varying ways. Software performance optimization within the parallel realm can be very different, as Herb Sutter and other are showing.

So, once again: is superlinear performance through parallelization possible? Can Amdahl's Law be broken? You tell me...

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB