At what project level are you currently applying TBB?

81 people participated in the poll, making the following selections:

• 75.3% (61 votes) - Just getting started (learning about TBB)
• 6.2% (5 votes) - Developing new software that applies TBB
• 6.2% (5 votes) - Modifying existing software to use TBB
• 4.9% (4 votes) - Designing new software that will apply TBB
• 4.9% (4 votes) - Working at multiple levels on multiple TBB projects
• 2.5% (2 votes) - Maintaining software that uses TBB

While it's clear that most people who participated are investigating TBB, it's also interesting to note the breakout for the developers who are actively using (or planning to use) TBB in actual projects. Over 11% of respondants reported that they are designing or developing new software that applies Threading Building Blocks. Almost 9% of respondants are either modifying existing software to apply TBB or maintaining software that already applies Threading Building Blocks. And another 5% of the responding developers are working at multiple levels on multiple TBB projects.

The poll results show that, eight months after TBB was launched as an open source project, there is a group of developers who are deploying TBB in new and existing applications. Meanwhile, there is a much larger group of developers who are interested in the Threading Building Blocks technology. When these projects that are applying TBB are completed to a degree that they can be made publicly available, they will provide a template that can be studied by other developers, as they design and develop their own projects that apply TBB for multithreading and scaling. It will be interesting to repeat this poll after some of these projects are completed and made public.

New poll: your OS for TBB development

The April Threading Building Blocks poll has been posted. This poll asks:

On what Operating System(s) do you develop your TBB applications?

The response options are:

• FreeBSD
• Linux
• MacOS
• Microsoft Windows
• Unix
• Unix on Windows (Cygwin, MinGW, UWIN)
• Other
• More than one OS

Even if you're not working on an actual TBB-related project yet, feel free to vote. Just select the operating system you're using for experimenting with TBB as you learn about it.

To vote, go to the TBB home page and scroll down a bit; you'll see the poll on the right side of the page.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

• Open Source Operating Systems that Offer TBB Packages
• Open Source Projects that Apply TBB

I consider Wikipedia one of the greatest new informational resources the Web has produced. It provides information to those who want to find out about something, but it also provides an open venue for anyone who wants to contribute information to the world in their area of expertise.

I encourage the Threading Building Blocks community to extend the TBB Wikipedia page by supplementing the information about TBB itself, or adding new information about your own TBB-related projects, and about Linux and other operating systems that are making TBB available through their package manager systems.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Commercial TBB supported Linux distros

The Commercial TBB site includes a System Requirements section that identifies several Linux distributions on which TBB has been tested. These distributions are:

• Red Hat Enterprise Linux* 3, 4, or 5
• Red Hat Fedora Core* 4, 5, or 6 (not supported on Itanium-based systems)
• Asianux* 2.0
• Red Flag DC Server* 5.0
• Haansoft Linux* Server 2006
• Miracle Linux v4.0
• SuSE Linux Enterprise Server* 9 or 10
• SGI Propack* 4.0 (supported on Itanium-based systems only)
• SGI Propack 5.0 (not with IA-32 architecture processors)
• Mandriva/Mandrake Linux 10.1.06 (not with Intel Itanium processors)
  Turbolinux GreatTurbo* Enterprise Server 10 SP1 (not with Intel Itanium processors)

This list tells us that commercial TBB has been installed and tested on these distributions, but it doesn't tell us which of these distributions offers or plans to offer TBB open source packages. In July 2007 (when TBB Open Source was announced), people associated with several of these distributions commented on TBB open source (see the "TBB's status with Operating System Vendors (OSVs)" TBB forum post), suggesting that they planned to make TBB "more easily assessible" on their system (Novell/OpenSUSE), or that they are bundling TBB with their distribution (Asianux and Turbolinux). So, I did some searching to see if I could find out the current status of TBB and these distributions.

Red Hat Enterprise Linux: no open source TBB package information found...

Fedora 8: As I reported in my last post, the tbb20_20070927oss stable release has apparently been packaged into Fedora 8 and is available as an RPM file on the Fedora Download Server.

Asianux, Red Flag, Haansoft Linux, Miracle Linux: the Asianux About page suggests that these are all essentially the same Linux distribution, just distributed by different vendors. I wasn't able to find any references to TBB in the Asianux bug list. I created an account so I could access the technical support site, but there is a new account approval procedure that blocked my access. Searching on the web, I found a description of a purchasable Value Pack of Intel software for Haansoft Linux. So, perhaps the way you get TBB onto Asianux and the related distributions is through purchasing and installing a commercial set of Intel software. I found no information about open source TBB packages for Asianux.

SuSE: no open source TBB package information found...

SGI Propack: this is a purchasable software package designed for SGI Altix computers.

Mandriva/Mandrake: For Mandriva, I searched the forums and mailing list archives, and did general web searches; no open source TBB package information turned up.

TBB in FreeBSD

FreeBSD has embraced open source TBB since soon after its inception. The latest update to the FreeBSD TBB port was made on February 7, 2008. I believe this means that the tbb20_20080207oss development release of TBB is available in FreeBSD. That release is no longer listed on the TBB Downloads site. Because of this, I'm not sure what will happen if you try to install TBB using the FreeBSD package manager. It depends on if there is a dependency in the FreeBSD package on a link to the original tbb20_20080207oss download, and if that the download link still exists (even though we can't navigate to it using a browser any longer).

Conclusion

Threading Building Blocks is used by many developers on Linux and FreeBSD systems. Packaging of TBB is actively under way for Debian Linux, Ubuntu Linux, and the Fedora Project; TBB is also available through the FreeBSD package manager.

It's likely that work is being done to package Threading Building Blocks for other Linux distributions; but, at the moment, there does not appear to be publicly available information about these efforts.

Fortunately, it's relatively easy to install TBB onto any Linux system. You install the source and build it; or you can download one of the prebuilt Linux binary downloads that are delivered with the TBB Commercial Aligned releases. Once you set your environment variables (see the tbbvars.* files), you'll be set to go!

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Threading Building Blocks can be installed on any Linux system by downloading the source and directly building TBB. I did it this way for my Gentoo system (see my "Threading Building Blocks, AMD Athlon 64 X2, and Gentoo Linux" post). But in the eight months since TBB became an open source project, a significant amount of work has also been done by developers in packaging TBB for specific Linux distributions, enabling installation of TBB using each distribution's package manager.

In this series of posts I'll provide an overview of the status of TBB on various Linux distributions. As far as I know, TBB will be included in several upcoming Linux releases, but is not yet available as a package in current stable Linux releases. Intel thoroughly tested TBB on several Linux distributions before TBB became an open source project, so these distributions would seem to be excellent candidates for packaging open source TBB. I'll investigate all of these as well, over the course of these blogs.

Ubuntu and Debian

As I reported in January, Threading Building Blocks is being packaged into Ubuntu Hardy Heron, which is expected to be released in April (the beta can be downloaded now). This came about through a request submitted to the Ubuntu bug list by Sadiq Jaffer, a regular visitor to the #tbb IRC channel. Though the period for syncing new packages from Debian had expired, the Ubuntu team made an exception for TBB. So, TBB will be available to Ubuntu Hardy Heron users using the Ubuntu package manager.

TBB being in Ubuntu Hardy Heron was of course possible only because of the efforts of people at Athena Capital Research who packaged TBB into Debian Sid. For Debian users who are working with the current stable Etch version, Athena's Roberto Sanchez has created a non-official set of Etch TBB packages. I installed TBB onto my Debian Etch machine using Debian's APT package manager and Roberto's Etch TBB packages, and described that process in my "Threading Building Blocks Debian Linux Packages" post.

Fedora

The status of Threading Building Blocks on Fedora is a bit confusing. The Fedora Package Database site indicates that TBB has been approved for inclusion in Fedora 8, but the package has not yet been implemented. The FedoraMD.org blog documented TBB's approval for Fedora 8 on February 13, 2008.

But further searching located a downloadable tbb-2.0-4.20070927.fc8.src.rpm file on the Fedora Download Server. This package is based on the tbb20_20070927oss release, which is now classified as a stable release. You can see an overview of the features that were new in this release in my "Threading Building Blocks Open Source Release Versions Matrix" post. The 20070927 release also includes changes that were initially released in the earlier 20070815 and 20070719 TBB releases.

Conclusion

Debian, Ubuntu, and Fedora are the three of the most widely used Linux distributions. In all three cases, TBB is (or will soon be) available using the respective distribution package managers.

In my next post I'll look into TBB availability on some other popular Linux distributions, and I'll also look at TBB on FreeBSD as well.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

"But wait," someone could complain, "your example so far is unfair because you've stacked the deck. The truth is that, when you find a superlinear speedup, what you've really found is an inefficiency in the sequential algorithm."

When parallelism is simpler

I had the good fortune to be able to speak with Herb Sutter today about these issues. We found ourselves in agreement in most areas. We agree, for example, that memory issues can make an enormous performance difference when you compare an algorithm run in parallel with a serial algorithm that retraces the data processing of the parallel algorithm in an element-by-element manner.

But even when I asked Herb to ignore memory issues (which is where I believe the greatest advantages will lie for some parallel algorithms), he remained quite adamant that superlinearity is possible. How so? Well, Herb mentioned things like algorithm complexity, and the known or unknown characteristics of the data. As I pondered this, Herb asked that I re-look at the last section of his "Going Superlinear" article. Here we find (among other things) a comparison between a simple parallel algorithm and the serial algorithm that retraces the parallel algorithm's data processing element-by-element. The serial algorithm, in this case, is:

more complex. It has to do more bookkeeping than a simple linear traversal. This additional work can be a small additional source of performance overhead ...

This is indeed correct. The "bread-slicing" example in my "Superlinearity Is Impossible" post would involve slightly more complicated programming than the simple incremented loop that could be easily parallelized using something like TBB.

Here's Herb's conclusion about the serial algorithm that mimics the parallel processing and the parallel algorithm:

when we're comparing the proposed algorithm with simple parallel search, we're not really comparing apples with apples. We are comparing:

• A complex sequential algorithm that has been designed to optimize for certain expected data distributions, and
• A simple parallel algorithm that doesn't make assumptions about distributions, works well for a wide range of distributions, and naturally takes advantage of the special ones the optimized one is trying to exploit ...

So, is superlinearity possible?

Have I been proven wrong? I don't think so. When we talk about "superlinear" speedups, I think it all comes down to a matter of definition: what do we each mean by "superlinear performance"? The comments posted to my "Superlinearity Is Impossible" post revealed that different people think about this quite differently.

My background is physics, and mathematical modeling and simulation. I'm accustomed to thinking in terms of an ideal world that doesn't really exist when I think of equations (a world full of infinitely-long wires, which can be approximated as having an infinitesimal thickness, etc.). I view Amdahl's Law as existing in this type of realm. Hence, I asked Herb to pretend memory access is instantaneous as we discussed whether or not superlinearity is possible.

In this "purist" theoretical way of looking at superlinearity and Amdahl's Law, I still consider superlinearity impossible. I view Amdahl's Law as akin to the "law" of gravity. Just because I see an airplane go up into the air doesn't mean the law of gravity has been broken. Other factors were involved.

The fact that duplication of the parallel algorithm's element-by-element processing can require a more complex serial algorithm means you really aren't comparing apples with apples, as Herb says. Nor can you; this situation seems inescapable.

Indeed, I ran several actual tests tonight on my quad-core system, writing some very simple programs where I eliminated advantages due to memory cache as much as possible. I found many cases where the more complex serial algorithm was slower than the simple algorithm that would have been parallelized; but surprisingly, sometimes my compiler's optimization actually made the more complex serial algorithm run faster than it's simpler cousin! If I turned the optimizer off, then the more complex algorithm always took more time to complete.

So, this is a complex realm. Looking at it all through a practical (i.e., non-purist, not rigidly theoretical) lens, I certainly agree that there are software engineering techniques which, when applied to software that is intended for a parallel environment, will produce programs that will complete their tasks in an amount of time that implies superlinear performance when compared with an equivalent serial algorithm. I do not, however, believe that this means that the computational limits represented by Amdahl's Law are being violated.

The implications for programming education

All this complexity implies that parallel programming might be best considered as being its own unique realm when it comes to performance optimization techniques. Even discounting the standard problems involving thread-safety, race conditions, deadlocks, etc., optimized parallel programming is a very different creature from optimized serial programming. To consider parallel programming as being a "next natural stage" that is somehow in sync with the serial programming most developers already understand is quite possibly a mistake, in varying ways. Software performance optimization within the parallel realm can be very different, as Herb Sutter and other are showing.

So, once again: is superlinear performance through parallelization possible? Can Amdahl's Law be broken? You tell me...

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

Intel and rival Advanced Micro Devices Inc. now have four-processor offerings, but companies predict the advent of "many-core" chips with dozens or even hundreds of microprocessors.

and about the problems such computers present to software developers:

"Everybody is madly racing toward multicore technology, and they don't have a clue about how to program it," said William Dally, a professor of computer science at Stanford University.

Of course, this is precisely the problem that Threading Building Blocks is designed to address. Having worked on the TBB open source project since the project was launched at OSCON last July, TBB continues to look like an excellent solution to the multicore problem to me. I wish TBB had been around in 1993 when I first started developing mathematical modeling and simulation software for operation on multiprocessor Sun machines!

But, who needs multithreaded programs???

Over and over, I've heard the question: "who really needs multithreaded programs on their home (or office) desktop computer?" This question reminds me so much of questions from the past, like "Who really needs more than 640 KBytes of memory on a PC? How could you ever use so much memory???"

The WSJ article includes some answers to the question of how all those dozens or hundreds of cores might be used. Here are a few snippets:

• jobs such as three-dimensional graphics
• automatically picking out faces from large databases of photographs
• helping a handheld device photograph and analyze the calorie content of each meal
• in the financial sector, ... millions of transactions need to be swiftly analyzed for patterns that can make investors money

Now, some of these applications may sound non-critical for the future survival of society. But that doesn't mean people won't want these apps if they become available. "Find me all the pictures in my thousands of digital images that include Sally" or "find me all the pictures on my hard drive that include both Sally and Toby"--I can easily imagine that application becoming one that is taken for granted in the relatively near future. "What? You don't have that app? How do you live without it? Huh? You still manually search through your digital images???"

It doesn't have to be a life-critical application in order for it to be an app that everyone wants to have.

"The future is really around media-rich applications," said Phil Hester, Advanced Micro's chief technology officer, who argued those jobs are particularly suited to chips with multiple types of microprocessor cores.

I've long agreed with this. And I think the investment by Microsoft and Intel is further evidence that the day when highly computational multithreaded applications are a standard on every home and office desktop (and even on many hand-held devices) is on its way.

I also expect that TBB will play an important role as all of this unfolds...

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

I've thought a lot about Herb's last article, which I commented on in my "Can Parallelism Achieve Superlinear Performance Gains" post. In that article, Herb used the example of searching for data in a situation where the data is clumped as representing a case where superlinear performance is possible when an application is multithreaded.

My further thoughts about this situation brought me to the conclusion that a multithreaded search compared with the single-threaded search Herb posited would indeed achieve apparent superlinear performance. But, that is because the sensible approach for searching for clumped data would not be an element-by-element linear search through an array of data. In other words, the superlinearity would be achieved only because the proposed single-threaded methodology was not the ideal approach to the problem.

What if you lost your ring in a loaf of bread?

Think about it this way. You are baking a loaf of bread. You suddenly realize that your favorite ring is no longer on your finger! You search all over the counters. Finally, you conclude your ring must be inside that loaf of bread that's baking in the oven.

So, you wait until the bread is fully baked, then begin your search for the ring. Now, your ring is a relatively large object that is located somewhere inside that loaf of bread. It's "clumped data" within the background of solid bread. So, if your objective is to find that ring as quickly as possible, what will your approach be? Will you start from one end of the loaf, and cut almost infinitely tiny slices until you reach the ring? Or, would common sense suggest that you chop the loaf into slices that are more ring-sized? Since all you have to do is touch the ring, and you'll have found it.

The single-threaded method of starting at one end of the loaf and slicing tiny slices will clearly not be the most efficient method. The parallel method of chopping the loaf into eighths so you, your family, and several friends can each work with 1/8th of the loaf to try to find the ring will in most cases attain a greater than 8 times efficiency than the tiny-slices-starting-at-one-end method.

But would this really be superlinear performance? I think not. Because you yourself could have (should have) cut the loaf into eighths, then into 16ths, etc. That would have been the most efficient single-threaded method of finding that clumped data (i.e., your lost ring).

Conclusion

Superlinear performance is impossible. However, when we're programming, our brains don't always think of strategies that would be common sense if we were facing a "real world" problem (like finding a relatively large hard object inside a loaf of bread). Hence, parallelism will sometimes appear to produce superlinear results. But if we apply the most efficient method to the problem in the first place, the illusion of superlinear performance will never appear.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

2008 March 4 TBB Development release

The CHANGES file for the tbb20_20080304oss development release highlights these changes:

• Task-to-thread affinity support, previously kept under a macro, now fully legalized.
• Work-in-progress on cache_aligned_allocator improvements.
• Pipeline really supports parallel input stage; it's no more serialized.
• Various improvements to code, tests, examples and Makefiles.

and these bug fixes:

• 119 - fix for scalable_malloc sometimes failing to return a big block.
• TR575 - fixed a deadlock occurring on Windows in startup/shutdown under some conditions.

An important thing to note here is the Bug 119 fix. This is a bug that was entered by "uj" into the public TBB bug database. I've been telling people for a while that, while the online TBB bug database doesn't contain every bug (the internal bug list hasn't yet been merged into the public bug database), it's still very worthwhile for TBB users to post bugs into the public database, because the TBB development team pays attention to everything that's posted there. If you find what you believe may be a bug, please do take the time to post the bug, and help improve TBB!

A quick look at a diff of the 20080226 and 20080304 releases shows significant changes (mostly additions) to the cache_aligned_allocator.cpp and pipeline.cpp files in src/tbb, and many changes in src/tbb/task.cpp related to affinity. A lot of other files have a few altered lines, many of which are again related to the affinity changes mentioned in the CHANGES file.

The test_pipeline.cpp and test_task.cpp files in the src/test directory were significantly modified to better test the new pipeline and task changes.

2008 March 11 TBB Development release

The tbb20_20080311oss development release CHANGES files lists the following changes:

• An enumerator added for pipeline filter types (serial vs. parallel).
• New task_scheduler_observer class introduced, to observe when threads start and finish interacting with the TBB task scheduler.
• task_scheduler_init reverted to not use internal versioned class; binary compatibility guaranteed with stable releases only.
• Various improvements to code, tests, examples and Makefiles.

To me, the new task_scheduler_observor class is quite interesting. It would seem that this class will empower developers with capability to tune their applications for greater efficiency and performance. I'd like to investigate this new class more thoroughly in a future post.

Looking at a diff between the 20080304 and 20080311 development releases, changes in src/tbb/task.cpp stand out in terms of number of changed lines (most of them additions). This backs up my expressed interest in the new task_scheduler_observor class, since much of the code for the observor is in the task.cpp file. Further investigation of this is promised!

Conclusion

It's clear that the TBB development team is hard at work. The team is working toward an upcoming significant release--maybe not a 3.0, but a 2.1 anyway. I encourage those who are experimenting with TBB to download and work with the later development releases. There's a lot there that is not available in the latest stable or commercial-aligned releases. And, from what I'm told, much of what's in these recent development releases--all the new features and improvements to the base code, the tests, the examples--is likely to solidify into a highly-stable commercial-aligned release sometime in the next few months. I heartily encourage TBB experimenters to work with the latest TBB development releases.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

• Introduced tbb_allocator to select between standard allocator and tbb::scalable_allocator when available.
• Removed spin-waiting in pipeline and concurrent_queue.
• Improved performance of concurrent_hash_map by using tbb_allocator.
• Improved support for Intel® Thread Checker.
• Various improvements to code, tests, examples and Makefiles.

I differenced the 20080226 code with the previous TBB development release (tbb20_20080207oss), to look more closely at the new changes.

concurrent_queue

The concurrent_queue changes involved lots of new lines. Most of the changes appear to be related to the elimination of spin-waiting. This is one of the most significant changes in the 20080226 version. Many of the changes are related to pthreads and pthread mutexes. There is also some Windows-specific new code.

If you've studied the TBB concurrent_queue source code previously, you'll want to take a look at the new code, since the changes are substantial.

pipeline

As with the concurrent_queue, spin-waiting was eliminated from the TBB pipeline component. The changes to accomplish this in pipeline were much smaller than was the case for concurrent_queue.

Changes in other TBB components

The atomic, cache_aligned_allocator, concurrent_hash_map, concurrent_vector, pipeline, scalable_allocator, and spin_rw_mutex TBB components were changed, but the number of new or changed lines of code for these components was relatively small.

Quite a few ASSERTs were added to the task source files. There were quite a lot of changes to the tbb_misc.* files, many related to the __TBB_WEAK_SYMBOLS defined variable. There were changes to itt_notify.cpp, and scattered changes in some other files.

Build changes

There were a lot of changes to the files in the build subdirectory: the *.inc files for the supported operating systems and the makefiles were modified. Changes were also made to many of the example problem makefiles, and the test makefiles. In addition, there were platform-related changes to the *_tbb_export.def files located in the src/tbb subdirectory.

There's probably not a lot of need for developers to look closely at the build changes, unless you've made your own changes to the build files and want to see how to merge your changes with the latest TBB build files.

Example and test program changes

There are some fairly small changes in the parallel_reduce/convex_hull and test_all/fibonacci examples (in the examples subdirectory). Several of the src/test programs included minor changes.

Conclusion

The tbb20_20080226oss development release embodies almost three weeks of development work (the prior development release is dated 20070207). The new release includes a lot of small improvements in TBB components, build and platform improvements, and fairly major performance improvements for the concurrent_queue and pipeline components.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB

To reiterate, what I'm trying to do is indeed a "hack"; I'm not at this point doing what could be considered a nicely rendered port of TBB into the Cygwin environment. My objective is simply to find and make the modifications necessary so that I can launch Cygwin and apply TBB within that environment.

Unknown OS redux, redux, ...

The TBB source code in many areas is tailored for individual operating systems(_WIN32, _WIN64, __linux__, or __APPLE__). Where the code cannot be compiled due to an undefined OS, the compiler stops and displays messages of this type:

In file included from ../../src/tbb/concurrent_queue.cpp:33:
../../src/tbb/tbb_misc.h:86:2: #error Unknown OS
make[1]: *** [concurrent_queue.o] Error 1

I had defined my operating system to be Cygwin. Hence, everywhere in the TBB code where specific code was defined for individual operating systems, I had to modify the code to provide a set of code to be compiled when the defined OS is Cygwin. Since I was building TBB using GCC, and since Cygwin is in many ways similar to Linux, I often applied the code assigned for __linux__ for my Cygwin TBB build. In most cases, this worked.

Taking tbb_misc.h as an example, the original code included this set of OS-specific source:

namespace tbb {

static volatile int number_of_workers = 0;

#if defined(__TBB_DetectNumberOfWorkers)
static inline int DetectNumberOfWorkers() {
    return __TBB_DetectNumberOfWorkers();
}
#else
#if _WIN32||_WIN64

static inline int DetectNumberOfWorkers() {
    if (!number_of_workers) {
        SYSTEM_INFO si;
        GetSystemInfo(&si);
        number_of_workers = static_cast(si.dwNumberOfProcessors);
    }
    return number_of_workers;
}

#elif __linux__ 

static inline int DetectNumberOfWorkers( void ) {
    if (!number_of_workers) {
        number_of_workers = get_nprocs();
    }
    return number_of_workers;
}

#elif __APPLE__

static inline int DetectNumberOfWorkers( void ) {
    if (!number_of_workers) {
        int name[2] = {CTL_HW, HW_AVAILCPU};
        int ncpu;
        size_t size = sizeof(ncpu);
        sysctl( name, 2, &ncpu, &size, NULL, 0 );
        number_of_workers = ncpu;
    }
    return number_of_workers;
}

#else

#error Unknown OS

#endif /* os kind */

#endif

My first attempt was to copy the __linux__ definition of the DetectNumberOfWorkers() function into #error Unknown OS section, and comment out the error message. In this case, it didn't work (it didn't compile). So, just to be able to move on, I hard-coded the number_of_workers variable to be 4 (since I'm doing this on my quad-core system). That's not how you'd do it if you were making a true port, mind you!

So, here's what my modified code segment looks like:

#else

//cygwin: use known setting (quad core)
static inline int DetectNumberOfWorkers( void ) {
    //if (!number_of_workers) {
    //    number_of_workers = get_nprocs();
    //}
    number_of_workers = 4;
    return number_of_workers;
}
//#error Unknown OS

#endif /* os kind */

Next followed similar changes in response to "Unknown OS" errors in src/tbb/itt_notify.cpp and src/tbb/cache_aligned_allocator.cpp. In both cases, I replaced the "Unknown OS" error line with the code that was defined for __linux__.

Issues of various kinds...

Next, my build produced an error in src/tbb/task.cpp:

src/tbb/task.cpp:400: error: invalid conversion from 'int*' to 'int32_t*'

The code in question is checking a set of multibyte characters for the string "GenuineIntel". This didn't seem critical to successful operation of TBB within the Cygwin environment, so I commented out the entire area of code that was producing the error and set the result to true:

//! True if running on genuine Intel hardware
static inline bool IsGenuineIntel() {
    bool result = true;
#if defined(__TBB_cpuid)
    char info[16];
    char *genuine_string = "GenuntelineI";
//cygwin: comment out line that caused error; set result to true
//    __TBB_x86_cpuid( reinterpret_cast(info), 0 );
    // The multibyte chars below spell "GenuineIntel".
    //if( info[1]=='uneG' && info[3]=='Ieni' && info[2]=='letn' ) {
    //    result = true;
    //}
//    for (int i = 4; i < 16; ++i) {
//        if ( info[i] != genuine_string[i-4] ) {
//            result = false;
//            break;
//        }
//    }
    result = true;  //cygwin
#elif __TBB_ipf
    result = true;
#else
    result = false;
#endif
    return result;
}

My next problem was related to build/version_info_linux.sh. I had created my cygwin.inc file using linux.inc as a template. version_info_linux.sh applies Linux commands to produce a version information string. Many scripting commands that are valid in Linux are not valid in Cygwin. So, I created a new file, build/version_info_cygwin.sh, and had my cygwin.inc file call that instead of calling the Linux version. I hard-coded variables as necessary simply to be able to move on quickly (again, this is a hack, not a port). Here's the core section of my version_info_cygwin.sh file:

#cygwin - based on version_info_linux.sh - many changes

echo "#define __TBB_VERSION_STRINGS \\"
#cygwin echo '"TBB: ' "BUILD_HOST\t\t"`hostname -s`" ("`arch`")"'" ENDL \'
echo '"TBB: ' "BUILD_HOST\t\tQUAD_CORE ("`arch`")"'" ENDL \'
echo '"TBB: ' "BUILD_OS\t\t"`head -1 /etc/issue | sed -e 's/\\\\//g'`'" ENDL \'
#cygwin echo '"TBB: ' "BUILD_KERNEL\t"`uname -rv`'" ENDL \'
echo '"TBB: ' "BUILD_KERNEL\t1.5.25"'" ENDL \'
echo '"TBB: ' "BUILD_GCC\t\t"`g++ -v &1 | grep 'gcc.*version'`'" ENDL \'
[ -z "$COMPILER_VERSION" ] || echo '"TBB: ' "BUILD_COMPILER\t"$COMPILER_VERSION'" ENDL \'
echo '"TBB: ' "BUILD_GLIBC\t2.3.5"'" ENDL \'
echo '"TBB: ' "BUILD_LD\t\t"`ld -v | grep 'version'`'" ENDL \'
echo '"TBB: ' "BUILD_TARGET\t$arch on $runtime"'" ENDL \'
echo '"TBB: ' "BUILD_COMMAND\t"$*'" ENDL \'
echo ""
echo "#define __TBB_DATETIME \""`date -u`"\""

Almost there!

My next execution of make brought me all the way to the ld process. There, I was told that the -lrt option was not valid. The fix for this was simple: I edited my cygwin.gcc.inc file and removed the -lrt option:

LIB_LINK_FLAGS = -shared
#cygwin LIBS = -lpthread -lrt -ldl
LIBS = -lpthread -ldl

The next error message was:

src/tbbmalloc/MemoryAllocator.cpp:339:
   #error highestBitPos() not implemented for this platform

This was somewhat similar to the earlier "Unknown OS" errors. But in this case, the base TBB code defines instructions for __ARCH_unknown. I chose to implement this code for Cygwin, to avoid having to get into the ASSEMBLER instructions that are defined for other operating systems. So, this area of my MemoryAllocator.cpp looks like this:

static inline unsigned int highestBitPos(unsigned int number)
{
    unsigned int pos;
#if __ARCH_x86_32||__ARCH_x86_64

# if __linux__||__APPLE__
    __asm__ ("bsr %1,%0" : "=r"(pos) : "r"(number));
# elif (_WIN32 && (!_WIN64 || __INTEL_COMPILER))
    __asm
    {
        bsr eax, number
        mov pos, eax
    }
# elif _WIN64 && _MSC_VER>=1400
    _BitScanReverse((unsigned long*)&pos, (unsigned long)number);
# else
//cygwin
//#   error highestBitPos() not implemented for this platform
    static unsigned int bsr[16] = {0,6,7,7,8,8,8,8,9,9,9,9,9,9,9,9};
    MALLOC_ASSERT( number>=64 && number>6 ];
# endif

#elif __ARCH_ipf || __ARCH_unknown
    static unsigned int bsr[16] = {0,6,7,7,8,8,8,8,9,9,9,9,9,9,9,9};
    MALLOC_ASSERT( number>=64 && number>6 ];
#else
#   error highestBitPos() not implemented for this platform
#endif
    return pos;
}

Success!

I re-executed make release, and it was apparently successful: release versions of libtbb.so and libtbbmalloc.so were created!

Next I tried running tbbvars.sh, then making the sub_string_finder "Getting Started" example problem. The TBB *.h files were not being found. I tried lots of different things, and it became apparent that the TBB shared object files I'd just created weren't being noticed by the sub_string_finder build process. No matter what I did with my path definitions, nothing worked.

Finally, I tried copying the shared object files to new names, with the file extension changed from .so to .dll. The make worked!

I ran sub_string_finder_extended and got the following results:

$ ./sub_string_finder_extended.exe
 Done building string.
 Done with serial version.
 Done with parallel version.
 Done validating results.
Serial version ran in 6.291 seconds
Parallel version ran in 1.627 seconds
Resulting in a speed up of 3.86663

TBB is working under Cygwin! I see a 3.86 speedup on my quad-core Windows system.

Not all of the TBB examples worked. For example, tacheon and other examples that involve graphics aren't working. This could be due to missing components in my Cygwin installation. I'm not too worried about that. These TBB example problems do work under my current Cygwin TBB build:

• concurrent_hash_map/count_strings
• GettingStarted/sub_string_finder
• parallel_reduce/primes
• parallel_while/parallel_preorder
• pipeline/textfilter
• task/tree_sum
• test_all/fibonacci

Conclusion

I'm quite thrilled to have TBB built and running using GCC under Cygwin, such that it fully utilizes all of my system's four cores. Next up: MinGW (Minimalist GNU for Windows); and ultimately a return to my "Building Threading Building Blocks on UWIN" effort.

Kevin Farnham, O'Reilly Media TBB Open Source Community, Freenode IRC #tbb, TBB Mailing Lists

Download TBB