Add another state, similar to reexecute, perhaps delayed_reexecute. Normally a reexecute calls an immediate spawn on the task. delayed_reexecute would set a local task* delayed_task variable (initially set to NULL) and then after the next successful get_task or steal_task the delayed_task would be spawned and the delayed_task reset to NULL.

This method adds a couple of if tests against the delayed_task variable, one in the get_task middle loop and one in the steal_task outer loop, so reasonably small overhead.

Unfortunately, my app is not is in a position to easily test this, but I will do so as soon as I can.

I've read Cilk papers some time ago. Can you, please, explain why
(A) work must be structured as balanced tree (I mean not ideally balanced, just sufficiently balanced)
not followed from
(B) work/span >> P
?

I can't imagine unbalanced tree with work/span >> P. Also take into account that stealing is expensive operation.

Dmitriy V'jukov

Wouldn't the following provide a solution?

If a tbb thread could recycle what is left of its task as a new task and then force a task steal rather than immediately checking to see what other work it had available then there is a chance it would do some useful work - possibly even helping the first tbb thread as it is busy resolving the saved data set (assuming that was broken into many tasks, which mine will be). Once it had completed the stolen task it would go back to normal processing of its task set.

I wouldn't claim my TBB knowledge to be anywhere near 100% yet... but I believe a TBB thread will always execute all task in its task set before looking to steal, so I don't think there is a way of simulating this process by playing around with levels in the stack of task lists.

It may be better if this thread moved onto some other task in its task set if such tasks were available and only looked to steal if there were no other tasks. Then what I am proposing is that the task that would otherwise block would temporarily become invisible to the thread, probably only for one 'get next task' call.

I.e. Have a task.spawntemporarilyinvisible() call

Mick

Unfortunately, I do not have any data sets to share. The customers suggesting these models have their own concerns about privacy and have either not shared more than the sample code I show above or have constrained me under a nondisclosure agreement. But I can share anything I come up with personally, and I can point them to this exchange in the hope that they may participate to the level they are able.

Thanks for your comments. I hope we'll continue to keep your interest.

Work-stealing need not be balanced. The key issues are work (total ops) and span (critical-path length). Parallelism is work/span. As long as there's substantially more parallelism than processors (work/span >> P), work-stealing will give linear speed-up. Your [Dmitriy V'jukov] example has a large span compared to work, and so the parallelism is minimal. For more details, see http://supertech.csail.mit.edu/cilk/lecture-1.ppt. Other lectures and materials can be found here: http://supertech.csail.mit.edu/cilk/.

Re [Charles E. Leiserson]: Unfortunately, not all computational parallelism falls into simple packages of tree-structured tasks and data dependencies.

Btw, work-stealing scheduler supports only parallelism structured as *balanced* tree. Am I right here?
So user MUST structure parallelism as *balanced* tree. Provided extreme case of unbalanced tree, user can get extremely bad performance.

For example following structure *allows* parallelism, but will be executed extremely badly by work-stealing scheduler:

work11 forks work21, work22

work21 forks nothing
work22 forks work31, work32

work31 forks nothing
work32 forks work41, work42
etc

Is my understanding correct?

What do about following approach?

// pseudo-code
void task::operator()
{
secondary_t* sec = get_associated_secondary_element();
if (sec)
{
if (false == sec->is_computed())
{
if (sec->try_lock())
{
sec->compute();
sec->unlock();
}
else
{
// in fifo order, not in lifo!
postpone_task(this);
// maybe next time
return;
}
}
}
// sec is computed, can use it
fork_child1(sec);
fork_child2(sec);
}

You can model the computation to performed as a dag (directed acyclic graph). Each vertex has a value to be computed which depends on the values of its successors in the dag. Sinks (vertices with out-degree 0) can be computed with no further exploration. A vertex whose value has been assigned can provide its value to its predecessors without further computation.

Here's a randomized nondeterministic algorithm to compute the dag which should work well using Cilk or TBB, although I haven't implemented it. It uses at most P*S1 stack space. Recursively walk the dag in parallel, spawning each exploration of a successor. Label edges as white, grey, and black. All edges are initially white. If you traverse a white edge, color it grey. If you return across a grey edge, color it black. Now, in your parallel recursive tree walk, when you visit a vertex, if its value has already been computed, return the value. If not and there exists at least one white edge, take a _random_ white edge. Otherwise (all edges are black or grey), take a _random_ grey edge. (You're now chasing the tail of the processor that colored the edge grey speculatively hoping to help out with that subdag.) Use locks only to guarantee the atomicity of these operations on a single node.

I'll look at this problem with my students, both theoretically and in practice. If you have any data sets, let me know. Also, if you implement it, I'd be curious as to the results.

To Charles: yes, in general I agree with you. Particularly driven by the philosophy of Intel Threading Building Blocks to reduce areas where caches can cool, locks are to be avoided as much as possible. Yet locks are included in Threading Building Blocks so if they are evil, they are at least a necessary evil.

Unfortunately, not all computational parallelism falls into simple packages of tree-structured tasks and data dependencies. Briefly I described a couple of these application structures in my intro, but let me try to elaborate one and see if anyone has an alternate approach to the problem. Imagine an object processing system (could be a mechanical CAD package, or maybe a 3D renderer, or some physics simulation) wherein the objects to process have shared dependencies to secondary objects. Those secondary objects may be sub-assemblies or caches of precomputed data that themselves may be computed in parallel. Our example program may not know a priori what is the topology of those dependencies: a reasonable approach in this case is to describe a set of tasks where each describes processing for one of the primary objects. Presumably these objects are localized in memory and so fit the general criteria for processing in parallel on separate processing elements (PEs). Then one PE encounters a reference to a secondary object that requires its own computation. What to do here? We could dispatch this thread to compute the secondary object, but what if a second PE working on another primary object runs into a reference to the same secondary? Do we duplicate the computation? We could, at the cost of extra memory and duplicated processing. Better if we could somehow suspend the second PE until the processing/preparation of the secondary object is completed (cached data of the primary cooling the whole time, admittedly), at which point both PEs could proceed on computation of their primary objects. Sounds like a resource lock to me, although ideally we'd like to keep the second PE busy, not off in an idle loop or some other process while the thread it was running is suspended on that lock. What would be really cool is to structure things so that both PEs share the work of processing the secondary object and then each return to their primary object processing once that work is done. But there's questions of oversubscription and undersubscription and memory use and stack splitting and more to deal with here. I don't claim to have answers for all these but I think they're worth a discussion.

But before that, let me back up and repose your question: do you see a means to efficiently compute in the face of such object dependencies? I'd love to find that I'm overlooking an obvious solution that doesn't require locks. Got an idea?