1,364 Posts served

5,674 Conversations started

Volatile: Almost Useless for Multi-Threaded Programming

By Arch Robison (Intel) (19 posts) on November 30, 2007 at 12:44 pm

There is a widespread notion that the keyword volatile is good for multi-threaded programming. I've seen interfaces with volatile qualifiers justified as "it might be used for multi-threaded programming". I thought was useful until the last few weeks, when it finally dawned on me (or if you prefer, got through my thick head) that volatile is almost useless for multi-threaded programming. I'll explain here why you should scrub most of it from your multi-threaded code.

Hans Boehm points out that there are only three portable uses for volatile. I'll summarize them here:

• marking a local variable in the scope of a setjmp so that the variable does not rollback after a longjmp.
• memory that is modified by an external agent or appears to be because of a screwy memory mapping
• signal handler mischief

None of these mention multi-threading. Indeed, Boehm's paper points to a 1997 comp.programming.threads discussion where two experts said it bluntly:

"Declaring your variables volatile will have no useful effect, and will simply cause your code to run a *lot* slower when you turn on optimisation in your compiler." - Bryan O' Sullivan

"...the use of volatile accomplishes nothing but to prevent the compiler from making useful and desirable optimizations, providing no help whatsoever in making code "thread safe". " - David Butenhof

If you are multi-threading for the sake of speed, slowing down code is definitely not what you want. For multi-threaded programming, there two key issues that volatile is often mistakenly thought to address:

1. atomicity
2. memory consistency, i.e. the order of a thread's operations as seen by another thread.

Let's deal with (1) first. Volatile does not guarantee atomic reads or writes. For example, a volatile read or write of a 129-bit structure is not going to be atomic on most modern hardware. A volatile read or write of a 32-bit int is atomic on most modern hardware, but volatile has nothing to do with it. It would likely be atomic without the volatile. The atomicity is at the whim of the compiler. There's nothing in the C or C++ standards that says it has to be atomic.

Now consider issue (2). Sometimes programmers think of volatile as turning off optimization of volatile accesses. That's largely true in practice. But that's only the volatile accesses, not the non-volatile ones. Consider this fragment:

    volatile int Ready;       

    int Message[100];      

    void foo( int i ) {      

        Message[i/10] = 42;      

        Ready = 1;      

    }

It's trying to do something very reasonable in multi-threaded programming: write a message and then send it to another thread. The other thread will wait until Ready becomes non-zero and then read Message. Try compiling this with "gcc -O2 -S" using gcc 4.0, or icc. Both will do the store to Ready first, so it can be overlapped with the computation of i/10. The reordering is not a compiler bug. It's an aggressive optimizer doing its job.

You might think the solution is to mark all your memory references volatile. That's just plain silly. As the earlier quotes say, it will just slow down your code. Worst yet, it might not fix the problem. Even if the compiler does not reorder the references, the hardware might. In this example, x86 hardware will not reorder it. Neither will an Itanium(TM) processor, because Itanium compilers insert memory fences for volatile stores. That's a clever Itanium extension. But chips like Power(TM) will reorder. What you really need for ordering are memory fences, also called memory barriers. A memory fence prevents reordering of memory operations across the fence, or in some cases, prevents reordering in one direction. Paul McKenney's article Memory Ordering in Modern Microprocessors explains them. Sufficient for discussion here is that volatile has nothing to do with memory fences.

So what's the solution for multi-threaded programming? Use a library or language extension hat implements the atomic and fence semantics. When used as intended, the operations in the library will insert the right fences. Some examples:

• POSIX threads
• Windows(TM) threads
• OpenMP
• TBB

For example, the parallel reduction template in TBB does all the right fences so you don't have to worry about them.

I spent part of this week scrubbing volatile from the TBB task scheduler. We were using volatile for memory fences because version 1.0 targeted only x86 and Itanium. For Itanium, volatile did imply memory fences. And for x86, we were just using one compiler, and catering to it. All atomic operations were in the binary that we compiled. But now with the open source version, we have to pay heed to other compilers and other chips. So I scrubbed out volatile, replacing them with explicit load-with-acquire and store-with-release operations, or in some cases plain loads and stores. Those operations themselves are implemented using volatile, but that's largely for Itanium's sake.  Only one volatile remained, ironically on an unshared local variable! See file src/tbb/task.cpp in the latest download if your curious about the oddball survivor.
- Arch

Categories: Multicore, Threading Building Blocks

Comments (23) 

By John "Z-Bo" Zabroski on December 22nd, 2007 at 5:18 pm
Volatile semantics depend on the language, but volatile essentially means it's possible to reach inside an object's state-process through some external modification. The higher truth it seems you are trying to lift up is that volatile is an attempt at pruning non-determinism, but it's too coarse grained because it interferes with the assumptions optimizing compilers make.

An example of an alternative to using volatile is to use a surrogate object that manages the resource. For example, in Java, Bill Pugh's Initialization on Demand Holder Idiom uses a surrogate object to wrap a place holder for a resource that hasn't been acquired yet. It's a little strange to think of it as a surrogate object, but it helps to realize that once the resource it's guarding is used, all threads effectively see the same shadow, even the the blueprint for each thread might use a different variable name to refer to the shadow. All threads can manipulate the shadow.

Fences and Atoms are, in my eyes, kinds of surrogate objects. I'm not sure if that is a useful metaphor, though. I'm just 23 and don't have enough experience writing multi-threaded programs with complicated resource contention issues.

Also, I think your opposition to the use of volatile might be found here: http://www.ddj.com/cpp/184403766 and he discusses it more here: http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf
Andrei also has two C++ standards survey papers related to this: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2004/n1680.pdf
http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2005/n1777.pdf

By Tom on February 8th, 2008 at 3:46 am
There are a couple of misunderstandings in your article. While it is true that using volatile alone and expecting this to make anything thread-safe is a naive (and wrong assumption), it is neither true that volatile is useless, or nearly so.

What volatile does is, it prevents the compiler to cache a value in a register and do optimisations that remove operations in which the variable is involved.
This is an important property, which is necessary in multi-threaded applications. Atomicity in loads or stores is not related to the C/C++ standard, but is a hardware feature (as long as the addressed units are no larger than register size). However, again, you miss the point here. It does not matter whether or not these are atomic.
What matters is that load-modify-store and compare-exchange functions can be made atomic by using the proper instructions (via intrinsics, assembly, or kernel functions). This is what is needed to properly synchronize data between threads. If you can't be sure that the compiler won't optimize out a variable, or hold it in a register, or performs any other smart stuff, then this doesn't work.

While it is true that volatile variables are a lot slower to access, even more so if atomic instructions are used (up to 10-15 times slower), the statement that this is "definitively not what you want with threads" shows that you really haven't understood. It is, in fact, EXACTLY what you want.
What you don't want to happen is one thread on one core/cpu increment a counter while you use the now invalid value in another thread on another cpu. What you don't want to happen is one thread freeing (or simply changing) memory that you are still accessing in another (currently waiting) thread. What you don't want to happen is two threads taking the same head element from a job queue at the same moment, performing the same work twice and finally calling delete on the same pointer twice.
All these issues can of course be safely synchronized by locking/unlocking before every access. However, THIS is what you don't want for the sake of performance.
Constructs using volatile variables in combination with atomic instructions (read up on "lockfree programming") offer a much better solution, especially in highly congested scenarios. It is a lot better to burn two dozen CPU cycles using atomic instructions on a volatile than having every access synchronized by two syscalls (lock/unlock) which will chew up several hundred to thousand cycles each.

By Arch Robison (Intel) on February 11th, 2008 at 8:29 am
What matters for multithreading is:

1. Atomicity
2. Visibility of memory operations
3. The order in which memory operations become visible.

Volatile in C and C++ flunks on all three counts. All volatile does is prevent a compiler from caching a variable, which is orthogonal to both points above. It just slows programs down. I'll go into points (1) and (2) in detail. My original post already flogged (3). As I'll show, (2) is enough to put a stake through volatile for portable multi-threaded programming.

As Tom notes, atomicity requires using the proper atomic instructions. Volatile does not address this. E.g., even if I declare x as volatile, x+=1 is not going to be compiled as an atomic increment. So special instructions outside the scope of the C/C++ standards must be used to access/modify x atomically. But that implies that every atomic access to x is outside the scope of the C/C++ standards. As far as the C/C++ standards are concerned, all the compiler sees is the address of x (or reference to x) being passed to routines outside the ken of the compiler. So declaring x as volatile is pointless; the compiler cannot cache loads/stores to x because in principle it does not see the loads/stores to x.

Of course for specific compiler implementations we might know that a volatile load or store of a certain size is always compiled as an atomic operation. That's how TBB implements its internal __TBB_load_with_acquire and __TBB_store_with_release operations for some platforms. But we declare only the formal parameter as pointer-to-volatile and do not declare the actual variable as volatile, because what we are doing is platform specific. Indeed, if we were squeaky-clean about it, we would not even declare the formal parameter as pointer-to-volatile, but hide this platform-specific detail completely by casting the formal parameter to pointer-to-volatile. The portable portions of code should not declare any variables as volatile. They should call __TBB_load_with_acquire and __TBB_store_with_release to do the atomic loads and stores.

Now consider point (2), where the ISO C/C++ volatile is downright counterproductive. (Except on Itanium, because of an Intel-specific interpretation of volatile.) Consider the job queue example. Let's assume the queue holds either zero or one elements. Such a queue can be implemented as a shared pointer R in memory that is either NULL or points to the queue's element. I'm simplifying the queue to make the core issue more obvious. With serious queue implementations the same issues strike with a vengance.

1. Thread 1 multiplies computes a matrix product M and atomically sets R to point to M.
2. Thread 2 waits until R!=NULL and then uses M as a factor to compute another matrix product.

In other words, M is a message and R is a ready flag.

Let's consider sequentially consistent machines. Never mind that memory fence issues that are critical to real machines and not addressed by volatile. I'll show that volatile is both a performance killer and useless for a sequentially consistent machine. Here's the key excerpt in the C99 standard:

The least requirements on a conforming implementation are:

• At sequence points, volatile objects are stable in the sense that previous accesses are
  complete and subsequent accesses have not yet occurred.

Note that only volatile objects are required to be "stable". Volatile accesses have no effect on non-volatile accesses. Thus to rely on volatile in the example requires declaring both R and M as volatile. Declaring M as volatile slows down operations on it significantly. In general, much of multithreaded programming relies on a notion of privatization, where one thread operates on an object (such as M) and then passes it off to another thread. At any point in time, the object is being accessed only by one thread. But if we are going to depend upon volatile to get the hand-off right, we have to declare the object as volatile. That inflicts a heavy penalty on access to the object. In principle, we just have to ensure that when handing off an object, that every location associated with the object was last written with a volatile store before the hand off, and that every first read of a location by the receiving thread is done with a volatile load after the hand-off. Keeping track of that information would definitely be a pain.

But it gets worse. Consider a thread handing off a std::map object to another thread. There's no way for a thread to even know all the locations inside the implementation std::map and mark them all volatile. I suppose a program could serialize the object into a volatile buffer, hand off the buffer, and reconstruct it from the buffer. That essentially inflicts all the pain of message-passing onto shared memory programming. Shared memory programming has enough pain. What we really want are memory fences that force the std::map to be written to memory by the sending thread before the receiving thread reads it. We do not want to turn off the optimizer, but merely enforce the order of some writes and reads.

To summarize, multi-threaded programming is about atomicity and very precise control of the order in which memory operations become visible. Volatile does not address atomicity. Marking an object as volatile turns off caching of the object's value, which is a terribly imprecise and inefficient way to achieve the desired order of visibility, because all locations passed between threads would have to be marked volatile. To get the correct order efficiently requires some notion of memory fencing, which is outside the current C/C++ standards, but will be in future versions of those standards.

By kappa on March 25th, 2008 at 12:29 am
link "Hans Boehm points out that there are only three portable uses for volatile" has title but not href

By Arch Robison (Intel) on March 25th, 2008 at 5:52 am
Thanks for pointing out the missing linke. It's now repaired.

By regehr on April 18th, 2008 at 8:54 am
Arch-- just wanted to point you to some results that may indicate that volatile is even less useful than one would hope, since compilers tend to not properly respect it.

http://www.cs.utah.edu/~regehr/papers/emsoft08_submit.pdf

John Regehr

By regehr on April 18th, 2008 at 10:57 am
Also here's an example (oddly, icc gets it right):

[regehr@babel ~]$ cat > foo.c
volatile int x;
void foo (void)
{
x;
}
[regehr@babel ~]$ icpc -S foo.c
[regehr@babel ~]$ cat foo.s
# -- Machine type IA32
# mark_description "Intel(R) C++ Compiler for applications running on IA-32, Version 10.1 Build 20070913 %s";
# mark_description "-S";
.file "foo.c"
.text
..TXTST0:
# -- Begin _Z3foov
# mark_begin;
.align 2,0x90
.globl _Z3foov
_Z3foov:
..B1.1: # Preds ..B1.0
ret #5.1
.align 2,0x90
# LOE
# mark_end;
.type _Z3foov,@function
.size _Z3foov,.-_Z3foov
.data
# -- End _Z3foov
.bss
.align 4
.align 4
.globl x
x:
.type x,@object
.size x,4
.space 4 # pad
.data
.section .note.GNU-stack, ""
# End
[regehr@babel ~]$

By Arch Robison (Intel) on April 18th, 2008 at 11:48 am
I liked the paper. Before I thought volatile was useless; now I'm scared of it :-)

What about checking volatile on fields and structures? A compiler could forget to propagate a volatile qualifier on struct to the fields inside. Likewise for a volatile array (as opposed to an array of volatile elements). The restrict keyword from C99 might offer further mischief. E.g., there may be transforms so focussed on restrict that they forget about volatile. Cast and inlining offers other possibilities for compiler error when changing the "to" and "from" types differ in volatile qualifiers.

One of the inventors of C (Dennis Ritchie) was against volatile (and const). See here.

By regehr on April 18th, 2008 at 3:11 pm
Thanks for the comments!

Definitely structs and arrays would be great to test.

That DMR essay is great -- I wonder how "restrict" snuck back in? I heard of a great study (don't have a reference handy unfortunately) where someone profiled programs' memory behavior in order to add in a maximal amount of restrict qualifiers, then recompiled and got no speedup at all :)

I think it is not hard to argue against all uses of volatile. As you say, it's a poor choice for communication and synchronization between threads. Register accesses can be done through function calls to asm stubs. That doesn't seem to leave many uses...

Anyway thanks for the example about moving memory operations past volatile operations, a friend of mine who works at a major embedded systems company didn't believe that this would ever be done by a compiler until I pointed him to this blog post.

By Yet Another Programming Weblog on April 24th, 2008 at 8:43 pm
anything other than manipulating memory mapped hardware, or for very limited communication between threads, it is very likely that you are making a mistake. Think carefully about what volatile means and about what it does not mean. En la misma linea,Volatile: Almost Useless for Multi-Threaded ProgrammingVarias discusiones en usenet: C++, volatile member functions, and threads, volatile guarantees? y Memory Barriers, Compiler Optimizations, etc., todas ellas en comp.programming.threadsUna discusión sobre lo que es, que no es y que quizás debiera ser

By Bitácora de mig21 (7781) on April 25th, 2008 at 10:46 am
anything other than manipulating memory mapped hardware, or for very limited communication between threads, it is very likely that you are making a mistake. Think carefully about what volatile means and about what it does not mean. En la misma linea,Volatile: Almost Useless for Multi-Threaded ProgrammingVarias discusiones en usenet: C++, volatile member functions, and threads, volatile guarantees? y Memory Barriers, Compiler Optimizations, etc., todas ellas en comp.programming.threads Una discusió

By Coding Relic on May 3rd, 2008 at 1:43 pm
Commenting on reddit, dmh2000 suggested two other articles on this topic for your edification and bemusement. I found them both interesting, so I'll pass them on here as well:volatile by Ian Lance TaylorVolatile: Almost Useless for Multi-Threaded Programmingby Arch Robison I also updated the formatting of this article, because it looked terrible in the RSS feed. I'm still figuring out this newfangled CSS thing all the youngsters seem to like.

By Spud on May 20th, 2008 at 9:02 am
Well, how about a working thread repeatedly checking if the job has been cancelled?
assuming that bool read writes are atomic will the following c++ snippet work as expected?

class WorkThread
{
	volatile bool abort;
public:
	void run()
	{
		...
		abort=false;
		while(job.notFinished())
		{
			job.doChunk();
			i f(abort)
				return;
		}
		...
	}
	void cancel()
	{
		abort=true;
	}
};

I know I will still need a mutex/waitcondition or similar to synchronize threads, but it would be shocking to find out that the above code could be executed in some arbitrary order.

By Arch Robison (Intel) on May 20th, 2008 at 8:30 pm
Yes, the above should work given the assertions. At worst it can hoist the read of "abort" above job.doChunk(), which presumably does no damage in an example like this.

By Nervousone on May 30th, 2008 at 10:26 pm
>void foo( int i ) {
> Message[i/10] = 42;
> Ready = 1;
> }
>The reordering is not a compiler bug.

In my opinion reordering as described IS a compiler bug. It's just stupid
and dangerous so it's gcc fault. Compiler should NEVER do that with
volatile. Read&write of those should ALWAYS stay in place. Someone
must had a reason to set variable volatile. Preceding and following code
blocks could be reordered as much as want, but one should expect
at least that preceeding code WAS executed and following WAS NOT.
Any other aproach is just bad and for me only option is ignore fact that
some bunch of people was not thinking first when projecting (damn comities)
then another bunch when writing (compiler) and I am happy at least I can
just turn optimization off in functions accessing volatile, but that not deny
stupidity of compilers ignoring volatile qualifier and doing whatever want.
I'm not talking now about other things as hardware caches or memory issues,
but have no idea who gave anyone right to consider volatile's in ANY
kind of optimisations ?!? That kind of data definitely shoud be excluded
from it entirely and unconditionary.

By Arch Robison (Intel) on June 2nd, 2008 at 11:06 am
The committees indeed think very hard about this sort of thing. Adding fence semantics to volatile was considered by the C++ committee. See N2016 for why adding fence behavior ("inter-thread visibility") to volatile was rejected.

Instead, C++ 200x has support for fencing via its atomic operations library. See Chapter 29 of the working draft. With that library, the example can be written correctly as:

    std::atomic_bool Ready;
    int Message[100];
    void foo( int i ) {
        Message[i/10] = 42;
        Ready = 1;
    }

Intel Threading Building Blocks has a class tbb::atomic<bool> that similarly makes the example work. Disclaimer: that is a shameless plug from TBB's architect :-)

By 夏天可是个好季节 - 博客园 on June 15th, 2008 at 5:36 am
Fence）！因此Intel技术博客上有文章指出：“volatile 对于多线程编程几乎没有任何用处”。 （2）当你使用Memory Barrier或者Memory Fence的时候你从某种程度上（取决于你使用了哪一种）阻止了CPU和编� ��器对执行顺序的改变。

By 夏天可是个好季节 - 博客园 on June 15th, 2008 at 5:51 am
的时候的时候，你阻止了编译器对相应内存代码的优化。阻止了大部 分的编译器造成的执行顺序的改变。但是你根本不能够阻止CPU对执行 顺序的改变！因此Intel技术博客上有文章指出：“volatile 对于多线程编程几乎没有任何用处”。 （2）当你使用Memory Barrier的时候你从某种程度上（取决于你使用了哪一种）阻止了CPU对� �行顺序的改变。

By cnblogs.com on June 15th, 2008 at 5:57 am
的时候的时候，你阻止了编译器对相应内存代码的优化。阻止了大部 分的编译器造成的执行顺序的改变。但是你根本不能够阻止CPU对执行 顺序的改变！因此Intel技术博客上有文章指出：“volatile 对于多线程编程几乎没有任何用处”。 （2）当你使用Memory Barrier的时候你从某种程度上（取决于你使用了哪一种）阻止了CPU对� �行顺序的改变。

By Ondrej Spanel on July 7th, 2008 at 1:30 am
I think this article by Andrei Alexandrescu (a few years old) outlines a usage of volatile which seems to be quite useful for multithreaded programming:

http://www.ddj.com/cpp/184403766

My summary would be: using volatile on built-in types is useless and dangerous, but using it on objects (and making use of C++ type checking) is very useful.

By David Schwartz on August 9th, 2008 at 9:30 pm
You are bang on about everything. Just to respond to:

"In my opinion reordering as described IS a compiler bug. It's just stupid
and dangerous so it's gcc fault. Compiler should NEVER do that with
volatile. Read&write of those should ALWAYS stay in place. Someone
must had a reason to set variable volatile. Preceding and following code
blocks could be reordered as much as want, but one should expect
at least that preceeding code WAS executed and following WAS NOT."

What you are saying is that the compiler should penalize legitimate users of 'volatile' (for the things it's documented to be safe for) so that you can abuse it.

Developers of threaded code have two choices:

1) They can extend 'volatile' so that it does what they want. People who use 'volatile' for the purposes suggested in the C standard will suffer a performance penalty. And since 'volatile' is only on or off, it will have to do everything (force ordering, force atomicity, force all types of visibility), so all code that uses it will be very slow.

2) That can let 'volatile' serve its intended purposes and add their own synchronization mechanisms that are finely-tuned to their specific requirements.

For reasons that should be obvious, '2' was selected. So 'volatile' does not force memory ordering because if it did, code that didn't need that would suffer a penalty for no reason. Instead, there are ways to force memory ordering where you need it, such as memory barriers.

By Irwin on August 27th, 2008 at 10:57 am
It is good to note that the volatile semantics of Java are such that volatile acts as a memory barrier and prevents reordering. So if you program in Java, don't scrape volatile from your language yet... it's still a very handy keyword !

By Arch Robison (Intel) on August 27th, 2008 at 2:42 pm
Right. In C# volatile also has the fence semantics. It's another example where Java/C# use tokens similar to C++, but have very different semantics.

What do you think?

Name (required)

Email (required; will not be displayed on this page)

Your URL (optional)

Comments (required)