g++ - Atomic swap in GNU C++

Question

I want to verify that my understanding is correct. This kind of thing is tricky so I'm almost sure I am missing something. I have a program consisting of a real-time thread and a non-real-time thread. I want the non-RT thread to be able to swap a pointer to memory that is used by the RT thread.

From the docs, my understanding is that this can be accomplished in g++ with:

// global
Data *rt_data;

Data *swap_data(Data *new_data)
{
#ifdef __GNUC__
    // Atomic pointer swap.
    Data *old_d = __sync_lock_test_and_set(&rt_data, new_data);
#else
    // Non-atomic, cross your fingers.                                          
    Data *old_d = rt_data;
    rt_data = new_data;
#endif
    return old_d;
}

This is the only place in the program (other than initial setup) where rt_data is modified. When rt_data is used in the real-time context, it is copied to a local pointer. For old_d, later on when it is sure that the old memory is not used, it will be freed in the non-RT thread. Is this correct? Do I need volatile anywhere? Are there other synchronization primitives I should be calling?

By the way I am doing this in C++, although I'm interested in whether the answer differs for C.

Thanks ahead of time.

Answer 1

Generally don't use volatile when writing concurrent code in C/C++. The semantics of volatile are so close to what you want that it is tempting but in the end volatile is not enough. Unfortunately Java/C# volatile != C/C++ volatile. Herb Sutter has a great article explaining the confusing mess.

What you really want is a memory fence. __sync_lock_test_and_set provides the fencing for you.

You will also need a memory fence when you copy (load) the rt_data pointer to your local copy.

Lock free programming is tricky. If you're willing to use Gcc's c++0x extensions, it's a bit easier:

#include <cstdatomic>

std::atomic<Data*> rt_data;

Data* swap_data( Data* new_data )
{
   Data* old_data = rt_data.exchange(new_data);
   assert( old_data != new_data );
   return old_data;
}

void use_data( )
{
   Data* local = rt_data.load();
   /* ... */
}