It’s still not quite where I want it to be, but I felt that at the very least, it deserved a proper status page. So here it is. A brand new page on my blog. As I’m still rather busy, I can’t promise that much will happen with the library over the summer, but when something significant happens, that page will be the first to know.

I have been working on it on and off for the last couple of months (although the pace slowed down a lot when I started at my new job), and it is actually in pretty good shape. To the best of my knowledge (it is difficult to be sure when it comes to multithreaded code), there are no race conditions or other nasty bugs. It is basically feature complete, although I’m still working on improving the performance of some of them.

And the documentation is, of course, totally missing. I’ve got everything I wrote for my masters thesis, but a major rewrite is in order since the focus is now very different. Another starting point is this, but obviously, there’s quite a bit of work left on the documentation as well.

But it came up in a discussion with a bunch of quite clever programmers who expressed interest in it, and since it’s always been my intention to release it some day, and since I’m actually getting fairly close to being satisfied with its state, well, I might as well put this up here for the world to see:

DikuSTM lives!

Well, my test case for this feature now takes ages to run. As I mentioned previously, a simple transaction modifying two integer variables under heavy contention can pull off almost two million transactions per second on my laptop.

My new test, in which each thread takes four variables and alternates between modifying two of them and reading the other two, runs perhaps ten thousand (!) times slower.

Of course I have several leads on how to fix this. The problem is largely all the performance-related “extras” I’ve been leaving out. For example, if a transaction fails to acquire a variable it needs, it simply aborts and immediately retries. In many cases, a better approach would be to block the thread, waiting for that variable to actually become available.

There are several other cases where I have a similar problem: I have to choose between delaying the thread for a moment with sleep() before attempting to continue, blocking it until some condition is true, or aborting the transaction entirely and starting over from scratch. At the moment, I generally just pick the easiest solutions (typically abort, and occasionally call sleep() a few times before we resort to that. Again, implementing some actual meaningful policies here would make a big difference. And tweaking these policies should help still more.

Another problem is that currently, I do not enforce a consistent global order when acquiring objects during a commit. This means I risk livelocks, again causing excessive rollbacks when multiple threads are competing over access to the same variables.

So I’m still optimistic. It should be possible to get performance back on track. But man, it’s depressing watching performance plummet like this.

Edit
And an update. After poking around a bit, it turned out that most of the time was being spent sleeping. When a transaction attempts to commit, if it can not acquire all the all the variables it needs, it retries a few times with a short delay (a couple of milliseconds) in between. If it doesn’t succeed after a few tries, it rolls back the entire transaction and starts over.

It turned out that these few, short sleep() calls brought CPU utilization down to something like 0.01%, and totally destroyed performance. Simply turning the sleep() call into a no-op brought me back to something more or less reasonable. I still need to improve on the above shortcomings, but now at least I can run my tests in less than an hour.

In the following, I’ll show a slightly modified version of one of my test cases. It shows how to call and use my library, from the user’s point of view. The test spawns a number of threads, which each wait on a barrier (because if one thread was allowed to run while another was being constructed, I’d get fewer concurrent transactions, and my test would be less likely to uncover race conditions), and then perform a fixed number of transactions. In each transaction, two transactional variables are opened for writing, one is decremented and the other incremented. If the sum of these variables is nonzero in any iteration, the thread registers a failure.

And if, after all the threads have terminated, the value of each of these variables is not what was expected, a failure is registered as well.

So from a testing point of view, there should be plenty of opportunity for things to go wrong. Just a single small race condition somewhere, and one of the many millions of reads would be inconsistent and the test would fail.

#include <stm.hpp> // my STM library

#include <boost/test/unit_test.hpp> // Boost.Test is used to supply a unit-testing framework
#include <boost/thread.hpp> // Boost.Thread is used as a threading API

// define the number of threads to run, and the number of iterations for each
enum { thread_count = 8, iterations = 200000 }; 

// The following are transactional variables. The shared template ensures that the contained value
// can only be accessed as part of a transaction, and provides the necessary metadata
// for checking validity and consistency
// Two such integers are created, both initialized to zero
stm::shared<int> val1(0);
stm::shared<int> val2(0); 

BOOST_AUTO_TEST_SUITE( threads ) // define a test suite named "threads"

// this function defines the body of our transaction. 
// We're passed a transaction, which can be used to open any "shared" variables
bool tx_func(stm::transaction& tx){
    // open both variables for writing
    int& a = val1.open_rw(tx);
    int& b = val2.open_rw(tx);

    // modify the variables freely
    --a;
    ++b;

    return a + b == 0; // Our transaction returns a bool. Other return types (or void) are also supported
}

// this class defines a thread. operator() is called as the thread's entry point
struct thread_functor{
    // In the constructor, the thread object is given a barrier it can synchronize on,
    // and a reference where it can write the its result (success/failure)
    thread_functor(boost::barrier& bar, int& res) : bar(bar), res(res) {}

    void operator()(){
        // when the thread is first created, we wait for the barrier
        // This ensures that no transactions are running until all threads have been constructed
        bar.wait(); 
        for (int i = 0; i < iterations; ++i){
            // for each iteration, pass our transaction function to the "atomic" function, which executes it atomically.
            // To get a non-void return type, we have to specify the template parameter explicitly 
            // (this can be avoided in C++0x using the return_of template to deduce the return type implicitly)

            // depending on the return value of the transaction, write success or failure back
            res = (res != 0) && stm::atomic<bool>(tx_func) ? 1 : 0;
        }
    }

    boost::barrier& bar;
    int& res;
};

// another transaction, to be executed after our helper threads terminate, 
// to verify that the right number of modifications have occurred
// note that here variables are opened for reading only
void verify(stm::transaction& tx) {
    const int& a = val1.open_r(tx);
    const int& b = val2.open_r(tx);

    BOOST_CHECK_EQUAL(-a, thread_count * iterations);
    BOOST_CHECK_EQUAL(b, thread_count * iterations);
}

// finally, we get to our test case itself
BOOST_AUTO_TEST_CASE ( short_concurrent_transactions )
{
    boost::barrier bar(thread_count);
    boost::thread_group gr;
    int res[thread_count]; // array of results
    // set all the results to an initial true/1 value (since each iteration "and"'s it together with the current result
    std::fill(res, res+thread_count, 1); 

    for (int i = 0; i < thread_count; ++i){
        gr.create_thread(thread_functor(bar, res[i])); // create the threads, passing the necessary parameters to each
    }

    gr.join_all(); // wait for all threads to terminate

    // verify that each thread return success
    for (int i = 0; i < thread_count; ++i){
        BOOST_CHECK_EQUAL(res[i], 1);
    }

    // run a final transaction to access both variables and check their final values
    stm::atomic(verify);
}

BOOST_AUTO_TEST_SUITE_END()

In the above, I used a function object to represent threads, and a regular function to represent transactions. Of course in both cases, either would work — a function object would potentially be more efficient as it is easier for the compiler to inline, but I used a function for brevity.

In C++0x, of course, lambdas could also have been used in both cases.

One of my design goals has been to make basic usage as simple and intuitive as possible, and I think I’ve succeeded so far. Any C++ programmer who is familiar with the STL algorithms or the Boost libraries, should find my library’s interface very straightforward. Note especially that all the transaction “magic”, of verifying validity and retrying transactions as needed, is completely invisible to the user. You simply define a function expressing what your transaction should do, and pass it to the atomic function.

In its current version, this test is able to execute around 1,800,000 transactions per second on my Core Duo 2GHz laptop. (Of course, with transactions as small as these, opening only two variables each, performance is a lot better than it would be in real-world transactions.

So that’s it for now. Of course I’ve got a few more user-facing features in the pipeline¹, and a lot of backend changes, but the basic functionality is there, and I’m pretty happy with it so far.

The connection works, but our router and the ethernet cables we ordered were apparently lost in the mail… So at the moment, we can have exactly one computer online at a time, and I pretty much have to sit right next to the adsl modem for the cable wehave to reach. Silly and annoying.

But overall, things are getting back to normal, and I’ve even had time to work on my thesis some. I’m going to put up a post soon with some code examples and such, but for now, the short version is that it works. I now have a prototype of the core functionality — a lot of importants features aren’t implemented yet (such as support for nested transactions), but the core works. I can start transactions, read and modify variables, and the system then attempts to commit, validates the modifications and if it fails, the transaction is rolled back and restarted.

Since the entire project is meant to take 6 months, and I have just over 3 months left, I’m pretty happy with this. Now that I’ve got a stable and functioning base to build on, future upgrades should go a lot faster.

It has really underlined to me just how deeply entrenched most Java, C and C++ programmers are in the imperative mindset. When we want to perform a transaction, we’ll need to execute something like the following pseudocode:

for (;;) {
  tx = new Transaction();
  try {
    // ... fill in body of the transaction ...
    tx.Validate();
    tx.Commit()
  }
  catch (InvalidTxException){
    tx.Rollback();
  }
}

And several STM systems actually require the user to write the above whenever a transaction is to be executed.

Some implementers have realized that this is both verbose, messy and error-prone. So they try to hide it behind a macro.

#define BEGIN_ATOMIC \
  for (;;) { \
    tx = new Transaction();\
    try { \

#define END_ATOMIC \
      tx.Validate();
      tx.Commit()
    }
    catch (InvalidTxException){
      tx.Rollback();
    }
  }

and now a transaction looks like this instead, from the user’s point of view:

BEGIN_ATOMIC
// ... fill in body of the transaction ...
END_ATOMIC

Which is obviously shorter and requires less duplicate code. On the other hand, it relies on macros. Eeeew, yuck!

In Java, most implementers have tried to extend the language, to add a language-level atomic statement, allowing code like this:

atomic {
  // ... fill in body of the transaction ...
}

And yes, this obviously works too, and has the potential to integrate much more closely with the compiler and type-checker. But it’s also a very imperative kind of thinking. “add another type of scope inside whichever function wants to perform the transaction”.

What astonishes me is that no one (apart from Peyton Jones, who implemented Haskell STM) have realized that what we really want is nothing more than a higher-order function. A function which implements the transaction infrastructure, and then calls a user-defined function.

What we’d like to do, ideally, is to define a function atomic, as in the following pseudocode:

T atomic(f : Tx -> T){
  for (;;) {
    tx = new Transaction();
    try {
      T ret = f(tx);
      tx.Validate();
      tx.Commit();
      return ret;
    }
    catch (InvalidTxException){
      tx.Rollback();
    }
  }
}

Of course, if we wanted to be really functional, we could use recursion instead of the loop, but let’s not get carried away. Ultimately, what we want is just to hide all the messy infrastructure of a transaction from the user.

The above declares a function atomic which takes as its parameter a function which, given a transaction (type Tx), returns type T.

atomic implements the messy transaction loop, and simply calls the user-supplied function inside it.

And presto, the user can now create a transaction like this:

int myTx(Tx tx) {
  // do whatever
}
int result = atomic(myTx);

Or, in a language which supports lambda expressions,

int result = atomic((tx) => {/* do whatever */ });

Bit nicer than messing around with user-implemented loops or macros, isn’t it? Of course, I’ve so far sought refuge within the peaceful realms of pseudocode, where anything is possible. We obviously can’t do this in Java or C++, can we?

No, and yes, in that order. In Java, I agree, we’re just screwed. Of course, we could just pass in an object instead of a function, and if it implements some ICallableTransactionBody interface, atomic could call its RunTx() method. That would work, but it’s not quite as elegant.

In C++, though, the situation is different. A large part of the standard library (all the STL algorithms) relies on higher-order functions. .

In C++, we can define atomic as follows:

template <typename Func>
void atomic(Func f){
  for (;;) {
    tx = new Transaction();
    try {
      f(tx);
      tx.Validate();
      tx.Commit();
    }
    catch (InvalidTxException){
      tx.Rollback();
    }
  }
}

First, I should mention that I cheated. I removed the return value. atomic now returns void. The reason is that it’s not quite trivial to infer the return type of a function in C++. TR1 does introduce a function std::result_of but it doesn’t work in every case. C++0x will fix this once and for all, by providing the necessary language features to implement it. But until then, we could hack around it, either by requiring the user to explicitly specify the return type (atomic<int>(myTx)), or by using result_of and simply telling the user to avoid the cases where it doesn’t work. But to keep the above code example simple, I simply removed the return type instead. But it can be solved, and my prototype does it.

Now, on with the show. We have now defined our atomic function. How do we call it?

In one of three ways, in decreasing order of familiarity:

First, with a function pointer. Hopefully every C++ programmer knows about these:

void myTx(Tx tx) {... }
atomic(myTx);

Fairly clean and simple. One downside is that because the atomic function is just passed a function pointer, the compiler may not be able to inline the call to myTx. It is doubtful that it’d make much difference here, but it is nevertheless a valid concern.

The second approach fixes this, as well as enabling some more flexibility: Functors, or function objects, which every C++ programmer should know about, but unfortunately, not everyone does:

struct myTx {
  void operator()(Tx tx) {...}
};
atomic(myTx());

Now we’re passing in an object of type myTx, which means that the compiler knows exactly which function to call: myTx::operator()(Tx). So it can inline this trivially, eliminating the above performance concern.

Perhaps more importantly, our function can now hold state. It is an object, so it can be given a constructor and destructor. It can be given other member functions to allow the user to interact with it. It has become a lot more versatile.

Finally, we can wait for C++0x, which introduces lambda expressions:

atomic([](Tx tx) { ...});

And what’s noteworthy is that the same atomic definition works in all three cases! So the user is actually given the choice of which method to use. Our STM library supports them all.

Isn’t this cleaner than the original approach, of requiring the user to implement the whole loop himself? So why have no one used it before?

Ultimately, I can think of two possible reasons:

• the writers, usually very clever researchers and academics, were not familiar with higher-order functions, or with functional programming in general, or
• the writers did not know that this was possible in C++

I suppose the second option is the most forgiveable. C++ is a big messy language, and not everyone are familiar with every corner of it. That’s fair enough, but even then, most of this could have been done in C too, by using function pointers. It’s not that esoteric, is it? But I sincerely hope it is not the first case. I could understand if Joe Coder at Insignificant Software Inc. had never heard of functional programming, but most of these are computer science professors!

Incidentally, this should also provide a nice example for when an imperative programmer asks “how would it benefit me to learn a functional language?”

Here we have a clear example of how using a very simple functional technique can significantly clean up the code and make it much more robust¹, even in an imperative language like C++.

Of course it is still early in the process, and while I’ve got some ideas for my implementation, and have started prototyping parts of it, most of the time has been spent catching up on all the existing work in the field.

Little did I know when I accepted the subject that there’d been written that many papers about it over the last decade. Phew… A commenter asked me for the references I found useful, so here’s an incomplete list:

Tranactional Memory Bibliography: This site contains an up-to-date and extremely comprehensive list of references to everything written aobut STM. Just the sheer number of links out from that site is depressing, but it’s obviously a wonderful place to get an overview of what’s available.

Transactional Memory: I bought this book after noticing that almost every paper on STM referenced it. And it was money well spent. The book is well written, explains the subject clearly, and covers pretty much everything I wanted to know. It is split into three main chapters (not counting the introduction):

The first chapter discusses TM purely from a user’s perspective. What should the API look like, what do we, as programmers want or need from a TM implementation, what should the semantics be, and what extensions might be useful? It explains a number of important concepts, such as weak/strong isolation, serializability and linearizability, direct and deferred update and many others. But mostly, this chapter is fantastic as a discussion of what TM research should aim towards. Without getting bogged down in implementation details, it focuses simply on how we’d like it to work (and on the many open questions where we still don’t know which semantics would make sense or be most useful).

The second chapter is useful as a shortcut, allowing newcomers such as me to catch up on the last decade’s research. It summarizes all the important papers on STM, from the earliest precursors dating back to the 70’s, up to 2006 when the book was published. The various implementations and results are briefly described, in enough detail for me to understand the main ideas presented, and determine whether to look up the paper in question and read it in full.

The third chapter gives a similar treatment to Hardware Transactional Memory, and since that is outside the scope of my project, I’ve skipped lightly over this for now, although I may just read it out of curiosity later on. In short, it’s a great book for anyone trying to find their way in the jungle of TM papers.

Haskell STM: Simon Peyton Jones and others at Microsoft Research have done some truly impressive work on a STM implementation for Concurrent Haskell. While those of us working in a messy language like C++ can’t aspire for such a clean and elegant implementation, there are still many good ideas we can borrow.

And finally, a few individual papers I’ve gotten a lot out of:

Software Transactional Memory Should Not be Obstruction-Free: As STM grew out of research into lock-free datastructures and distributed systems, the early work was downright obsessed about making transactions nonblocking. It took this paper to take a step back and ask whether this requirement actually makes sense for the rest of us. And guess what? It doesn’t. It complicates the STM implementations, and prevents some very useful optimizations.

What Really Makes Transactions Faster followed up on the above, discussing a number of other ways in which the performance of STM systems could be improved.

And finally, the earlier thesis that drew me into the subject:

A Software Transactional Memory Library for C++: This is the first implementation I’ve seen which seriously targets C++. Of course many prior C++ implementations exist, but they’ve generally relied on either C or Java idioms, resulting in an uglier, more error-prone interface, and the inability to perform a number of optimizations. This paper describes a proper modern C++ implementation, relying on generic programming, template metaprogramming and all the other tricks in a C++ programmer’s toolbox.

But now it is official. I handed in all the paperwork (in six copies, what do they need all that paper for?), got the signature from my advisor and everything else yesterday. I have now officially started on my thesis. The subject?

Implementing a Software Transactional Memory (STM) library in C++0x. (Or rather, in C++, using whatever C++0x features current compilers have support for. In particular, I can see good uses of rvalue references and lambda expressions, which, luckily, the Visual C++ 2010 beta has support for.

And I am super psyched up about it. The subject seems a lot of fun. Of course I’ve got a lot of reading to do, catching up on all the existing research in the area (Two weeks ago, I had barely heard of STM before, so I can’t claim to be an expert on the field yet ;))

However, the C++0x aspect seems to be almost untouched in previous work¹, which is obviously nice. Feels good to be able to contribute something new.

My deadline is March 1st, 2010, almost exactly 6 months from now. I’m going to push myself to get a prototype up as early as possible. So the first few weeks are going to be busy. Once I get started properly, I’ll be able to vary the pace a bit more, taking more time out for the blog, work and everything else.