Case Study: LineRate's C++98 to C++11 Migration

Have you heard about C++11 and the huge changes in the language?
Are you terrified to turn on the flag in your compiler?
Are your developers clamouring for modern language features?

The LineRate team has been excited about the new C++11 features for a few years: lambdas, range-based for loops, move semantics, 'auto' keyword, etc... However, we haven't upgraded. We needed to upgrade Boost and GCC to C++11 compatible versions, along with a few other libraries. Now that we have upgraded, we don't know what level of effort and amount of time are required to do the upgrade. With such a large language change there are many oppotunities for things to go wrong and maybe we will need to make extensive code changes. LineRate has extensive regression testing, so we are not overly concerned about introducing new bugs. Our regression test suite runs nightly and has good code coverage. Let's dive in...

Build System Changes

For better or worse, our build system is SCons. It is simple to add the compiler flag to enable C++11 building, just add '-std=c++11' to the CXXFLAGS and off you go. Since this is going to be an experiment for the time being, we'll make it optional. Easy enough, add a command-line switch to SCons to determine if this flag should be added or not. The same can be done with Make by adding a new target. Our changes are something like this:

cpp11 = int(ARGUMENTS.get('cpp11', 0))
cpp11Flags = []
if (1 == cpp11):
    cpp11Flags.append('-std=c++11');
    print "C++11 Mode Enabled"
env.AppendUnique(CXXFLAGS = cpp11Flags)

This works. We can see in the SCons jobs that it is passing '-std=c++11' to gcc:

g++48 -o some_code.o -c -O2 -Wall -Werror -std=c++11 -I/some/includes some_code.cc

Now come the build failures! Time to port the code to the new standard.

Product and Unit Test Code Changes

Our codebase is made up of numerous languages. The C++ portion of our codebase is approximately 200k SLOC according to sloccount, 135k SLOC of product code and 65k SLOC of unit tests. Typically the product code is highly scrutinized for adhering to our coding style guidelines while rules are a little looser in the unit tests. This resulted in many source changes in the unit tests attributed to one thing (code change #1 below).

Brief aside on the topic of coding style... We use Google's coding style guide with a few small modifications. These have been well considered and help avoid some of the common pitfalls in C++. Where the C++98 standard was lacking, we often use Boost. These things include heavy use of boost::shared_ptr<T>, boost::bind(), and boost::unordered_map<K,V>.

The code changes needed were minimal. I attribute this to our extensive use of Boost. Their libraries handle the difference in compiler capabilities automatically. All of the changes fell into 6 categories:

(ERROR) boost::shared_ptr<T>::~~operator==~~operator bool() is now explicit
(ERROR) std::make_pair<T,U> args must be rvalue references
(WARNING) std::auto_ptr<T> is deprecated in favor of std::unique_ptr<T>
(ERROR) std::unique_ptr<T> does not support copy assignment
(ERROR) nullptr keyword is required when initializing with 0
(ERROR) constexpr class variables instead of const

Let's explore each one of these independently.

1. boost::shared_ptr<T>::operator bool() is now explicit

Boost changed the shared_ptr<T>'s boolean operator. Conversion to bool is now explicit in C++11. This means that old valid code now becomes a compilation error, so it is easy to find and fix:

boost::shared_ptr<T> myT;
if (myT) { doSomething(); } // error!
if (myT != nullptr) { doSomething(); } // OK
if (!myT.empty()) { doSomething(); } // OK

bool MyClass::isNotNull() const { return myT_; } // error!
bool MyClass::isNotNull() const { return myT_ != nullptr; } // OK
bool MyClass::isNotNull() const { return !myT_.empty(); } // OK

2. std::make_pair<T,U> args must be rvalue references

In C++98, std::make_pair<T,U> takes references and then copies from them into the new std::pair<T,U>. This function has been removed and rvalue references must be provided. The rvalue refs end up getting moved from, so be careful!

Depending on the compiler, your options, and your STL implementation, this one might work:

std::pair<int, int> pairMaker(int v1, int v2) {
    return std::make_pair(v1, v2); // error!
}

The following will work:

std::pair<int, int> pairMaker(int v1, int v2) {
    int a{v1}, b{v2}; 
    return std::make_pair(a, b); // OK
}

std::pair<int, int> pairMaker(int v1, int v2) {
    return std::make_pair(std::move(v1), std::move(v2)); // OK
}

3. std::auto_ptr<T> is deprecated in favor of std::unique_ptr<T>

There were many evils with std::auto_ptr<T>, but it was the only available smart pointer which could release. In the modern day, std::unique_ptr<T> can also release. A simple global search and replace will do the trick. Except for the code change #4. See below.

This one does not pertain to you if you are not compiling with -Werror flag enabled. But, if you are not then perhaps you should. There are plenty of arguments for it.

4. std::unique_ptr<T> does not support copy assignment

In the bad old days of std::auto_ptr<T> there existed an assignment operator to transfer ownership from one auto_ptr to another. This is one of the many evils of std::auto_ptr<T>. I performed a complete search and replace of std::auto_ptr with std::unique_ptr in our codebase because of #3 above. A few places were relying on the assignment behavior, but the compiler immediately flagged it as an error. The fix is easy.

std::auto_ptr<int> p(new int); // deprecated!
std::auto_ptr<int> q = p; // deprecated!

std::unique_ptr<int> up(new int);
std::unique_ptr<int> q;
q = p; // error!
q.reset(p.release()); // OK

5. nullptr keyword is required when initializing with 0

The new nullptr keyword has a different type than NULL or 0. Its type is std::nullptr_t. It is preferable because it provides stronger type checking by the compiler. Fortunately, it is not an error since this is used all over our codebase. In some cases it does provide a warning. Fixing those are simple:

class MyClass {
  public:
    MyClass() : myPtr_{0} {} // error!
    MyClass() : myPtr_{nullptr} {} // OK
  private:
    std::unique_ptr<int> myPtr_;
};

6. constexpr class variables instead of const

~~Variables declared inside a class as const need to be constexpr if they are non-integral types.~~ GCC's in-class initializer of static members extension is now part of the standard, but as constexpr. The compile will catch these nicely for you.

class MyClass {
    static const double foo = 0.0; // error!
    static constexpr double foo = 0.0; // OK
};

Other

The only other issue was found in the unit tests. The test counts the number of copies and the number of destructions are called. It expected 3, but only 2 happened. There was a failure due to the number of copies. MOVE! One of the copies became a move. Getting moves is a reason we are migrating to C++11. It is great to know that this actually happened without any other changes.

Staged Migration Strategy

Now that we established that the migration is straight-forward and works (except for unknown regression bugs), we still need to address the issue of making the code build with or without the conditional flag being passed to the compiler by the build system. This software is for a software-based network appliance and we need to run tests extensively. It would be great to put these changes in, but for now we need to only enable this conditionally. Some of the changes such as usage of the 'nullptr' keyword are not available for C++98. Boost provides some macros for just this situation.

Macro Magic and Static Asserts

Macros are not very C++ish, but it is what we have.

An extensive list of macros are available from Boost which will use information such as the version and vendor of the compiler and STL to determine what C++11 (and other) features are available. To get around the 'const vs constexpr' problem listed about in #6, use the BOOST_CONSTEXPR_OR_CONST macro. For some of the others you may need to write some of your own. Here is the set that I am using:

#ifdef BOOST_NO_CXX11_NULLPTR
#define LRS_NULLPTR NULL
#else
#define LRS_NULLPTR nullptr
#endif

#ifdef BOOST_NO_CXX11_RVALUE_REFERENCES
#define LRS_MOVE
#else
#define LRS_MOVE std::move
#endif

#ifdef BOOST_NO_CXX11_SMART_PTR
#define LRS_UNIQUE_PTR std::auto_ptr
#else
#define LRS_UNIQUE_PTR std::unique_ptr
#endif

Another thing to consider is adding static asserts. These are executed at compile-time and do not change the run-time behavior of the code. It is non-trivial to reason about the copy/move default functions being generated by the compiler in C++11. The developer needs to take into account the copy/move semantics of not only the class being written, but also all of the member variables within the class. Always defining these 5 static asserts below will help. When a code change happens which changes one the compilation will fail and produce an easy to read compilation error. This forces the author to evaluate and either change the class or change the static assertion to align with the new behavior. Don't forget to wrap it in Boost's macro!

#ifdef BOOST_HAS_STATIC_ASSERT
static_assert(std::is_default_constructible<MyClass>::value,
              "MyClass is not default constructible.");
static_assert(std::is_copy_constructible<MyClass>::value,
              "MyClass is not copy constructible.");
static_assert(std::is_copy_assignable<MyClass>::value,
              "MyClass is not copy assignable.");
static_assert(std::is_nothrow_move_constructible<MyClass>::value,
              "MyClass is not no-throw move constructible.");
static_assert(std::is_nothrow_move_assignable<MyClass>::value,
              "MyClass is not no-throw move assignable.");
#endif

Summary

The migration to C++ has been a breeze. The use of Boost and Boost's macros were a large benefit. Minimal code changes were required and the compiler was able to point them out. Regression analysis has yet to be run, but hopefully there are no unexpected results. While we're validating the stability and performance of the C++11 version of our software, we can keep developing new features in C++98. After these changes, switching back and forth between C++98 and C++11 is as easy as a SCons option.

Total SLOC changed of the roughly 200k of C++:

.tg {border-collapse:collapse;border-spacing:0;border-color:#aaa;border-width:1px;border-style:solid;} .tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#aaa;color:#333;background-color:#fff;} .tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#aaa;color:#fff;background-color:#f38630;} .tg .tg-s6z2{text-align:center} .tg .tg-lyaj{background-color:#FCFBE3;text-align:center} .tg .tg-z2zr{background-color:#FCFBE3}