Macro Programming, Unit Testing

Macro programming is hell. Abuse the preprocessor and soon you’ll be joining the ranks of the fallen in the Turing tarpit.

The problem is compounded by the infamously unhelpful error messages of GCC. Today, I’ve spent a good part of an hour trying to decipher and debug this particularly mystifying one :

In file included from test_bitvector.c:14:0:
checkhelper.h:79:34: error: '#' is not followed by a macro parameter
 #define checkAssertXXInt(X, OP, Y) do { \
                                  ^

Check the source and see if you can find the problem. I’ll make your life really easy and promise you that the error is in this small snippet :

#define checkAssertXXInt(X, OP, Y) do { \
  unsigned __int128 _ck_x = (unsigned __int128) (X); \
  uintmax_t _ck_x_hi = (_ck_x >> 64); \
  uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
  unsigned __int128 _ck_y = (unsigned __int128) (Y); \
  uintmax_t _ck_y_hi = (_ck_y >> 64); \
  uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
  ck_assert_msg(_ck_x OP _ck_y, \
    "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX"#msg, #X#OP#Y, \
    #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo); \
} while (0)

Found out ? No ? First hint : although GCC is technically right and there is a problem in the line shown, it would be much more helpful to show another (symmetrical) error.

Still nothing ? There is no shame : the macro parameter msg used with the # operator in line 87 was not declared in line 79.

Now imagine ferreting the little troll, among a flurry of cascading errors, from a 150-line source that includes 5 other headers. Maddening enough for a Pirsig treatise.

* * *

Ironically, the error occurred whilst I prepared a common header for my unit testing modules.

As a scientist who mostly writes hundred-line scripts, this is the very first time I am designing unit tests. I was worried that the learning curve for a testing library would be steep, and so I was immediately sold by the no-frills proposition of Check, a unit test framework for C. It comes complete with a tutorial that — frankly — if you are an academic programer like me, tells you everything you need to know. (Although I find the diff notation of the examples less friendly than it could be).

It took me half an hour to understand Check, and then I was joyously writing and running my tests. Of course, just as soon, I was already wondering how it could be perfected :

  1. Have the assertion helper macros (e.g., ck_assert_int_eq) allow the printing of extra info, like ck_assert_msg does ;
  2. Solve a small bug in the helper macros : they show the inspected expressions by embedding them directly in the printfed string. The problem : what happens when the expression has a modulo operator (%) ? Exactly ! Printf takes it as an escape character, and catastrophe ensues ;
  3. Have helper macros for the 128-bit integers that GCC offers as an extension to the standard integers of C ;
  4. Solve an slight aesthetical problem, since I much prefer camelCase for the helper functions than snake_case (But I reserve UGLY_UPPER_SNAKE_CASE for preprocessor macros with UGLY_SECONDARY_EFFECTS, which you MUST_REMEMBER, because THEY_BITE !) ;
  5. Include automatically the main() function — which is very much the same for every test. I took the opportunity to make it parse the command-line to set the verbosity of the output.

Below you’ll find the fruits of those amendments. Or if you prefer, here’s a link for the source repo. It is released under the same LGPL 2.1 license as Check. No warranties : if you choose to use it, and your computer turns into an Antikytherean device,  or your code opens a portal to Baator, I am not liable.

/*
 Name        : checkhelper.h -- Helper header for Check unit test library
 Contributor : Eduardo A. do Valle Jr., 2014-01-27
 License     : LGPL 2.1 --- see http://check.sourceforge.net/COPYING.LESSER
 */

#ifndef CHECK_HELPER_H_
#define CHECK_HELPER_H_

#include <assert.h>
#include <check.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>


/* --- Alternative macros for unit testing --- */

#define checkAbort ck_abort

#define checkAssert ck_assert

#define checkAbortInfo ck_abort_msg

#define checkAssertInfo ck_assert_msg

#define checkAssertInt(X, OP, Y) do { \
  intmax_t _ck_x = (X); \
  intmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jd, '%s'==%jd", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertUInt(X, OP, Y) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%ju, '%s'==%ju", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertXInt(X, OP, Y) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX, '%s'==%jX", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertString(X, OP, Y) do { \
  const char* _ck_x = (X); \
  const char* _ck_y = (Y); \
  ck_assert_msg(0 OP strcmp(_ck_y, _ck_x), "Assertion '%s' failed: '%s'==\"%s\", '%s'==\"%s\"", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertIntInfo(X, OP, Y, msg, ...) do { \
  intmax_t _ck_x = (X); \
  intmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jd, '%s'==%jd. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertUIntInfo(X, OP, Y, msg, ...) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%ju, '%s'==%ju. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertXIntInfo(X, OP, Y, msg, ...) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX, '%s'==%jX. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertStringInfo(X, OP, Y, msg, ...) do { \
  const char* _ck_x = (X); \
  const char* _ck_y = (Y); \
  ck_assert_msg(0 OP strcmp(_ck_y, _ck_x), \
    "Assertion '%s' failed: '%s'==\"%s\", '%s'==\"%s\". "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)


#ifdef __SIZEOF_INT128__

    #define checkAssertXXInt(X, OP, Y) do { \
      unsigned __int128 _ck_x = (unsigned __int128) (X); \
      uintmax_t _ck_x_hi = (_ck_x >> 64); \
      uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
      unsigned __int128 _ck_y = (unsigned __int128) (Y); \
      uintmax_t _ck_y_hi = (_ck_y >> 64); \
      uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
      ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX", #X#OP#Y, #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo); \
    } while (0)

    #define checkAssertXXIntInfo(X, OP, Y, msg, ...) do { \
      unsigned __int128 _ck_x = (unsigned __int128) (X); \
      uintmax_t _ck_x_hi = (_ck_x >> 64); \
      uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
      unsigned __int128 _ck_y = (unsigned __int128) (Y); \
      uintmax_t _ck_y_hi = (_ck_y >> 64); \
      uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
      ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX. "#msg, #X#OP#Y, #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo, ## __VA_ARGS__); \
    } while (0)

#endif


/* --- Unless CHECK_HELPER_TEST_SUITE_NAME is defined with the name to use, testSuite() will be called to build the suite to run --- */

#ifndef CHECK_HELPER_TEST_SUITE_NAME
    #define CHECK_HELPER_TEST_SUITE_NAME testSuite
#endif


/* --- Unless CHECK_HELPER_MAIN_NAME is defined with the name to use, main() will be used --- */

#ifndef CHECK_HELPER_MAIN_NAME
    #define CHECK_HELPER_MAIN_NAME main
#endif


/* --- Provide a main() function by default, unless CHECK_HELPER_NO_MAIN is defined --- */

#ifndef CHECK_HELPER_NO_MAIN

    Suite * CHECK_HELPER_TEST_SUITE_NAME(void);  // declaration

    int CHECK_HELPER_MAIN_NAME(int argc, char **argv) {

        enum print_output mode = CK_NORMAL;

        if      (argc>1 && (strcmp(argv[1], "-v")==0 || strcmp(argv[1], "--verbose")==0)) {
            mode = CK_VERBOSE;
        }
        else if (argc>1 && (strcmp(argv[1], "-q")==0 || strcmp(argv[1], "--quiet")==0)) {
            mode = CK_SILENT;
        }
        else if (argc>1 && (strcmp(argv[1], "-m")==0 || strcmp(argv[1], "--minimal")==0)) {
            mode = CK_MINIMAL;
        }
        else if (argc>1 && (strcmp(argv[1], "-n")==0 || strcmp(argv[1], "--normal")==0)) {
            mode = CK_NORMAL;
        }
        else if (argc>1 && (strcmp(argv[1], "-e")==0 || strcmp(argv[1], "--environment")==0)) {
            mode = CK_ENV;
        }
        else if (argc>1) {
            #ifndef CHECK_HELPER_USAGE_STRING
            printf("usage: [test_executable] [verbosity flags: -(-q)uiet | -(-m)inimal | -(-n)ormal (default) | -(-v)erbose | -(-e)nvironment ]\n"
                   "       if -e or --environment get verbosity from environment variable CK_VERBOSITY (values: silent minimal normal verbose)\n");
            #else
            printf(CHECK_HELPER_USAGE_STRING);
            #endif
            return 1;
        }

        Suite *suite = CHECK_HELPER_TEST_SUITE_NAME();

        SRunner *suiteRunner = srunner_create(suite);
        srunner_run_all(suiteRunner, mode);
        int numberFailed = srunner_ntests_failed(suiteRunner);
        srunner_free(suiteRunner);

        return (numberFailed == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
    }

#endif

#endif

Wow ! Much Homebrew. Very Numpy. So Scipy. Such OpenCV

The first time I tried to install NumPy+SciPy in my Mac, it turned into a Kafkaesque nightmare, out of which I only managed to surface due to luck and grit. (Only to have, a few weeks later, a system update breaking my MacPorts and sending everything back to hell.)

The second time around, I traded freedom for comfort, and went with Enthought Python Distribution (now Enthought Canopy).  EPD came with an impressive list of available packages, and, more importantly : it just worked. It was also generously available at no fee for academic use, an offer from which I’ve profited.

Recently though, I became a latecomer to Homebrew, enticed by their taglines (‘The missing package manager of OS X’, ‘MacPorts driving you to drink ? Try Homebrew !’) and by their oneliner installation procedure (look for ‘Install Homebrew’ at their homepage).

So far, I am incredibly impressed — I’ve done fresh installations of Python, Nose, NumPy, OpenCV, GCC (!), SciPy, Bottleneck, wxPython and PIL. All went smoothly, installing and testing without smoke. My command-line history reveals just how easy it was :

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

brew install python

/usr/local/bin/pip-2.7 install nose

/usr/local/bin/pip-2.7 install numpy

brew install opencv

brew install gcc49

brew install scipy

/usr/local/bin/pip-2.7 install bottleneck

brew install wxwidgets

/usr/local/bin/pip install pil

The only pitfall (if it may even be called so) is that some Python packages prefer the homebrew installer, and some prefer pip — but quick error and trial works just fine to find out.

Often homebrew installer will discreetly guide you through the process, like when I asked ‘brew install wxpython’, and it told me that there was no such package, but that  ‘wxwidgets’ already came with the wxPython bindings. That kind of gentle bending of Unix philosophy, on behalf of preserving the user sanity, never fails to win my respect.

Now : maybe homebrew is running so smoothly only because I have EPD already installed in this machine, all exoteric dependences having been previously solved. I also had Xcode fully installed and operational, a requirement for most interesting tools working on OS X at all. Remark also that I am still running Mountain Lion.

Homebrew’s express requirements seem to be quite modest, however : the Command-line Tools for Xcode, and a bash or zsh-compatible shell (the default terminal is fine). Additionally, it resides in a branch independent from EPD, so it probably can’t cound on the latter’s dependences. In a few weeks, I intend to do a fresh installation of Mavericks on this machine, and we will know for sure.