Macro Programming, Unit Testing

Macro programming is hell. Abuse the preprocessor and soon you’ll be joining the ranks of the fallen in the Turing tarpit.

The problem is compounded by the infamously unhelpful error messages of GCC. Today, I’ve spent a good part of an hour trying to decipher and debug this particularly mystifying one :

In file included from test_bitvector.c:14:0:
checkhelper.h:79:34: error: '#' is not followed by a macro parameter
 #define checkAssertXXInt(X, OP, Y) do { \
                                  ^

Check the source and see if you can find the problem. I’ll make your life really easy and promise you that the error is in this small snippet :

#define checkAssertXXInt(X, OP, Y) do { \
  unsigned __int128 _ck_x = (unsigned __int128) (X); \
  uintmax_t _ck_x_hi = (_ck_x >> 64); \
  uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
  unsigned __int128 _ck_y = (unsigned __int128) (Y); \
  uintmax_t _ck_y_hi = (_ck_y >> 64); \
  uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
  ck_assert_msg(_ck_x OP _ck_y, \
    "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX"#msg, #X#OP#Y, \
    #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo); \
} while (0)

Found out ? No ? First hint : although GCC is technically right and there is a problem in the line shown, it would be much more helpful to show another (symmetrical) error.

Still nothing ? There is no shame : the macro parameter msg used with the # operator in line 87 was not declared in line 79.

Now imagine ferreting the little troll, among a flurry of cascading errors, from a 150-line source that includes 5 other headers. Maddening enough for a Pirsig treatise.

* * *

Ironically, the error occurred whilst I prepared a common header for my unit testing modules.

As a scientist who mostly writes hundred-line scripts, this is the very first time I am designing unit tests. I was worried that the learning curve for a testing library would be steep, and so I was immediately sold by the no-frills proposition of Check, a unit test framework for C. It comes complete with a tutorial that — frankly — if you are an academic programer like me, tells you everything you need to know. (Although I find the diff notation of the examples less friendly than it could be).

It took me half an hour to understand Check, and then I was joyously writing and running my tests. Of course, just as soon, I was already wondering how it could be perfected :

  1. Have the assertion helper macros (e.g., ck_assert_int_eq) allow the printing of extra info, like ck_assert_msg does ;
  2. Solve a small bug in the helper macros : they show the inspected expressions by embedding them directly in the printfed string. The problem : what happens when the expression has a modulo operator (%) ? Exactly ! Printf takes it as an escape character, and catastrophe ensues ;
  3. Have helper macros for the 128-bit integers that GCC offers as an extension to the standard integers of C ;
  4. Solve an slight aesthetical problem, since I much prefer camelCase for the helper functions than snake_case (But I reserve UGLY_UPPER_SNAKE_CASE for preprocessor macros with UGLY_SECONDARY_EFFECTS, which you MUST_REMEMBER, because THEY_BITE !) ;
  5. Include automatically the main() function — which is very much the same for every test. I took the opportunity to make it parse the command-line to set the verbosity of the output.

Below you’ll find the fruits of those amendments. Or if you prefer, here’s a link for the source repo. It is released under the same LGPL 2.1 license as Check. No warranties : if you choose to use it, and your computer turns into an Antikytherean device,  or your code opens a portal to Baator, I am not liable.

/*
 Name        : checkhelper.h -- Helper header for Check unit test library
 Contributor : Eduardo A. do Valle Jr., 2014-01-27
 License     : LGPL 2.1 --- see http://check.sourceforge.net/COPYING.LESSER
 */

#ifndef CHECK_HELPER_H_
#define CHECK_HELPER_H_

#include <assert.h>
#include <check.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>


/* --- Alternative macros for unit testing --- */

#define checkAbort ck_abort

#define checkAssert ck_assert

#define checkAbortInfo ck_abort_msg

#define checkAssertInfo ck_assert_msg

#define checkAssertInt(X, OP, Y) do { \
  intmax_t _ck_x = (X); \
  intmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jd, '%s'==%jd", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertUInt(X, OP, Y) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%ju, '%s'==%ju", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertXInt(X, OP, Y) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX, '%s'==%jX", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertString(X, OP, Y) do { \
  const char* _ck_x = (X); \
  const char* _ck_y = (Y); \
  ck_assert_msg(0 OP strcmp(_ck_y, _ck_x), "Assertion '%s' failed: '%s'==\"%s\", '%s'==\"%s\"", #X#OP#Y, #X, _ck_x, #Y, _ck_y); \
} while (0)

#define checkAssertIntInfo(X, OP, Y, msg, ...) do { \
  intmax_t _ck_x = (X); \
  intmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jd, '%s'==%jd. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertUIntInfo(X, OP, Y, msg, ...) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%ju, '%s'==%ju. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertXIntInfo(X, OP, Y, msg, ...) do { \
  uintmax_t _ck_x = (X); \
  uintmax_t _ck_y = (Y); \
  ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX, '%s'==%jX. "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)

#define checkAssertStringInfo(X, OP, Y, msg, ...) do { \
  const char* _ck_x = (X); \
  const char* _ck_y = (Y); \
  ck_assert_msg(0 OP strcmp(_ck_y, _ck_x), \
    "Assertion '%s' failed: '%s'==\"%s\", '%s'==\"%s\". "#msg, #X#OP#Y, #X, _ck_x, #Y, _ck_y, ## __VA_ARGS__); \
} while (0)


#ifdef __SIZEOF_INT128__

    #define checkAssertXXInt(X, OP, Y) do { \
      unsigned __int128 _ck_x = (unsigned __int128) (X); \
      uintmax_t _ck_x_hi = (_ck_x >> 64); \
      uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
      unsigned __int128 _ck_y = (unsigned __int128) (Y); \
      uintmax_t _ck_y_hi = (_ck_y >> 64); \
      uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
      ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX", #X#OP#Y, #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo); \
    } while (0)

    #define checkAssertXXIntInfo(X, OP, Y, msg, ...) do { \
      unsigned __int128 _ck_x = (unsigned __int128) (X); \
      uintmax_t _ck_x_hi = (_ck_x >> 64); \
      uintmax_t _ck_x_lo = (_ck_x & 0xFFFFFFFFFFFFFFFF); \
      unsigned __int128 _ck_y = (unsigned __int128) (Y); \
      uintmax_t _ck_y_hi = (_ck_y >> 64); \
      uintmax_t _ck_y_lo = (_ck_y & 0xFFFFFFFFFFFFFFFF); \
      ck_assert_msg(_ck_x OP _ck_y, "Assertion '%s' failed: '%s'==%jX.%016jX, '%s'==%jX.%016jX. "#msg, #X#OP#Y, #X, _ck_x_hi, _ck_x_lo, #Y, _ck_y_hi, _ck_y_lo, ## __VA_ARGS__); \
    } while (0)

#endif


/* --- Unless CHECK_HELPER_TEST_SUITE_NAME is defined with the name to use, testSuite() will be called to build the suite to run --- */

#ifndef CHECK_HELPER_TEST_SUITE_NAME
    #define CHECK_HELPER_TEST_SUITE_NAME testSuite
#endif


/* --- Unless CHECK_HELPER_MAIN_NAME is defined with the name to use, main() will be used --- */

#ifndef CHECK_HELPER_MAIN_NAME
    #define CHECK_HELPER_MAIN_NAME main
#endif


/* --- Provide a main() function by default, unless CHECK_HELPER_NO_MAIN is defined --- */

#ifndef CHECK_HELPER_NO_MAIN

    Suite * CHECK_HELPER_TEST_SUITE_NAME(void);  // declaration

    int CHECK_HELPER_MAIN_NAME(int argc, char **argv) {

        enum print_output mode = CK_NORMAL;

        if      (argc>1 && (strcmp(argv[1], "-v")==0 || strcmp(argv[1], "--verbose")==0)) {
            mode = CK_VERBOSE;
        }
        else if (argc>1 && (strcmp(argv[1], "-q")==0 || strcmp(argv[1], "--quiet")==0)) {
            mode = CK_SILENT;
        }
        else if (argc>1 && (strcmp(argv[1], "-m")==0 || strcmp(argv[1], "--minimal")==0)) {
            mode = CK_MINIMAL;
        }
        else if (argc>1 && (strcmp(argv[1], "-n")==0 || strcmp(argv[1], "--normal")==0)) {
            mode = CK_NORMAL;
        }
        else if (argc>1 && (strcmp(argv[1], "-e")==0 || strcmp(argv[1], "--environment")==0)) {
            mode = CK_ENV;
        }
        else if (argc>1) {
            #ifndef CHECK_HELPER_USAGE_STRING
            printf("usage: [test_executable] [verbosity flags: -(-q)uiet | -(-m)inimal | -(-n)ormal (default) | -(-v)erbose | -(-e)nvironment ]\n"
                   "       if -e or --environment get verbosity from environment variable CK_VERBOSITY (values: silent minimal normal verbose)\n");
            #else
            printf(CHECK_HELPER_USAGE_STRING);
            #endif
            return 1;
        }

        Suite *suite = CHECK_HELPER_TEST_SUITE_NAME();

        SRunner *suiteRunner = srunner_create(suite);
        srunner_run_all(suiteRunner, mode);
        int numberFailed = srunner_ntests_failed(suiteRunner);
        srunner_free(suiteRunner);

        return (numberFailed == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
    }

#endif

#endif

Withdrawal Syndrome

Ceremonial tea whiskAlthough I have my personal opinion, the effort of ranking things like Ubuntu Linux, Microsoft Windows and Apple OS X from ‘better to worse’ is clearly Lilliputian. Therefore I shall abstain.

But I’ll report that I was recently given a Windows 8 laptop, and an Android ‘Jelly Bean’ tablet. Much to my embarrassment, I am finding both barely operatable.

Whilst I’ve been under the pain and delight of OS X and iOS, Windows went through several transformations, and is now barely recognizable from the last time we’ve met (XP, if you believe). Meanwhile, Android is for me completely new (I went directly from Simbian to iOS) and equally alien.

As a wide-eyed Computer Science student, I was put off by the design philosophy of OS X and iOS : the ‘my way or the highway’ appearance, the monastic simplicity, the detachment between ends and means achieved through an extreme cloaking of the mechanisms. The system reeked of coercion and… insincerity.

Now, as an overworked Engineering professor, the same dreaded features became desirable. I no longer want to choose one from 50 color schemes for my window manager, or worse, design my own color scheme. I want, out of the box, one beautiful color scheme that works. I know : it is kinda sad, and totally against the hacker ethos. But nowadays, the investment of the scarce time I have for hacking is decided very judiciously.

Once upon a time, the Rube Goldberg design of Windows and Android would make my delight for weeks — no checkmark would go unexplored, no deep dialog box would remain unopened. It was a given that I would have to reinstall the system from scratch a few times, for I would “customize” (i.e., torture) it to the point of hopelessness.

I wouldn’t mind the updates. I wouldn’t mind the nagging confirmations. I wouldn’t mind having to reinstall the system from scratch to get rid of all ‘free !’ useless/noxious third-party bundled software. I wouldn’t mind the inconsistent GUI design. I wouldn’t mind that the system, fresh out of the box, would let a driver do a “fatal exception”, or an essential application “execute an illegal operation” — either demanding hours of troubleshooting.

All that mattered was that once everything was sorted out — days or weeks later — I would get a system customized and optimized to my exacting tastes. A system that kept no secrets from me. A system of my own.

Magic BulletNowadays, I want to easily unwrap a beautiful package — at each step feeling that my gestures follow a rhythm etched in the very fabric of the continuum — and have the hardware satisfyingly fall in the void of my hands, heavily filling the negative space between my fingers, caressing my skin with the touch of finely crafted rich materials. The software comes alive with a barely conscious command. A few discrete questions and the device and I are one, its assimilation into my collective accomplished, my diffusion into its anatomy complete and unnoticed.

For this privilege, I am willing to go as far (the horror !) as actually buying an Apple product.

There must be a 12-steps program for that.

EDIT 17/01 : I was complaining about my Windows 8 experience on twitter, and this dialog happened. I have to recognize they are charming.

Must everything be so difficult ? (Part 1)

I am trying to avoid ranting too much on this blog, but the computing industry is not cooperating.

So, I bought (actually, I was given) an HP Mini 1000 (modelo 1030NR) netbook computer. I would have find it useful for talks and short trips if it weren’t for the incredibly absurd choice of HP of: 1) including a non-standard external video port; 2) failing to provide the adapter to a standard VGA or DVI for more than a year after the machine has hit the market. No, I’m not kidding. Nevertheless, I still found it rather convenient for leisurely browsing the web or doing quick jobs on Microsoft Office.

However I’ve been noticing that the netbook has become slower and slower with the passing months, to the point that, lately, it’s been as useful as as self-heating paperweight. That’s when I’ve decided to do a fresh install of Ubuntu Netbook Edition and use it only for web browsing.

After not little struggle to have Ubuntu NE installed in an SD Card (I ended up using the Universal USB Installer in Parallels, since the solution described in the Mac section creates a filesystem that apparently will only boot on a Mac machine — on a PC, it is recognized as “unformatted”).

I’ve booted — finally ! — to Ubuntu NE, but only to discover that the default installation does not include the proprietary Broadcom firmwares for the (in)famous B43 kernel module.  But, wait ! You can still install then quite easily, by clicking an icon on the system notification area: “Install Additional Drivers”. The only thing is: you have to connect to the Internet. Which you can’t, since you don’t have the drivers in first place. Talk about a Catch-22 !

Fortunately, I’ve found an walkthrough to solve that circular dependency. I quote:

If you do not have any other means of Internet access on your computer, you will have to install b43-fwcutter and patch packages from the install media. After that you will need to setup firmware manually (without the firmware automatically downloading and being set up).

Step 1

b43-fwcutter is located on the Ubuntu install media under ../pool/main/b/b43-fwcutter/ and patch is located under ../pool/main/p/patch/ or both in the official repositories online. Double click on the package to install or in a terminal (under the desktop menu Applications > Accessories > Terminal) navigate to the folder containing the package and issue the following command:

/b43-fwcutter/$ sudo dpkg -i b43-fwcutter*

Step 2

On a computer with Internet access, download the required firmware files from http://downloads.openwrt.org/sources/wl_apsta-3.130.20.0.o and http://mirror2.openwrt.org/sources/broadcom-wl-4.150.10.5.tar.bz2

Step 3

Copy the downloaded files to your home folder and execute the following commands consecutively in a terminal to extract and install the firmware:

~$ tar xfvj broadcom-wl-4.150.10.5.tar.bz2
~$ sudo b43-fwcutter -w /lib/firmware wl_apsta-3.130.20.0.o
~$ sudo b43-fwcutter --unsupported -w /lib/firmware broadcom-wl-4.150.10.5/driver/wl_apsta_mimo.o

Step 4

Under the desktop menu System > Administration > Hardware/Additional Drivers, the b43 drivers can be activated for use. Note: A computer restart may be required before using the wifi card. LiveCD/LiveUSB Note: The install media contents are mounted under /cdrom of the filesystem.

Step 5

For temporary use with the LiveCD and LiveUSB environments, instead of a computer restart, in a terminal issue the following commands:

~$ sudo modprobe -r b43 ssb
~$ sudo modprobe b43

Note: Allow several seconds for the network manager to scan for available networks before attempting a connection.

End of the drama ? And they connected happily ever after ? Not quite. I’ve got the wireless card working — for about 4 or 5 seconds. And then it would either drop the connection, or fail to connect altogether, or even stop showing the available networks. What sortilege could be keeping my card from its ethernet blessing ?

A quick inspection on dmesg revealed the matter:

b43-phy0 ERROR: Fatal DMA error: 0x00000400, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000
b43-phy0 ERROR: This device does not support DMA on your system. Please use PIO instead.

Again, Saint Google had a pointer to the solution. To load the modules durint an active session,  type:

sudo modprobe -r b43 ssb
sudo modprobe b43 pio=1 qos=0

For making the settings permanent, type:

sudo touch /etc/modprobe.d/b43.conf
echo "options b43 pio=1 qos=0" | sudo tee -a /etc/modprobe.d/b43.conf

Then, and only then, I’ve got the system working !

Two more hints :

  1. If you are trying to install Ubuntu on the Netbook and the program is stalling, try to unmark both options (download updates, install third party proprietary software) — you can always do those later !
  2. After having all the pains above to make everything working, don’t use the “Install Proprietary Hardware” automated GUI of Ubuntu — it will only mess everything up.

(When computers were created, weren’t they supposed to solve our problems ?)

Oh Turing, Why Hath Thou Forsaken Me ?

I am flabbergasted by how the simplest things can become huge hairy juicy nightmares when it comes to computers. Making your printer get the margins straight, for example. Or imposing a booklet the way you want and not the way the application software/printer driver guess you should.

In the aftermath of the death of my Dell D620, and the ensuing hell of finding an adequate successor, it happened: I have bought a Mac. The ones with Mac OS X, yes. What can I say ? The experience has so far had its ups:

  1. The Time Machine, I don’t know how I was living without one of those: just plug your back-up disk and instant peace-of mind;
  2. The Dashboard: it turns out good software come indeed in small packages;
  3. The wonderfully designed power cord, beautiful and functional. The 1920’s would be so proud;
  4. The head-turning, sensuous to touch, “you deserve it” case (though in crime-ridden Brazil, the bling-bling factor is not always an advantage).

It has also had its downs:

  1. Keyboard nightmares: no home, end, page up, page down keys. Wait: no forward delete key ! Two whole extra modifier keys (“command” and “function”), which are actually important (forward delete, for example, is function+delete). Completely different short-cuts. Talk me about “intuitive” !
  2. The interface design is sleek, but I would gladly trade some of the sleekness for more functional windows, with more predictable stacking behavior (nowadays, 17.4% of all I type is either commant+tab or command+` vainly looking for lost windows before I give up and resort to Exposé), and resizable from the entire border, not only from the isty bitsy lower right corner;
  3. Don’t get me started about the standard behaviour of pasting over folders, which instead of sensibly merging the trees, just crushes the poor older folder. I inaugurated my Mac OS life by loosing all my photo collection (save the few folders in the last paste). While I restored the backup and reorganized everything again, I repeated this mantra “Finder is not safe. Finder will bite your hand. Watch out for Finder. Finder is not safe.”
  4. The “Magic” Mouse, which is now serving as a R$ 220 magic paper weight, in all its bluetooth glory. I prefer mice with right buttons which actually work, and I much prefer mice which do not require other mice to bootstrap by clicking on Bluetooth... Mouse of Valle... Connect... every single time the computer wakes up ! (Think on this: R$ 220 is about US$ 136, so the “Magic” Mouse costs magically twice it’s original US price ! And you talk about taxation abuse in America ?)  (Think on this also: does anything bluetooth actually work ? Isn’t it time for customers to declare it dead already ?)

Of course, I could not survive in this hostile environment but for Parallels, a virtual machine environment which allows you to run your faithful Windows applications side by side to your Mac ones (in its seamless “Coherence” mode). The advantage of Parallels, in comparison to alternatives without a price tag like VirtualBox is that it is worry free: it guides you to install your copy of Windows and it has a great compatibility with software you might want to run, like Microsoft Office. And then it configures networking and printing so you can use everything out-of-box.

Or so it should.

I was facing the strangest of problems: printing from Microsoft Word 2007 to my HP 8500 All-in-one printer on the Windows XP virtual machine using the conveniently pre-installed “Use Mac Printer” resulted in an exaggerated top margin and a cut bottom. It was like the host machine was trying to “fit” the already typeset page into the printable area of a new page, resulting in disaster. I have tried the “chat support” of Parallels (you get 30 days of free support, then you pay per event) but the person behind the screen was as sympathic as inneffective, suggesting things like “let’s change the paper size to A4” (okay, I’d love to — I much prefer ISO paper sizes — but this one must be “US Letter”) and “lets now try to reduce the top margin in 0,3 mm” (no, just no, for ugly turnarounds I don’t need tech support, I can do it alone with the doors locked and the lights off).

The solution, as I suspected, was to bypass the Mac printer driver completely. I downloaded the newest driver from HP and installed in the virtual machine. This was not without its hassles — my printer is on ethernet and HP driver will refuse to cooperate if the machine and the printer are on different IP subnets (assuming there might be firewall blocks ?) so I had to change the networking mode of the virtual machine from “Shared” to “Bridged” . This last step was unsettling easy — from the Parallels Tools taskbar icon, hit Devices... Network... Configure... and then select one of the Bridged adapters. I have just selected “Default Adapter“. I didn’t even had to restart the virtual machine. (I am still waiting for the other shoe to drop.)

Then, and only then, the printer spit out a page worthy of WYSIWYG.

(P.S.: Happy end ? Imagine the feeling of a user, at 2 a.m., getting 5 mangled envelopes out of each batch of 10, and thus finally realizing why every digital press in the town had told him “sorry, we don’t do envelopes”.)

(P.P.S: Little sobering exercise for the computing industry: type on Google “I hate X” × “I love X”, substituting X for some major companies on the field. Compare the hits. Interesting, uh ?)

e-UnCommerce

My notebook unexpectedly broke, prompting me to look for another. A choice not to make lightly, since one’s notebook is one’s co-brain. I made a lot of research and, after some hesitation, narrowed the list to two or three possibilities. But it seems that people are not particularly interested in selling stuff nowadays:

  1. After fighting the usability nightmare of Shop1’s website, I’ve managed to complete an order. Only to be greeted by an uninformative e-mail on payment authorization, asking me to call them. That phone number, of course, is always busy.
  2. With Shop2, it was more straightforward: I had to call them, since they lack e-commerce for Higher Education customers. That phone, of course, was not available at the time I could call (late afternoon). But due to the epic failure of Shop1, I’ve re-considered them. They can send me the machine. Within 10 days.
  3. Shop3 has an e-commerce website, in addition to several physical shops in my town. The website shipping wait is awful, but maybe one of the physical shops has the product in stock, who knows ? Well, I don’t know who knows, because none of the many company websites will tell you. And I’ve tried calling 5 different phone numbers to no avail.

Aren’t “bad economy”  times supposed to make people desperate to sell?