Getting Started

Installation

First you need to download and install NovaProva. Here are several ways you can do this, starting with the easiest.

From The OpenSUSE Build Service

NovaProva is available in installable repositories for many recent Linux distributions at the OpenSUSE Build Service.

For Debian/Ubuntu systems, copy and paste the following commands.

# Choose a distro
distro=xUbuntu_12.04

# This is the base URL for the NovaProva repo
repo=http://download.opensuse.org/repositories/home:/gnb:/novaprova/$distro

# Download and install the repository GPG key
wget -O - $repo/Release.key | sudo apt-key add -

# Add an APT source
sudo bash -c "echo 'deb $repo ./' > /etc/apt/sources.list.d/novaprova.list"
sudo apt-get update

# Install NovaProva
sudo apt-get install novaprova

See here for RedHat and SUSE instructions.

From A Release Tarball

First, download a release tarball.

NovaProva is intended to be built in the usual way that any open source C software is built. It has a configure script generated by autoconf, which you run before building. To build you need to have various pieces of software installed, starting with a typical C development environment and adding the Valgrind header file valgrind.h and the XML and BFD libraries. On an Ubuntu system, these commands download and install them.

# on Ubuntu
sudo apt-get install -y \
    gcc g++ git autoconf automake libxml2-dev libxml2-utils \
    pkg-config valgrind binutils-dev zlib1g-dev libiberty-dev

Once you have those prerequisites installed, you can download, install and build NovaProva.

# download the release tarball from https://github.com/novaprova/novaprova/releases

tar -xvf novaprova-1.1.tar.bz2
cd novaprova-1.1
./configure
make
make install

From Read-Only Git

For advanced users only. NovaProva needs several more tools to build from a Git checkout than from a release tarball, mainly for the documentation. You will need to have Doxygen, XML::LibXML, Sphinx, and Breathe installed. On an Ubuntu system, these commands download and install them.

# on Ubuntu
# install all the prereqs above, then add...
sudo apt-get install -y doxygen libxml-libxml-perl python-pip
sudo pip install breathe Sphinx

Once you have those prerequisites installed, you can clone, install and build NovaProva.

git clone https://github.com/novaprova/novaprova.git
cd novaprova
autoreconf -iv
./configure
make
make install

Building a Test Executable

Because you’re testing C code, the first step is to build a test runner executable. This executable will contain all your tests and the Code Under Test and will be linked against the NovaProva library and whatever other libraries your Code Under Test needs. Typically, this is done using the check: make target to both build and run the tests.

Start by creating a Makefile containing:

# Makefile
all: libmycode.a

MYCODE_SOURCE=      mycode.c
MYCODE_OBJS=        $(MYCODE_SOURCE:.c=.o)

libmycode.a: $(MYCODE_OBJS)
        ar ruv $@ $(MYCODE_OBJS)
        ranlib $@

NOVAPROVA_CFLAGS= $(shell pkg-config --cflags novaprova)
NOVAPROVA_LIBS= $(shell pkg-config --libs novaprova)

CFLAGS= -g $(NOVAPROVA_CFLAGS)

check: testrunner
        ./testrunner

TEST_SOURCE= mytest.c
TEST_OBJS=  $(TEST_SOURCE:.c=.o)

testrunner:  $(TEST_OBJS) libmycode.a
        $(LINK.c) -o $@ $(TEST_OBJS) libmycode.a $(NOVAPROVA_LIBS)

clean:
        $(RM) testrunner libmycode.a $(TEST_OBJS) $(MYCODE_OBJS)

NovaProva uses the GNOME pkgconfig system to make it easy to find the right set of compile and link flags.

Note that you only need to compile the test code mytest.c with NOVAPROVA_CFLAGS. NovaProva does not use any magical compile options or do any pre-processing of your test code or Code Under Test.

However, you should make sure that at least the test code is built with the -g option to include debugging information. NovaProva uses that information to discover tests at runtime.

You do not need to provide a main routine. NovaProva provides a default main routine which implements a number of useful command-line options. You can write your own later, but you probably won’t need to.

Now let’s create an example Code Under Test. It contains the function myatoi which has the same signature and semantics as the well-known atoi function in the standard C library. We have a header file:

/* mycode.h */
#ifndef __mycode_h_
#define __mycode_h_ 1

extern int myatoi(const char *);

#endif /* __mycode_h_ */

and a source file:

/* mycode.c */
#include "mycode.h"

int myatoi(const char *s)
{
    int v = 0;

    for ( ; *s ; s++)
    {
        v *= 10;
        v += (*s - '0');
    }

    return v;
}

The last piece of the puzzle is writing some tests. Each test is a single C function which takes no parameters and returns void. Unlike other unit test frameworks, there’s no API to call or magical macro to use to register tests with the library. Instead you just name the function test_something, and NovaProva will automatically create a test called something which calls the function.

For example, let’s create a test called simple which exercises the most basic functionality of myatoi().

/* mytest.c */
#include <np.h>         /* NovaProva library */
#include "mycode.h" /* declares the Code Under Test */

static void test_simple(void)
{
    int r;

    r = myatoi("42");
    NP_ASSERT_EQUAL(r, 42);
}

The macro NP_ASSERT_EQUAL checks that it’s two integer arguments are equal, and if not fails the test. Note that if the assert fails, the test function terminates immediately. If the test function gets to it’s end and returns naturally, the test is considered to have passed.

If we build run this test we get output something like this.

% make check
./testrunner
np: starting valgrind
np: running
np: running: "mytest.simple"
PASS mytest.simple
np: 1 run 0 failed

As expected, the test passed.

NovaProva organises tests into a tree whose node names are derived from the test source directory, test source filename, and test function name. This tree is pruned down to the smallest possible size at which the root of the tree is unique. So the name mytest.simple derives from the name of the function test_simple in source file mytest.c.

Now let’s add another test. The myatoi() function is supposed to convert the initial numeric part of the argument string, i.e. to stop when it sees a non-numeric character. Let’s feed it a string which will exercise this behaviour and see what happens.

/* add this to the end of mytest.c */

static void test_initial(void)
{
    int r;

    r = myatoi("4=2");
    NP_ASSERT_EQUAL(r, 4);
}

Running the tests we see:

% make check
./testrunner
np: starting valgrind
np: running
np: running: "mytest.simple"
PASS mytest.simple
np: running: "mytest.initial"
EVENT ASSERT NP_ASSERT_EQUAL(r=532, 4=4)
at 0x80529F2: np::spiegel::describe_stacktrace (np/spiegel/spiegel.cxx)
by 0x804C0FC: np::event_t::with_stack (np/event.cxx)
by 0x804B2D2: __np_assert_failed (uasserts.c)
by 0x804AC27: test_initial (mytest.c)
by 0x80522D0: np::spiegel::function_t::invoke (np/spiegel/spiegel.cxx)
by 0x804C731: np::runner_t::run_function (np/runner.cxx)
by 0x804D5C4: np::runner_t::run_test_code (np/runner.cxx)
by 0x804D831: np::runner_t::begin_job (np/runner.cxx)
by 0x804E0D4: np::runner_t::run_tests (np/runner.cxx)
by 0x804E22C: np_run_tests (np/runner.cxx)
by 0x804AB12: main (main.c)

FAIL mytest.initial
np: 2 run 1 failed
make: *** [check] Error 1

Note also that the new test failed. Immediately after the “np: running:” message we see that the NP_ASSERT_EQUAL macro has failed, and printed both its arguments as well as a stack trace. We expected the variable r to equal to 4 but its actual value at runtime was 532; clearly the myatoi function did not behave correctly. We found a bug!

And now you’re testing with NovaProva. The remainder of this document contains everything you need to know to get the best out of NovaProva. Best of luck and good testing!