Getting Started

Installing NovaProva

First you need to download and install NovaProva. Here are several ways you can do this, starting with the easiest.

Linux: From The OpenSUSE Build Service

NovaProva is available in installable repositories for many recent Linux distributions at the OpenSUSE Build Service.

For Debian/Ubuntu systems, copy and paste the following commands.

# Choose a distro
distro=xUbuntu_20.04

# This is the base URL for the NovaProva repo
repo=http://download.opensuse.org/repositories/home:/gnb:/novaprova/$distro

# Download and install the repository GPG key
wget -O - $repo/Release.key | sudo apt-key add -

# Add an APT source
sudo bash -c "echo 'deb $repo ./' > /etc/apt/sources.list.d/novaprova.list"
sudo apt-get update

# Install NovaProva
sudo apt-get install novaprova

See here for RedHat and SUSE instructions.

Note: from release 1.5 onwards, a combination of RedHat and OBS policies make it impossible to provide OBS-built NovaProva RPMs for RHEL6, though that platform is still technically supported by RedHat and would otherwise be supported by NovaProva through to the RHEL6 retirement date of 30 November 2020. The specific issue is availability of a compiler with C++11 support.

MacOS: From Homebrew

From release 1.5, NovaProva supports MacOS and provides a Homebrew tap for quick and easy installation on your Mac. Open Terminal and paste these commands:

brew update
brew tap novaprova/novaprova
brew install novaprova

From A Release Tarball

First, download a release tarball.

NovaProva is intended to be built in the usual way that any open source C software is built. It has a configure script generated by autoconf, which you run before building. To build you need to have various pieces of software installed, starting with a typical C development environment and adding the Valgrind header file valgrind.h and the XML and BFD libraries. Here are some example commands which download and install them.

# on Ubuntu
sudo apt-get install -y \
    gcc g++ git autoconf automake libxml2-dev libxml2-utils \
    pkg-config valgrind binutils-dev zlib1g-dev libiberty-dev

# on RHEL / Fedora
sudo yum install -y \
   gcc gcc-c++ autoconf automake libxml2-devel pkgconfig \
   valgrind valgrind-devel binutils-devel zlib-devel

# on MacOS
# EITHER install Apple Xcode to get Apple's clang-based compiler, OR
brew install gcc
# then all these
brew install autoconf automake libxml2 pkg-config binutils gettext
# Valgrind is very useful but you might have packaging issues
brew install valgrind

Note that from release 1.5, NovaProva requires a compiler with C++11 support, for example gcc 4.8.2 or later. On some platforms NovaProva has been successfully tested with the clang compiler as well as gcc.

Once you have those prerequisites installed, you can download, install and build NovaProva.

# download the release tarball from https://github.com/novaprova/novaprova/releases

tar -xvf novaprova-1.1.tar.bz2
cd novaprova-1.1
./configure
make
make install

From Read-Only Git

For advanced users only. NovaProva needs several more tools to build from a Git checkout than from a release tarball, mainly for the documentation. You will need to have Doxygen, XML::LibXML, Sphinx, and Breathe installed. Here are some example commands which download and install them.

# on Ubuntu
# install all the prereqs above, then add...
sudo apt-get install -y doxygen libxml-libxml-perl \
    python-breathe sphinx-common

# on RHEL / Fedora
# install all the prereqs above, then add...
sudo yum install -y doxygen perl-XML-LibXML \
    python-breathe python-sphinx

# on MacOS
brew install doxygen
# then EITHER install these Python dependencies globally
sudo pip install breathe Sphinx
# OR install them into a virtual env
python -m venv novaprova.venv
source novaprova.venv/bin/activate
pip install --upgrade pip # you might need to do this
pip install breathe Sphinx

Once you have those prerequisites installed, you can clone, install and build NovaProva.

git clone https://github.com/novaprova/novaprova.git
cd novaprova
automake -ac   # ignore the errors
autoreconf -iv
./configure
make
make install

Building and Running Tests

Because you’re testing C code, the first step is to build a test runner executable. This executable will contain all your tests and the Code Under Test and will be linked against the NovaProva library and whatever other libraries your Code Under Test needs. Typically, this is done using the check: make target to both build and run the tests.

All the code for this example is included in NovaProva, in the doc/examples/01_simple/ directory.

Let’s start by creating a Makefile containing:

# Makefile
all: libmycode.a

MYCODE_SOURCE=      mycode.c
MYCODE_OBJS=        $(MYCODE_SOURCE:.c=.o)

libmycode.a: $(MYCODE_OBJS)
        ar ruv $@ $(MYCODE_OBJS)
        ranlib $@

NOVAPROVA_CFLAGS= $(shell pkg-config --cflags novaprova)
NOVAPROVA_LIBS= $(shell pkg-config --libs novaprova)

CFLAGS= -g $(NOVAPROVA_CFLAGS)

check: testrunner
        ./testrunner

TEST_SOURCE= mytest.c
TEST_OBJS=  $(TEST_SOURCE:.c=.o)

testrunner:  $(TEST_OBJS) libmycode.a
        $(LINK.c) -o $@ $(TEST_OBJS) libmycode.a $(NOVAPROVA_LIBS)
        @[ `uname -s` = Darwin ] && dsymutil $@

clean:
        $(RM) testrunner libmycode.a $(TEST_OBJS) $(MYCODE_OBJS)

NovaProva uses the GNOME pkgconfig system to make it easy to find the right set of compile and link flags.

Note that you only need to compile the test code mytest.c with NOVAPROVA_CFLAGS. NovaProva does not use any magical compile options or do any pre-processing of your test code or Code Under Test.

However, you should make sure that at least the test code is built with the -g option to include debugging information. NovaProva uses that information to discover tests at runtime.

You do not need to provide a main routine. NovaProva provides a default main routine which implements a number of useful command-line options. You can write your own later, but you probably won’t need to.

Now let’s create an example Code Under Test. It contains the function myatoi which has the same signature and semantics as the well-known atoi function in the standard C library. We have a header file:

/* mycode.h */
#ifndef __mycode_h_
#define __mycode_h_ 1

extern int myatoi(const char *);

#endif /* __mycode_h_ */

and a source file:

/* mycode.c */
#include "mycode.h"

int myatoi(const char *s)
{
    int v = 0;

    for ( ; *s ; s++)
    {
        v *= 10;
        v += (*s - '0');
    }

    return v;
}

The last piece of the puzzle is writing some tests. Each test is a single C function which takes no parameters and returns void. Unlike other unit test frameworks, there’s no API to call or magical macro to use to register tests with the library. Instead you just name the function test_something, and NovaProva will automatically create a test called something which calls the function.

For example, let’s create a test called simple which exercises the most basic functionality of myatoi().

/* mytest.c */
#include <np.h>         /* NovaProva library */
#include "mycode.h" /* declares the Code Under Test */

void test_simple(void)
{
    int r;

    r = myatoi("42");
    NP_ASSERT_EQUAL(r, 42);
}

The macro NP_ASSERT_EQUAL checks that it’s two integer arguments are equal, and if not fails the test. Note that if the assert fails, the test function terminates immediately. If the test function gets to it’s end and returns naturally, the test is considered to have passed.

If we build run this test we get output something like this (some details of the log messages have been replaced with ... for clarity):

% make check
./testrunner
np: [...][...][INFO] starting valgrind
np: [...][...][INFO] NovaProva Copyright (c) Gregory Banks
np: [...][...][INFO] Built for O/S linux architecture x86_64
np: running
np: running: "mytest.simple"
PASS mytest.simple
np: 1 run 0 failed

As expected, the test passed.

NovaProva organises tests into a tree whose node names are derived from the test source directory, test source filename, and test function name. This tree is pruned down to the smallest possible size at which the root of the tree is unique. So the name mytest.simple derives from the name of the function test_simple in source file mytest.c.

Now let’s add another test. The myatoi() function is supposed to convert the initial numeric part of the argument string, i.e. to stop when it sees a non-numeric character. Let’s feed it a string which will exercise this behaviour and see what happens.

/* add this to the end of mytest.c */

static void test_initial(void)
{
    int r;

    r = myatoi("4=2");
    NP_ASSERT_EQUAL(r, 4);
}

Running the tests we see:

% make check
./testrunner
np: [...][...][INFO] starting valgrind
np: [...][...][INFO] NovaProva Copyright (c) Gregory Banks
np: [...][...][INFO] Built for O/S linux architecture x86_64
np: running
np: running: "mytest.simple"
PASS mytest.simple
np: running: "mytest.initial"
EVENT ASSERT NP_ASSERT_EQUAL(r=532, 4=4)
  at mytest.c:34
at 0x80529F2: np::spiegel::describe_stacktrace (/home/gnb/Software/novaprova/np/spiegel/spiegel.cxx:738)
by 0x804C0FC: np::event_t::with_stack (/home/gnb/Software/novaprova/np/event.cxx:26)
by 0x804B2D2: __np_assert_failed (/home/gnb/Software/novaprova/uasserts.c:54)
by 0x804AC27: test_initial (/home/gnb/Software/novaprova/doc/examples/01_simple/mytest.c:34)
by 0x80522D0: np::spiegel::function_t::invoke (/home/gnb/Software/novaprova/np/spiegel/spiegel.cxx:638)
by 0x804C731: np::runner_t::run_function (/home/gnb/Software/novaprova/np/runner.cxx:575)
by 0x804D5C4: np::runner_t::run_test_code (/home/gnb/Software/novaprova/np/runner.cxx:705)
by 0x804D831: np::runner_t::begin_job (/home/gnb/Software/novaprova/np/runner.cxx:762)
by 0x804E0D4: np::runner_t::run_tests (/home/gnb/Software/novaprova/np/runner.cxx:145)
by 0x804E22C: np_run_tests (/home/gnb/Software/novaprova/np/runner.cxx:865)
by 0x804AB12: main (/home/gnb/Software/novaprova/main.c:135)

FAIL mytest.initial
np: 2 run 1 failed
make: *** [check] Error 1

Note also that the new test failed. Immediately after the “np: running:” message we see that the NP_ASSERT_EQUAL macro has failed, and printed both its arguments as well as a stack trace. We expected the variable r to equal to 4 but its actual value at runtime was 532; clearly the myatoi function did not behave correctly. We found a bug!

And now you’re testing with NovaProva. The remainder of this document contains everything you need to know to get the best out of NovaProva. Best of luck and good testing!