Integration and Regression Testing Environment
Since its beginning the strongSwan project has been using a virtualized integration and regression testing environment to ensure the quality of each release. The test results of the latest release are always published online.
The test environment is based on KVM and reproducible guest images based on Debian. In earlier releases it was based on User-Mode-Linux (UML) and a Gentoo base image.
|Hardware virtualization support is required (Intel VT/AMD-V). This means that running the testing environment in a VM itself generally doesn’t work. It might with newer VMware products, though.|
The host system currently has to run the same architecture as the guests, whose
architecture can be changed via
testing/testing.conf or in
testing/testing.conf.local that is not managed by Git. The only fully supported
architecture currently is
On Debian/Ubuntu (must be recent, Debian 6.0
squeeze won’t work), the following
packages are required to build the testing environment:
* debootstrap * e2fsprogs * kvm * libvirt-bin * parted * qemu * wget * gcc * make
All the required scripts, config files and test suites are located in the
testing directory of the Git repository or a release tarball.
Before starting the build you may want to modify some of the variables in the
testing.conf file or
testing.conf.local that is not managed by Git.
For instance, the
TESTDIR variable defines the root directory in which the
whole testing environment will be built and by setting any of the
no you can avoid re-building parts of the environment that have
already been built earlier.
Building is started by executing the
Several applications are built from sources
testing/scripts/recipes) in the root image.
To rebuild and install strongSwan from the current or an arbitrary source tree
within the root image the
script is provided since version 5.2.1.
Since version 5.8.0, the
creates the keys and certificates used in the test scenarios dynamically. The
scripts has to be run once after the root image has been created (the guest images
are [re-]built automatically).
On Ubuntu with AppArmor installed (the default)
libvirt uses the latter to
secure virtual machines. When creating a guest
libvirt automatically creates
AppArmor profiles. The problem is that these profiles don’t reflect the
multi-layered disk images we use (each guest’s image is linked to a common root
image which in turn is linked to a common base image). One option to fix this is
security_driver = "none" in
/etc/libvirt/qemu.conf and execute
sudo service libvirt-bin restart
When the strongSwan testing environment has been put into place by running the
make-testing script, you are ready to start up the KVM instances by calling
The virtual topology looks like this:
runs the automated tests. If the
<testnames> argument is omitted all tests are
executed. Otherwise only the tests listed will be run as shown in the following
./do-tests ikev2/net2net-psk ikev2/net2net-cert
Each test is divided into the following phases:
Load the test-specific guest configuration if any is provided.
pretest.datscript found in each test directory is executed. Among other commands, strongSwan is started on the IPsec hosts.
evaltest.datscript evaluates if the test has been successful.
posttest.datscript terminates the test e.g. by stopping strongSwan on the IPsec hosts. It is also responsible to clean things up (e.g. firewall rules) that were set up in
Restore the default configuration on every host (new files have to be deleted manually in
The test results and configuration files for all tests are stored in a folder
labeled with the current date and time in directory specified by the
TESTRESULTSDIR variable in
The same results are also automatically transferred to the Apache server running on guest winnetou and can be accessed via the URL
Instead of running tests automatically with
do-tests it is possible to preload
a test scenario with the script:
Individual configuration files can be changed and any command can be executed by logging on to a guest host directly (via SSH or a console window). No password is required to login as root.
Also, the sources for every software built during
make-testing are mounted at
/root/shared which allows to change and recompile these components. Please
note that before being able to successfully compile one of the components inside
a guest you first have to execute
make clean in the component’s source
directory (the same applies if you later want to rebuild the images with
After you finished testing, the default configuration can be restored with the following command (newly created files have to be deleted manually):