Blog blog("Baptiste Wicht"); (Posts about Tools)

Switching from vim to neovim

Baptiste Wicht — Wed, 20 Sep 2023 17:25:24 GMT

As mentioned in my last article, I am now using neovim instead of vim. I just wanted to comment shortly on this change.

I have been using vim as my IDE for many years now (more than 10 years at least). But at the beginning of the year, I switched to neovim instead.

neovim is a fork of vim, so it is quite similar, but it has some very important differences. It started in 2015, after a multithreading patch was rejected from vim.

Here are some of the advantages:

Very powerful Builtin Langauge Server Protocol (LSP)
Faster startup
Builtin LUA support (the configuration can be written entirely in LUA)
Asynchronous tasks (was later added to vim)

A lot of things are compatible between both editors. When I started with neovim, I simply copied over my vim configuration and started using neovim with only very minor changes.

But what really convinced me to keep using neovim was the LSP. This feature and the great plugins that make use of it greatly simplify how to deeply integrate C++ into neovim. I am now taking advantage of the clang tooling to analyze C++ code on the fly. And I get great autocompletion as well.

All of this is doable with vim as well, but the last time I tried with vim, it was a nightmware to configure. With the help of (quite a few) plugins, I could setup automcompletion in a great way. This allows me to jump to declarations and definitions very easily.

I am far from being a neovim expert, but I am very happy with this tool. It definitely makes my life easier for configuring complex features, compared to vim.

If you are interested, you can find my neovim configuration in my dotfiles repository<https://github.com/wichtounet/dotfiles>. It's very fresh because it was previously saved on another place and was recently copied there. I plan to improve it and little by little rewrite it in LUA. There are still some experimental stuff and it could be improved significantly. But I am really happy with the features configured.

What about you? What do you think of neovim?

Jenkins Tip: Send notifications on fixed builds in declarative pipeline

Baptiste Wicht — Wed, 07 Jun 2017 07:52:57 GMT

In my previous post, I presented a few news about Jenkins and about the fact that I switched to declarative pipelines and Github Organization support for my projects.

The main issue I had with this system is that I lost the ability to get notifications on build that recover. Normally, I would get an email indicating that build X was back to normal, but I haven't found a way to solve that for declarative pipeline.

By following a few posts on StackOverflow, I now have the solution and it is the same problem that was already present in scripted pipelines. Namely, the status of the current build is not set early enough for the notification. Basically, you have to set the notification yourself. Here is what a declarative pipeline looks like:

pipeline {
    agent any

    stages {

        // Normal Stages

        stage ('success'){
            steps {
                script {
                    currentBuild.result = 'SUCCESS'
                }
            }
        }
    }

    post {
        failure {
            script {
                currentBuild.result = 'FAILURE'
            }
        }

        always {
            step([$class: 'Mailer',
                notifyEveryUnstableBuild: true,
                recipients: "baptiste.wicht@gmail.com",
                sendToIndividuals: true])
        }
    }
}

There are two important things here. First, a new stage (success) is added that simply set the result of the current build to SUCCESS once it is done. It must be the last stage on the pipeline. This could also be added as the last step of the last stage instead of adding a new stage, but I think it's clearer like this. The second thing is the failure block in which the result of the current build is set to FAILURE. With these two things, the Mailer plugin now sends notification when a build has been fixed.

I hope that will help some of you. I personally think that it should be much easier than that. All this boilerplate is polluting the pipeline that should be kept more maintainable, but for now it seems, it's the nicest way to achieve that, short of handling all conditions in the post block and sending mail directly there, but that would result in even more boilerplate code.

Jenkins Declarative Pipeline and Awesome Github Integration

Baptiste Wicht — Sun, 04 Jun 2017 18:52:10 GMT

This post is about some news about Jenkins and how I've updated my Jenkins usage. This may be a bit of an enthusiastic post ;)

At the beginning of Jenkins, the best way to define the commands to be executed for your builds was simply to write the commands in the Jenkins interface. This worked quite well. Later on, Jenkins introduced the notion of Pipeline. Instead of a single set of commands to be executed, the build was defined in multi-stages pipeline of commands. This is defined as a Groovy script. One big advantage of this is that all the code for creating the build is inside the repository. This has the advantage that each build is reproducible. This enabled to define complex pipelines of commands for your builds. Moreover, this also allows to have a clean view of which steps are failing and which steps are taking how much of the time of the build. For instance, here it's a view of the pipeline steps for my DLL project:

I think that's pretty cool :)

They recently added a new feature, the declarative pipelines. Instead of scripting the Pipeline in Groovy, the new system uses its own syntax, completely declarative, to put blocks together and add ways of doing actions at specific points and setting environment and so on. I think the new syntax is much nicer than the Groovy scripted Pipeline way, so I started converting my scripts. I'll give an example in a few paragraphs. But first, I'd like to talk about Github integration. Before, every time I created a new project, I add to add it to Jenkins by creating a new project, updating the link to the Github project and a few things in order to add it. This is not so bad but what if you want to build on several branches and keep track of the status of the branches and maybe of the Pull Requests as well. All of this is now very simple. You can now declare the Github organizations (and users) you are of building projects from and the projects inside the organization will be automatically detected as long as they have a Jenkinsfile inside. That means that you'll never have to create a project yourself or handle branches. Indeed, all the created projects can now handle multiples. For instance, here is the status of the two current branches of my dll project:

It's maybe not the best example since one branch is failing and the other is unstable, but you can see that you can track the builds for each branch in a nice way.

A very good feature of this integration is that Jenkins will now automatically marks commits on your Github with the status of your builds at no cost! For instance, here is the status on my ETL project after I configured on Jenkins and made the first builds:

Pretty cool I think :)

Another nice thing in Jenkins is the Blue Ocean interface. This is an alternative interface, especially well-suited for multi-branch projects and pipelines. It looks much more modern and I think it's quite good. Here are a few views of it:

The Activity view for the last events of the project:

The Branches view for the status of each branch:

The view of the status of a build:

The status of the tests for a given build:

It's likely that it won't appeal to everyone, but I think it's pretty nice.

If we get back to the declarative Pipeline, here is the declarative pipeline for my Expression Templates Library (ETL) project:

pipeline {
    agent any

    environment {
       CXX = "g++-4.9.4"
       LD = "g++-4.9.4"
       ETL_MKL = 'true'
    }

    stages {
        stage ('git'){
            steps {
                checkout([
                    $class: 'GitSCM',
                    branches: scm.branches,
                    doGenerateSubmoduleConfigurations: false,
                    extensions: scm.extensions + [[$class: 'SubmoduleOption', disableSubmodules: false, recursiveSubmodules: true, reference: '', trackingSubmodules: false]],
                    submoduleCfg: [],
                    userRemoteConfigs: scm.userRemoteConfigs])
            }
        }

        stage ('pre-analysis') {
            steps {
                sh 'cppcheck --xml-version=2 -j3 --enable=all --std=c++11 `git ls-files "*.hpp" "*.cpp"` 2> cppcheck_report.xml'
                sh 'sloccount --duplicates --wide --details include/etl test workbench > sloccount.sc'
                sh 'cccc include/etl/*.hpp test/*.cpp workbench/*.cpp || true'
            }
        }

        stage ('build'){
            steps {
                sh 'make clean'
                sh 'make -j6 release'
            }
        }

        stage ('test'){
            steps {
                sh 'ETL_THREADS=-j6 ETL_GPP=g++-4.9.4 LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/opt/intel/mkl/lib/intel64:/opt/intel/lib/intel64\" ./scripts/test_runner.sh'
                archive 'catch_report.xml'
                junit 'catch_report.xml'
            }
        }

        stage ('sonar-master'){
            when {
                branch 'master'
            }
            steps {
                sh "/opt/sonar-runner/bin/sonar-runner"
            }
        }

        stage ('sonar-branch'){
            when {
                not {
                    branch 'master'
                }
            }
            steps {
                sh "/opt/sonar-runner/bin/sonar-runner -Dsonar.branch=${env.BRANCH_NAME}"
            }
        }

        stage ('bench'){
            steps {
                build job: 'etl - benchmark', wait: false
            }
        }
    }

    post {
        always {
            step([$class: 'Mailer',
                notifyEveryUnstableBuild: true,
                recipients: "baptiste.wicht@gmail.com",
                sendToIndividuals: true])
        }
    }
}

There is nothing really fancy about, it's probably average. Moreover, since I'm not an expert on pipelines and I've just discovered declarative pipelines, it may not be optimal, but it works. As you'll see there are some problems I haven't been able to fix.

The first part declares the environment variables for the build. Then, the multiple build stages are listed. The first stage checkout the code from the SCM. This ugly piece of code is here to allow to checkout the submodules. It is the only solution I have found so far. It's very ugly but it works. The second steps is simply some basic static analysis. The next step is the classical build step. Then, the tests are run. In that case, I'm using a script because the tests are compiled with several different sets of options and it was much easier to put that in a script that in the Pipeline. Moreover, that also means I can run them standalone. The variables in the line to run the script is another problem I haven't been able to fix so far. If I declare these variables in an environment block, they are not passed to the script for some reason, so I had to use this ugly line. The next two blocks are for Sonar analysis. If you start with Sonar, you can simply the second block that passed the branch information to Sonar. Unfortunately, Sonar is very limited in terms of Git branches. Each branch is considered as another totally different project. That means the false positives defined in the master branch will not be used in the second branch. Therefore, I kept a clean master and several different projects for the other branches. Once Sonar improves this branch handling stuff, if they ever do, I'll be able to get rid of one of these conditional stages. The last stage is simple running the benchmark job. Finally, the post block is using the Mailer plugin to send failed builds information. Again, there is a problem here since this does not send "Back to normal" information as it used to do before. I've asked this question on StackOverflow, but haven't received an answer so far. I'll post a better solution on this blog once I have one. If any of you have some solutions to these problems, don't hesitate to post in the comments below or to contact me on Github.

Here it is. I really think Jenkins is getting even greater now with all this cool stuff and I advice you to try it out!

Migrated from owncloud 5 to Nextcloud 11

Baptiste Wicht — Fri, 03 Mar 2017 08:08:58 GMT

For several years now I've been using Owncloud running on one of my servers. I'm using simply using as a simple synchronization, I don't use any of the tons of fancy features they keep adding. Except from several synchronization issues, I haven't had too much issues with it.

However, I have had a very bad time with updates of Owncloud. The last time I tried, already long ago, was to upgrade from 5.0 to 6.0 and I never succeeded without losing all the configuration and having to do the resync. Therefore, I've still an Owncloud 5.0 running. From this time, I had to say that I've been lazy and didn't try again to upgrade it. Recently, I've received several mails indicating that this is a security threat.

Since I was not satisfied with updates in Owncloud and its security has been challenged recently, I figured it would be a good moment to upgrade to Nextcloud which is a very active fork of Owncloud that was forked by developers of Owncloud.

I haven't even tried to do an upgrade from such an old version to the last version of Nextcloud, it was doomed to fail. Therefore, I made a new clean installation. Since I only use the sync feature of the tool, it does not really matter, it is just some time lost to sync everything again, but nothing too bad.

I configured a new PostgreSQL on one of my servers for the new database and then installed Nextcloud 11 on Gentoo. It's a bit a pain to have a working Nginx configuration for Nextcloud, I don't advice to do it by hand, better take one from the official documentation, you'll also gain some security. One very bad thing in the installation process is that you cannot choose the database prefix, it's set like Owncloud. The problem with that is that you cannot install both Owncloud and Nextcloud on the same database which would be more practical for testing purpose. It's a bit retarded in my opinion, but not a big problem in the end. Other than these two points, everything went well and it was installation pretty nicely. Then, you should have your user ready to go.

As for the interface, I don't think there is a lot to tell here. Most of it is what you would except from this kind of tool. Moreover, I very rarely use the web interface or any of the feature that are not the sync feature. One thing that is pretty cool I think is the monitoring graphs in the Admin section of the interface. You can the number of users connected, the memory used and the CPU load. It's pretty useful if you share your Nextcloud between a lot of different users.

I didn't have any issue with the sync either. I used the nextcloud-client package on Gentoo directly and it worked perfectly directly. It took about 10 minutes to sync everything again (about 5GB). I'll have to do the same thing on my other computer as well, but I don't think I'll have any issue.

So far, I cannot say that this is better than Owncloud, I just hope the next upgrade will fare better than they did on Owncloud. Moreover, I also hope that the security that they promise is really here and I won't have any problem with it. I'll see in the future!

For more information about nextcloud and owncloud compared to each other, you can read this in-depth comparison, by David Feldman.

PVS-Studio on C++ Library Review

Baptiste Wicht — Tue, 20 Dec 2016 08:40:12 GMT

PVS-Studio is a commercial static analyzer for C, C++ and C#. It works in both Windows and Linux.

It has been a long time since I wanted to test it on my projects. I contacted The PVS-Studio team and they gave me a temporary license so that I can test the tool and make a review.

I tried the static analyzer on my Expression Templates Library (ETL) project. This is a heavily-templated C++ library. I tried it on Linux of course.

Usage

The installation is very simple, simply untar an archive and put the executables in your PATH (or use absolute paths). There are also some deb and rpm packages for some distributions. You need strace to make the analyzer work, it should be available on any Linux platform.

The usage of PVS-Studio on Linux should be straightforward. First, you can use the analyzer directly with make and it will detect the invocations of the compiler. For instance, here is the command I used for ETL:

pvs-studio-analyzer trace -- make -j5 debug/bin/etl_test

Note that you can use several threads without issues, which is really great. There does not seem to be any slowdown at this stage, probably only collecting compiler arguments.

This first step creates a strace_out file that will be used by the next stage.

Once, the compilation has been analyzed, you can generate the results with the analyze function, for which you'll need a license. Here is what I did:

pvs-studio-analyzer analyze -l ~/pvs_studio/PVS-Studio.lic -j5

Unfortunately, this didn't work for me:

No compilation units found
Analysis finished in 0:00:00.00

Apparently, it's not able to use the strace_out it generated itself...

Another possibility is to use the compilation database from clang to use PVS-Studio. So I generated my compile_commands.json file again (it was not up to date...) with Bear. And then, you only need to run the analyze step:

pvs-studio-analyzer analyze -l ~/pvs_studio/PVS-Studio.lic -j5

Make sure you have the same compiler configured than the one used to generate the compilation database to avoid errors with compiler arguments.

Unfortunately, this just printed a load of crap on my console:

(L8Pu(]-'Lo8h>uo(_uv(uo2(->'2h_u(uo2(uvU2K h>'o8a=}Lkk;x[G^%cuaa8acr[VS%
$ckUaoc8 c'8>_-o-8>U2cu/kau==-8>c-=cU2]Uf=c U2=u%c&kU__->j}c@uvu2%cJ
(L8Pu(]-'Lo8h>uo(_uv(uo2(->'2h_u(uo2(uvU2K h>'o8a=}Lkk;JVJ^%cuaa8acr[VS%
$ckUaoc8 c'8>_-o-8>U2cu/kau==-8>c-=cU2]Uf=c U2=u%c&kU__->j}c@uvu2%cJ
(L8Pu(]-'Lo8h>uo(_uv(uo2(->'2h_u(uo2(uvU2K h>'o8a=}Lkk;*[G^%cuaa8acr[VS%
$ckUaoc8 c'8>_-o-8>U2cu/kau==-8>c-=cU2]Uf=c U2=u%c&kU__->j}c@uvu2%cJ
(L8Pu(]-'Lo8h>uo(_uv(uo2(->'2h_u(uo2(uvU2K h>'o8a=}Lkk;b[b^%cuaa8acr[VS%
$ckUaoc8 c'8>_-o-8>U2cu/kau==-8>c-=cU2]Uf=c U2=u%c&kU__->j}c@uvu2%cJ
(L8Pu(]-'Lo8h>uo(_uv(uo2(->'2h_u(uo2(uvU2K h>'o8a=}Lkk;[[x^%cuaa8acr[VS%
$ckUaoc8 c'8>_-o-8>U2cu/kau==-8>c-=cU2]Uf=c U2=u%c&kU__->j}c@uvu2%cJ

Pretty nice, isn't it ?

Let's try again in a file:

pvs-studio-analyzer analyze -o results.log -l ~/pvs_studio/PVS-Studio.lic -j5

The time is quite reasonable for the analysis, it took much less time than the compilation time. In total, it took 88 seconds to analyze all the files. It's much faster than the clang static analyzer.

This time it worked, but the log file is not readable, you need to convert it again:

plog-converter -t errorfile -o errors results.log

And finally, you can read the results of the analysis in the errors file.

Results

Overall, PVS-Studio found 236 messages in the ETL library, I was expecting more. I also wish there was an HTML report that include the source code as well as the error message. I had to lookup at the code for each message (you could integrate it in vim and then use the quickfix window to do that). There are some visualization but in things like QtCreator or LibreOffice which I don't have nor want on my computer.

Let's look at the results. For each message, I'll include the message from PVS-Studio and the code if it's relevant.

The first is about using the comma:

include/etl/traits.hpp:674:1: error: V521 Such expressions using the ',' operator are dangerous. Make sure the expression is correct.
include/etl/traits.hpp:674:1: error: V685 Consider inspecting the return statement. The expression contains a comma.

template <typename E>
constexpr std::size_t dimensions(const E& expr) noexcept {
    return (void)expr, etl_traits<E>::dimensions();
}

Here I'm simply using the comma operand to ignore expr to avoid a warning. To make this compile in C++11, you need to do it in one line otherwise it's not a constexpr function. It's probably not perfect to use this construct, but there is no problem here.

There is a bunch of these, let's filter them, it remains 207 warnings. Let's jump to the next one:

include/etl/impl/blas/fft.hpp:29:1: error: V501 There are identical sub-expressions to the left and to the right of the '==' operator: (DFTI_SINGLE) == DFTI_SINGLE

inline void fft_kernel(const std::complex<float>* in, std::size_t s, std::complex<float>* out) {
    DFTI_DESCRIPTOR_HANDLE descriptor;

    void* in_ptr = const_cast<void*>(static_cast<const void*>(in));

    DftiCreateDescriptor(&descriptor, DFTI_SINGLE, DFTI_COMPLEX, 1, s); //Specify size and precision
    DftiSetValue(descriptor, DFTI_PLACEMENT, DFTI_NOT_INPLACE);         //Out of place FFT
    DftiCommitDescriptor(descriptor);                                   //Finalize the descriptor
    DftiComputeForward(descriptor, in_ptr, out);                        //Compute the Forward FFT
    DftiFreeDescriptor(&descriptor);                                    //Free the descriptor
}

Unfortunately, the error is inside the MKL library. Here, I really don't think it's an issue. There is pack of them. I forgot to exclude non-ETL code from the results. Once filter from all dependencies, 137 messages remain.

include/etl/eval_functors.hpp:157:1: warning: V560 A part of conditional expression is always false: !padding.

This is true, but not an issue since padding is a configuration constant that enables the use of padding in vector and matrices. There was 27 of these at different locations and with different configuration variables.

include/etl/op/sub_view.hpp:161:1: note: V688 The 'i' function argument possesses the same name as one of the class members, which can result in a confusion.

This is again true, but not a bug in this particular case. It is still helpful and I ended up changing these to avoid confusion. Again, there was a few of these.

etl/test/src/conv_multi_multi.cpp:23:1: error: V573 Uninitialized variable 'k' was used. The variable was used to initialize itself.

This one is in the test code:

for (size_t k = 0; k < etl::dim<0>(K); ++k) {
    for (size_t i = 0; i < etl::dim<0>(I); ++i) {
        C_ref(k)(i) = conv_2d_valid(I(i), K(k)); // HERE
    }
}

I don't see any error, k is initialized correctly to zero in the first loop. This is a false positive for me. There were several of these in different places. It seems to that the use of the operator() is confusing for PVS-Studio.

include/etl/traits.hpp:703:1: note: V659 Declarations of functions with 'rows' name differ in the 'const' keyword only, but the bodies of these functions have different composition. This is suspicious and can possibly be an error. Check lines: 693, 703.

template <typename E, cpp_disable_if(decay_traits<E>::is_fast)>
std::size_t rows(const E& expr) { //693
    return etl_traits<E>::dim(expr, 0);
}

template <typename E, cpp_enable_if(decay_traits<E>::is_fast)>
constexpr std::size_t rows(const E& expr) noexcept { //703
    return (void)expr, etl_traits<E>::template dim<0>();
}

Unfortunately, this is again a false positive because PVS-Studio failed to recognized SFINAE and therefore the warning is wrong.

include/etl/builder/expression_builder.hpp:345:1: note: V524 It is odd that the body of '>>=' function is fully equivalent to the body of '*=' function.

This one is interesting indeed. It is true that they are exactly because in ETL >> is used for scalar element-wise multiplication. This is quite interesting that PVS-Studio points that out. There was a few of these oddities but all were normal in the library.

etl/test/src/compare.cpp:23:1: error: V501 There are identical sub-expressions to the left and to the right of the '!=' operator: a != a

Again, it is nice that PVS-Studio finds that, but this is done on purpose on the tests to compare an object to itself. If I remove all the oddities in the test cases, there are only 17 left in the headers. None of the warnings on the test case was serious, but there was no more false positives either, so that's great.

include/etl/impl/vec/sum.hpp:92:1: error: V591 Non-void function should return a value.

template <typename L, cpp_disable_if((vec_enabled && all_vectorizable<vector_mode, L>::value))>
value_t<L> sum(const L& lhs, size_t first, size_t last) {
    cpp_unused(lhs);
    cpp_unused(first);
    cpp_unused(last);
    cpp_unreachable("vec::sum called with invalid parameters");
}

This one is interesting. It's not a false positive since indeed the function does not return a value, but there is a __builtin_unreachable() inside the function and it cannot be called. In my opinion, the static analyzer should be able to handle that, but this is really a corner case.

include/etl/sparse.hpp:148:1: note: V550 An odd precise comparison: a == 0.0. It's probably better to use a comparison with defined precision: fabs(A - B) < Epsilon.

inline bool is_zero(double a) {
    return a == 0.0;
}

This is not false, but again this is intended because of the comparison to zero for a sparse matrix. There were 10 of these in the same class.

include/etl/impl/blas/fft.hpp:562:1: note: V656 Variables 'a_padded', 'b_padded' are initialized through the call to the same function. It's probably an error or un-optimized code. Consider inspecting the 'etl::size(c)' expression. Check lines: 561, 562.

dyn_vector<etl::complex<type>> a_padded(etl::size(c));
dyn_vector<etl::complex<type>> b_padded(etl::size(c));

It's indeed constructed with the same size, but for me I don't think it's an odd pattern. I would not consider that as a warning, especially since it's a constructor and not a assignment.

include/etl/dyn_base.hpp:312:1: warning: V690 The 'dense_dyn_base' class implements a copy constructor, but lacks the '=' operator. It is dangerous to use such a class.

This is again a kind of corner case in the library because it's a base class and the assignment is different between the sub classes and not a real assignment in the C++ sense.

include/etl/impl/reduc/conv_multi.hpp:657:1: warning: V711 It is dangerous to create a local variable within a loop with a same name as a variable controlling this loop.

for (std::size_t c = 0; c < C; ++c) {
    for (std::size_t k = 0; k < K; ++k) {
        conv(k)(c) = conv_temp(c)(k);
    }
}

This is again a false positive... It really seems that PVS-Studio is not able to handle the operator().

include/etl/impl/pooling.hpp:396:1: error: V501 There are identical sub-expressions to the left and to the right of the '||' operator: P1 || P2 || P1

template <size_t C1, size_t C2, size_t C3,size_t S1, size_t S2, size_t S3, size_t P1, size_t P2, size_t P3, typename A, typename M>
static void apply(const A& sub, M&& m) {
    const size_t o1 = (etl::dim<0>(sub) - C1 + 2 * P1) / S1 + 1;
    const size_t o2 = (etl::dim<1>(sub) - C2 + 2 * P2) / S2 + 1;
    const size_t o3 = (etl::dim<2>(sub) - C3 + 2 * P3) / S3 + 1;

    if(P1 || P2 || P1){

Last but not least, this time, it's entirely true and it's in fact a bug in my code! The condition should be written like this:

if(P1 || P2 || P3){

This is now fixed in the master of ETL.

Conclusion

The installation was pretty easy, but the usage was not as easy as it could because the first method by analyzing the build system did not work. Fortunately, the system supports using the Clang compilation database directly and therefore it was possible to use.

Overall, it found 236 warnings on my code base (heavily templated library). Around 50 of them were in some of the extend libraries, but I forgot to filter them out. The quality of the results is pretty good in my opinion. It was able to find a bug in my implementation of pooling with padding. Unfortunately, there was quite a few false positives, due to SFINAE, bad handling of the operator() and no handling of __builtin_unreachable. The remaining were all correct, but were not bug considering their usages.

To conclude, I think it's a great static analyzer that is really fast compared to other one in the market. There are a few false positives, but it's really not bad compared to other tools and some of the messages are really great. An HTML report including the source code would be great as well.

If you want more information, you can consult the official site. There is even a way to use it on open-source code for free, but you have to add comments on top of each of your files.

I hope it was helpful ;)

Short review of Bullseye Coverage

Baptiste Wicht — Fri, 16 Sep 2016 11:25:44 GMT

Bullseye is a commercial Code Coverage analyzer. It is fully-featured with an export to HTML, to XML and even a specific GUI to see the application.It costs about 800$, with a renewal fee of about 200$ per year.

I'm currently using gcov and passing the results to Sonar. This works well, but there are several problems. First, I need to use gcovr to generate the XML file, that means two tools. Then, gcov has no way to merge coverage reports. In my tests of ETL, I have seven different profiles being tested and I need the overall coverage report. lcov has a merge feature but it is slow as hell (it takes longer to merge the coverage files than to compile and run the complete test suite seven times...). For now, I'm using a C++ program that I wrote to combine the XML files or a Python script that does that, but neither are perfect and it needs maintenance. Finally, it's impossible to exclude some code from the coverage report (there is code that isn't meant to be executed (exceptional code)). For now, I'm using yet another C++ program that I wrote to do this from comments in code.

Bullseye does have all these feature, so I got an evaluation license online and tried this tool and wrote a short review of it.

Usage

The usage is pretty simple. You put the coverage executables in your PATH variable and activate coverage globally. Then, we you compile, the compiler calls will be intercepted and a coverage file will be generated. When the compilation is done, run the program and the coverage measurements will be filled.

The coverage results can then be exported to HTML (or XML) or visualized using the CoverageBrowser tool:

The main view of the Bullseye tool code coverage results

It's a pretty good view of the coverage result. You have a breakdown by folders, by file, by function and finally by condition. You can view directly the source code:

The source view of the Bullseye tool code coverage results

If you want to exclude some code from your coverage reports, you can use a pragma:

switch (n) {
    case 1: one++; break;
    case 2: two++; break;
    case 3: three++; break;
    #pragma BullseyeCoverage off
    default: abort();
    #pragma BullseyeCoverage on
}

So that the condition won't be set as uncovered.

As for the coverage, it's pretty straightforward. For example:

covmerge -c -ffinal.cov sse.cov avx.cov

and it's really fast. Unfortunately, the merging is only done at the function level, not at the statement or at the condition level. This is a bit disappointing, especially from a commercial tool. Nevertheless, it works well.

Conclusion

To conclude, Bullseye seems to be a pretty good tool. It has more features than standard gcov coverage and all features are well integrated together. I have only covered the features I was interested in, there are plenty of other things you can look at on the official website.

However, if you don't need the extra features such as the visualizer (or use something like Sonar for this), or the merge or code excluding, it's probably not worth paying the price for it. In my case, since the merge is not better than my C++ tool (both do almost the same and my tool does some basic line coverage merging as well) and I don't need the visualizer, I won't pay the price for it. Moreover, they don't have student or open source licensing, therefore, I'll continue with my complicated toolchain :)

Use templight and Templar to debug C++ templates

Baptiste Wicht — Mon, 08 Feb 2016 07:11:18 GMT

C++ has some very good tools to debug, profile and analyze source files and executables. This all works well for standard runtime program. But, when you are using templates, you sometimes want these tools to act at compile-time. And at this point the support is much more scarce.

templight and Templar and two tools that are trying to fix this issue.

From the templight site:

Templight is a Clang-based tool to profile the time and memory consumption of template instantiations and to perform interactive debugging sessions to gain introspection into the template instantiation process.

and Templar is a visualization tool for the traces generated by templight.

Installation

Unfortunately, the templight installation is not user-friendly at all. You need to clone the complete LLVM/Clang tree and add templight inside it before compiling the complete clang toolchain. But that is the case for all clang-based tools... You also need to patch clang but that may not be necessary in the future. The complete instructions are available here.

The installation of Templar is much more convenient:

git clone https://github.com/schulmar/Templar.git
cd Templar
git checkout feature/templight2
qmake .
make
sudo make install

The branch feature/templight2 has much more features than the master and should support both Qt4 and Qt5, but I have only tested it on Qt4.

Example

Let's use the class Fibonacci function as an example:

#include <iostream>

template <std::size_t N>
struct Fibonacci {
    static constexpr const std::size_t value = Fibonacci<N-1>::value + Fibonacci<N-2>::value;
};

template <>
struct Fibonacci<1> {
    static constexpr const std::size_t value = 1;
};

template <>
struct Fibonacci<0> {
    static constexpr const std::size_t value = 0;
};

int main(){
    std::cout << "Fibonacci<5>:" << Fibonacci<5>::value << std::endl;
}

Nothing fancy here, we're simply printing the fifth Fibonacci number on the console.

You can compile it with templight++:

templight++ -Xtemplight -profiler -Xtemplight -memory -Xtemplight -ignore-system -std=c++14 main.cpp

All the templight options starts with -Xtemplight and then you can use any clang++ options. This will generate a a.memory.trace.pbf file in the current directory. You can then run Templar. use File > Open Trace to open the trace file. This should open a window of this sort:

The top-left panel contains the source code of the application, automatically refreshed whenever you move in the template tree. In the top right, there is the template instantiation graph. In the bottom left, you'll see a list of list of files to be able to filter them and in the bottom right, you'll see the list of templates events. You can sort the list of template events by duration which is really convenient. You can then select Fibonacci<5> by double clicking it in the list (once sorted, it should be near the top). This should give you a tree looking something like that:

The edgy nodes are template instantiations and the round nodes are template memoization. We can directly see that each instantiation was only done once. I think this graph view is really helpful if you need to debug computation done at compile time. You can see that that not all nodes are displayed, this is because there is a limit on the displayed depth. Simply click on Fibonacci<3> and the remaining nodes will be shown.

I have already used this tool to find the most time-consuming templates in ETL an DLL. This is a great tool to indicate where you should focus on improving the template compile-time. I have also been able to find some unnecessary instantiations that could be avoided (either with SFINAE or with refactorings).

templight also contains a fully-fledged debugger for template programs, but I haven't tested it.

Conclusion

In conclusion, I would say that templight and Templar are really helping with template debugging and profiling. There is a real lack of tools in this domain and I hope to see more tools of this kind in the future. I hope this will help you develop template-heavy programs or template metaprograms.

Continuous Performance Management with CPM for C++

Baptiste Wicht — Sun, 14 Jun 2015 12:02:57 GMT

For some time, I have wanted some tool to monitor the performance of some of my projects. There are plenty of tools for Continuous Integration and Sonar is really great for continuous monitoring of code quality, but I haven't found anything satisfying for monitoring performance of C++ code. So I decided to write my own. Continous Performance Monitor (CPM) is a simple C++ tool that helps you running benchmarks for your C++ programs and generate web reports based on the results. In this article, I will present this tool. CPM is especially made to benchmark several sub parts of libraries, but it perfectly be used to benchmark a whole program as well.

The idea is to couple it with a Continuous Integration tool (I use Jenkins for instance) and run the benchmarks for every new push in a repository. With that, you can check if you have performance regression for instance or simply see if your changes were really improving the performance as much as you thought.

It is made of two separate parts:

A header-only library that you can use to benchmark your program and that will give you the performance results. It will also generate a JSON report of the collected data.
A program that will generate a web version of the reports with analysis over time, over different compilers or over different configurations.

CPM is especially made to benchmark functions that takes input data and which runtime depends on the dimensions of the input data. For each benchmark, CPM will execute it with several different input sizes. There are different ways to define a benchmark:

two_pass: The benchmark is made of two part, the initialization part is called once for each input size and then the benchmark part is repeated several times for the measure. This is the most general version.
global: The benchmark will be run with different input sizes but uses global data that will be randomized before each measure
simple: The benchmark will be run with different input sizes, data will not be randomized
once: The benchmark will be run with no input size.

Note: The randomization of the data can be disabled.

You can run independent benchmarks or you can run sections of benchmarks. A section is used to compared different implementations of the same thing. For instance, I use them to compare different implementation of convolution or to see how ETL compete with other Expression Templates library.

Examples

I've uploaded three generated reports so that you can have look at some results:

Results of the ETL benchmark (one page per benchmark): Site 1 Source 1
Results of simple ETL / Blaze / Eigen3 comparison: Site 2 Source 2
Results of the CPM dummy examples: Site 3 Source 3

Run benchmarks

There are two ways of running CPM. You can directly use the library to run the benchmarks or you can use the macro facilities to make it easier. I recommend to use the second way since it is easier and I'm gonna try to keep it stable while the library can change. If you want an example of using the library directly, you can take a look at this example. In this chapter, I'm gonna focus on the macro-way.

The library is available here, you can either include as a submodule of your projects or install it globally to have access to its headers.

The first thing to do is to include the CPM header:

#define CPM_BENCHMARK "Example Benchmarks"
#include "cpm/cpm.hpp"

You have to name your benchmark. This will automatically creates a main and will run all the declared benchmark.

Define benchmarks

Benchmarks can be defined either in a CPM_BENCH functor or in the global scope with CPM_DIRECT_BENCH.

simple

CPM_DIRECT_BENCH_SIMPLE("bench_name", [](std::size_t d){ std::this_thread::sleep_for((factor * d) * 2_ns ); })

The first argument is the name of the benchmark and the second argument is the function that will be benchmarked by the system, this function takes the input size as input.

global

CPM_BENCH() {
    test a{3};
    CPM_GLOBAL("bench_name", [&a](std::size_t d){ std::this_thread::sleep_for((factor * d * a.d) * 1_ns ); }, a);
}

The first argument is the name of the benchmark, the second is the function being benchmarked and the following arguments must be references to global data that will be randomized by CPM.

two_pass

CPM_DIRECT_BENCH_TWO_PASS("bench_name",
    [](std::size_t d){ return std::make_tuple(test{d}); },
    [](std::size_t d, test& d2){ std::this_thread::sleep_for((factor * 3 * (d + d2.d)) * 1_ns ); }
)

Again, the first argument is the name. The second argument is the initialization functor. This functor must returns a tuple with all the information that will be passed (unpacked) to the third argument (the benchmark functor). Everything that is being returned by the initialization functor will be randomized.

Select the input sizes

By default, CPM will invoke your benchmarks with values from 10 to 1000000, multiplying it by 10 each step. This can be tuned for each benchmark and section independently. Each benchmark macro has a _P suffix that allows you to set the size policy:

CPM_SIMPLE_P(
    VALUES_POLICY(1,2,3,4,5,6),
    "simple_a_n",
    [](std::size_t d){ std::this_thread::sleep_for((factor * d) * 1_ns ); });

You can also have several sizes (for multidimensional data structures or algorithms):

CPM_DIRECT_BENCH_TWO_PASS_P(
    NARY_POLICY(VALUES_POLICY(16, 16, 32, 32, 64, 64), VALUES_POLICY(4, 8, 8, 16, 16, 24)),
    "convmtx2",
    [](std::size_t d1, std::size_t d2){ return std::make_tuple(dmat(d1, d1), dmat((d1 + d2 - 1)*(d1 + d2 - 1), d2 * d2)); },
    [](std::size_t /*d1*/, std::size_t d2, dmat& a, dmat& b){ b = etl::convmtx2(a, d2, d2); }
)

Configure benchmarks

By default, each benchmark is run 10 times for warmup and then repeated 50 times, but you can define your own values:

#define CPM_WARMUP 3
#define CPM_REPEAT 10

This must be done before the inclusion of the header.

Define sections

Sections are simply a group of benchmarks, so instead of putting several benchmarks inside a CPM_BENCH, you can put them inside a CPM_SECTION. For instance:

CPM_SECTION("mmul")
    CPM_SIMPLE("std", [](std::size_t d){ std::this_thread::sleep_for((factor * d) * 9_ns ); });
    CPM_SIMPLE("fast", [](std::size_t d){ std::this_thread::sleep_for((factor * (d / 3)) * 1_ns ); });
    CPM_SIMPLE("common", [](std::size_t d){ std::this_thread::sleep_for((factor * (d / 2)) * 3_ns ); });
}
CPM_SECTION("conv")
    CPM_TWO_PASS("std",
        [](std::size_t d){ return std::make_tuple(test{d}); },
        [](std::size_t d, test& d2){ std::this_thread::sleep_for((factor * 5 * (d + d2.d)) * 1_ns ); }
        );
    CPM_TWO_PASS("fast",
        [](std::size_t d){ return std::make_tuple(test{d}); },
        [](std::size_t d, test& d2){ std::this_thread::sleep_for((factor * 3 * (d + d2.d)) * 1_ns ); }
        );
}

You can also set different warmup and repeat values for each section by using CPM_SECTION_O:

CPM_SECTION_O("fft",11,51)
    test a{3};
    test b{5};
    CPM_GLOBAL("std", [&a](std::size_t d){ std::this_thread::sleep_for((factor * d * (d % a.d)) * 1_ns ); }, a);
    CPM_GLOBAL("mkl", [&b](std::size_t d){ std::this_thread::sleep_for((factor * d * (d % b.d)) * 1_ns ); }, b);
}

will be warmup 11 times and run 51 times.

The size policy can also be changed for the complete section (cannot be changed independently for benchmarks inside the section):

CPM_SECTION_P("mmul",
    NARY_POLICY(STD_STOP_POLICY, VALUES_POLICY(1,2,3,4,5,6), VALUES_POLICY(2,4,8,16,32,64)))
    test a{3};
    test b{5};
    CPM_GLOBAL("std", [&a](std::size_t d1,std::size_t d2, std::size_t d3){ /* Something */ }, a);
    CPM_GLOBAL("mkl", [&a](std::size_t d1,std::size_t d2, std::size_t d3){ /* Something */ }, a);
    CPM_GLOBAL("bla", [&a](std::size_t d1,std::size_t d2, std::size_t d3){ /* Something */ }, a);
}

Run

Once your benchmarks and sections are defined, you can build you program as a normal C++ main and run it. You can pass several options:

./debug/bin/full -h
Usage:
  ./debug/bin/full [OPTION...]

  -n, --name arg           Benchmark name
  -t, --tag arg            Tag name
  -c, --configuration arg  Configuration
  -o, --output arg         Output folder
  -h, --help               Print help

The tag is used to distinguish between runs, I recommend that you use a SCM identifier for the tag. If you want to run your program with different configurations (compiler options for instance), you'll have to set the configuration with the --configuration option.

Here is a possible output:

 Start CPM benchmarks
    Results will be automatically saved in /home/wichtounet/dev/cpm/results/10.cpm
    Each test is warmed-up 10 times
    Each test is repeated 50 times
    Time Sun Jun 14 15:33:51 2015

    Tag: 10
    Configuration:
    Compiler: clang-3.5.0
    Operating System: Linux x86_64 3.16.5-gentoo


 simple_a(10) : mean: 52.5us (52.3us,52.7us) stddev: 675ns min: 48.5us max: 53.3us througput: 190KEs
 simple_a(100) : mean: 50.1us (48us,52.2us) stddev: 7.53us min: 7.61us max: 52.3us througput: 2MEs
 simple_a(1000) : mean: 52.7us (52.7us,52.7us) stddev: 48.7ns min: 52.7us max: 53us througput: 19MEs
 simple_a(10000) : mean: 62.6us (62.6us,62.7us) stddev: 124ns min: 62.6us max: 63.5us througput: 160MEs
 simple_a(100000) : mean: 161us (159us,162us) stddev: 5.41us min: 132us max: 163us througput: 622MEs
 simple_a(1000000) : mean: 1.16ms (1.16ms,1.17ms) stddev: 7.66us min: 1.15ms max: 1.18ms througput: 859MEs

-----------------------------------------
|            gemm |       std |     mkl |
-----------------------------------------
|           10x10 | 51.7189us | 64.64ns |
|         100x100 | 52.4336us | 63.42ns |
|       1000x1000 | 56.0097us |  63.2ns |
|     10000x10000 | 95.6123us | 63.52ns |
|   100000x100000 | 493.795us | 63.48ns |
| 1000000x1000000 | 4.46646ms |  63.8ns |
-----------------------------------------

The program will give you for each benchmark, the mean duration (with confidence interval), the standard deviation of the samples, the min and max duration and an estimated throughput. The throughput is simply using the size and the mean duration. Each section is directly compared with an array-like output. Once the benchmark is run, a JSON report will be generated inside the output folder.

Continuous Monitoring

Once you have run the benchmark, you can use the CPM program to generate the web reports. It will generate:

1 performance graph for each benchmark and section
1 graph comparing the performances over time of your benchmark sections if you have run the benchmark several time
1 graph comparing different compiler if you have compiled your program with different compiler
1 graph comparing different configuration if you have run the benchmark with different configuration
1 table summary for each benchmark / section

First you have to build and install the CPM program (you can have a look at the Readme for more informations.

Several options are available:

Usage:
  cpm [OPTION...]  results_folder

      --time-sizes             Display multiple sizes in the time graphs
  -t, --theme arg              Theme name [raw,bootstrap,boostrap-tabs] (default:bootstrap)
  -c, --hctheme theme_name     Highcharts Theme name [std,dark_unica] (default:dark_unica)
  -o, --output output_folder   Output folder (default:reports)
      --input arg              Input results
  -s, --sort-by-tag            Sort by tag instaed of time
  -p, --pages                  General several HTML pages (one per bench/section)
  -d, --disable-time           Disable time graphs
      --disable-compiler       Disable compiler graphs
      --disable-configuration  Disable configuration graphs
      --disable-summary        Disable summary table
  -h, --help                   Print help

There are 3 themes:

bootstrap: The default theme, using Bootstrap to make a responsive interface.
bootstrap-tabs: Similar to the bootstrap theme except that only is displayed at the same time for each benchmark, with tabs.
raw : A very basic theme, only using Highcharts library for graphs. It is very minimalistic

For instance, here are how the reports are generated for the ETL benchmark:

cpm -p -s -t bootstrap -c dark_unica -o reports results

Here is the graph generated for the "R = A + B + C" benchmark and different compilers:

and its summary:

Here is the graph for a 2D convolution with ETL:

And the graph for different configurations of ETL and the dense matrix matrix multiplication:

Conclusion and Future Work

Although CPM is already working, there are several things that could be done to improve it further:

The generated web report could benefit from a global summary.
The throughput evaluation should be evaluated more carefully.
The tool should automatically evaluate the number of times that each tests should be run to have a good result instead of global warmup and repeat constants.
A better bootstrapping procedure should be used to determine the quality of the results and compute the confidence intervals.
The performances of the website with lots of graphs should be improved.
Make CPM more general-purpose to support larger needs.

Here it is, I have summed most of the features of the CPM Continuous Performance Analysis tool. I hope that it will be helpful to some of you as well.

If you have other ideas or want to contribute something to the project, you can directly open an issue or a pull request on Github. Or contact me via this site or Github.

Sonar C++ Community Plugin Review

Baptiste Wicht — Thu, 14 May 2015 15:09:59 GMT

It's been a long time since I have written on this blog. I have had quite a lot of work between my Ph.D and my teaching. I have several projects going on, I'll try to write updates on them later on.

Some time ago, I wrote an article about the official C++ plugin for Sonar <http://baptiste-wicht.com/posts/2014/10/sonarqube-inspections-for-cpp-projects.html>. I was quite disappointed by the quality of a plugin. I was expecting more from an expensive official plugin.

There is an open-source alternative to the commercial plugin: sonar-cxx-plugin <https://github.com/wenns/sonar-cxx>. I already tested it quite some time ago (a development version of the 0.9.1 version) and the results were quite bad. I'm using C++11 and C++14 in almost all of my projects and the support was quite bad at that time. Happily, this support has now gotten much better :) In this article, I'll talk about the version 0.9.2.

Usage

The usage of this plugin is very easy, you don't need any complicated build wrapping techniques for it. You simply need to complete a sonar-project.properties file:

sonar.projectKey=etl
sonar.projectName=etl
sonar.projectVersion=1.0

sonar.sourceEncoding=UTF-8
sonar.sources=include,test,workbench
sonar.language=c++

After that, you simply have to use sonar-runner as for any other Sonar project:

sonar-runner

And the analysis will be run.

I haven't had any issues with the analysis. However, the plugin is not yet completely C++11/C++14 compatible, therefore I'm encountering a lot of parser errors during the analysis. When an error is encountered by the parser, the line is skipped and the parser goes to the next line. This means that the analysis of the line is incomplete, which may lead to false positives or to missing issues. This comes from that sonar-cxx uses its own parser, which is to on par with clang-compatible parsers for instance.

Here is the Sonar summary of my ETL project:

Inspections

This plugin supports some inspections on itself. Nevertheless, you have to enable since it seems that most of them are disable by default. Code duplication is also automatically generated during the analysis:

The philosophy of this project is not to develop all inspections, but to integrate with other tools. For instance, cppcheck is already supported and the integration works perfectly. Here are the tools that sonar-cxx supports for quality analysis:

cppcheck
valgrind
Vera++
RATS
PC-Lint

I have only tested cppcheck for now. I plan to use valgrind running on my tests in the future. I don't plan to use the others.

It should also be noted that the plugin supports compiler warnings coming from G++ and Visual Studio. I don't use this since I compile all my projects with -Werror.

The biggest absent here is Clang, there is no support for its warnings, its static-analyzer or its advanced clang-tidy tool. If clang-tidy support does not come in the near future, I'm planning to try to add it myself, provided I find some time.

You can have to some inspections on one of my project:

As with any Sonar projects, you have access to the Hotsposts view:

Unit Tests Integration

I have been able to integrate my unit tests results inside Sonar. The plugin expects JUnit compatible format. Several of C++ unit test libraries already generates compatible format. In my case, I used Catch and it worked very well.

What is even more interesting is the support for code coverage. You have to run your coverage-enabled executable and then use gcovr to generate an XML file that the plugin can read.

This support works quite well. The only thing I haven't been able to make work is the execution time computation of the unit tests, but that is not something I really care about.

Here are the coverage results for one of my files:

Conclusion

Pros

Support of a lot of external tools
Very easy to use
Duplicated code analysis
Very good code coverage analysis integration

Cons

Too few integrated inspections
Limited parsing of C++
Not fully compatible with C++11/C++14
False positives
Not enough love for clang (compiler warnings, clang-tidy, tooling, static-analyzer, ...)

The provided metrics are really good, the usage is quite simple and this plugin supports some external tools adding interesting inspections. Even if this plugin is not perfect, it is a very good way to do Continuous Quality Analysis of your C++ projects. I personally find it superior to the official plugin. The usage is more simple (no build-wrapper that does not work), it supports more external tools and supports JUnit reports. On the other hand, it has much less integrated inspections and rely more on external tools. Both have problems with modern C++ features.

What I would really like in this plugin is the support of the clang-tidy analyzer (and other Clang analysis tools) and also complete C++11/C++14 support. I honestly think that the only way to fix the latter is to switch to Clang parsing with libtooling rather than developing an in-house parser, but that is not up to me.

I will definitely continue to use this plugin to generate metrics for my C++ projects. I use it with Jenkins which launch the analysis every time I push to one my git repositories. This plugin definitely shows promises.

SonarQube inspections for C++ projects

Baptiste Wicht — Sun, 05 Oct 2014 12:55:46 GMT

Back in the days, when I used to develop in Java (I hadn't discovered the wonders of C++ :) ), I used Sonar a lot for my projects. Sonar is a great tool for quality inspections of a project. Sonar has been made for Java and is mostly free and opensource (some plugins are commercial) to inspect Java projects. Unfortunately, this is not the case for C++ inspection. Indeed, the C++ plugin cost 7000 euros (more than 8500$). As I mostly work on C++ for opensource and school projects, I'm definitely not able to buy it. I wanted for a long time to test the commercial C++ plugin. For this article, sonarsource provided me with a short (very short) time license for the C++ plugin.

There is also another option for C++ which is the C++ community plugin: https://github.com/wenns/sonar-cxx. I have tested it some time ago, but I was not satisfied with it, I had several errors and had to use a dev version to make it work a bit. Moreover, the C++11 support is inexistant and management of parsing error is not really satisfying. But maybe it is good for you. This article will only focus on the commercial plugin.

Usage

For each project that you want to analyze with Sonar, you have to create a sonar-project.properties files describing some basic information about your project.

Then, there are two ways to inspect a C++ project. The first one and recommended one is to use the build-wrapper executable. It is a sub project that you have to download and install alongside Sonar. It works by wrapping the commands to your build systems:

build-wrapper make all

and this should generate enough informations for not having to fill each field in the project configuration. The, you have to use the sonar-runner program to upload to Sonar.

I tried it on several projects and there seems to be a problem with the includes. It didn't include the header files in the Sonar inspections.

I finally ended up using manual configuration of the Sonar project and the header files were included correctly. However, you normally have to include many information in the configuration including all macros for instance. For now, I haven't bothered generating them and it doesn't seem to impact too much the results.

When I look in the log, it seems that there are still a lot of parsing errors. They seem mostly related to some compiler macro, especially the __has_feature__ macro of clang. This is the same problem with the build-wrapper. When I don't use the build-wrapper I also have other problems with macros for unit testing.

I also have other errors coming during the inspection, for instance:

error directive: This file requires compiler and library support for the ISO C++
2011 standard. This support is currently experimental, and must be enabled with
the -std=c++11 or -std=gnu++11 compiler options

I think it comes from the fact that I compile with std=c++1y and that Sonar does not support C++14.

Inspections

Here is the results of inspection on my ETL project:

I really like the web interface of Sonar, it really sums well all the information and the various plugins play quite nice with each other. Moreover, when you check issues, you can see directly the source code very clearly. I really think this is the strong point of Sonar.

Here is the Hotspots view for instance:

Or the Time Machine view:

The issues that are reported by Sonar are quite good. On this project there is a lot of them related to naming conventions because I don't follow the conventions configured by default. However, you can easily configure the inspections to give your own naming regex or simple enable/disable some inspections.

There are some good inspections:

Some missing explicit keyword
Some commented block of code that can be removed
An if-elseif construct that should have had a else
Files with too high complexity

However, there are also some important false positives. For instance:

In here, there are no reasons to output this issue since the operator is deleted. It proves that the C++11 support is rather incomplete. I have other false positives of the same kind for = default operators and constructors. Here is another example:

In this case, the varadic template support is mixed with the old ellipsis notation, making it again a lack of C++11 support. There are also other false positives for instance because of lambdas, but all of them were related to C++11.

Various

If you don't think you have enough quality rules, you can also include the one from cppcheck simply by givin the path to cppcheck in sonar-project.properties. I think this is great, since it works all by itself. You can also create your own rule, but you'll have to use XPath for path.

If you want, you can also include unit test reports inside Sonar. I haven't tested this support since they only support cppunit test reports and I use only Catch for my unit tests. It would have been great if JUnit format would have been supported since many tool support it.

The last option that is supported by this plugin is the support of GCOV reports for code coverage information. I haven't been able to make it work, I had errors indicating that the source files were not found. I didn't figure this out. It may come from the fact that I used llvm and clang to generate the GCOV reports and not G++.

Conclusion

First, here are some pros and cons for the C++ support in SonarQube.

Pros

Good default inspections
Great web interface.
cppcheck very well integrated
Issues are easily configurable

Cons

C++11 support is incomplete and no C++14 support
build-wrapper support seems instable. It should be integrated directly into sonar.
Unit tests support is limited to cppunit
Haven't been able to make Code Coverage work
Macro support not flexible enough
Too expensive
Quite complicated
No support for other static analyzer than cppcheck

The general web interface feeling is quite good, everything looks great and the report are really useful. However, the usage of the tool does not feel very professional. I had a lot more problems than I expected to use it. I was also really disappointed by the C++11. The syntax seems to be supported but not the language feature in the inspections, making the C++11 support completely useless. This is weird since they cite C+11 as supported. Moreover, there not yet any C++14 support, but this is less dramatic. It is also a bit sad that they limit the import to cppcheck and no other static analyzers and the same stands for cppunit.

In my opinion, it is really an inferior product compared to the Java support. I was expecting more from a 8500 dollars product.

For now, I won't probably use it anymore on my projects since all of them use at least C++11, but I will probably retry Sonar for C++ in the future hoping that it will become as the Sonar Java support.

Software Reliability Presentation

Baptiste Wicht — Sat, 07 Jun 2014 14:45:33 GMT

Software Reliability

In behalf of my school (College of Engineering and Architecture of Fribourg), I presented a shoft presentation about Software Reliability. In this presentation, I outline the main issues about the subject and propose some solutions:

Software Validation
Defensive Programming
Software Analysis Tools

In the Software Analysis Tools, I present three tools: cppcheck, Valgrind and the Clang Static analyzer. Several examples are presented for each tools as well as some recommendations for using them. A short presentation of SonarQube is also performed.

I thought that it could be of some interest to some of the readers, so here it is:

Don't hesitate if you have any comments or questions about the presentation ;)

The source code for the examples is available on Github.

Install and Use CLang Static Analyzer on a CMake project

Baptiste Wicht — Wed, 09 Apr 2014 14:39:11 GMT

I recently started a bit of work on my compiler (eddic) again. I started by adapting it to build on CLang with libc++. There was some minor adaptions to make it compile, but nothing really fancy. It now compiles and runs fine on LLVM/Clang 3.4 with the last version of libc++. I'm gonna use some features of C++14 in it and I plan to refactor some parts to make it more STL-correct. I also plan to use only CLang on eddic right now, since C++14 support of GCC is not released right now.

I decided it was a good time to try again the CLang static analyzer.

Installation

If, like me, you're using Gentoo, the static analyzer is directly installed with the sys-devel/clang package, unless you disabled the static-analyzer USE flag.

If your distribution does not ship the static analyzer directly with CLang, you'll have to install it manually. To install it from sources, I advise you to follow the Official Installations instruction.

Usage

The usage of CLang static analyzer can be a bit disturbing at first. Most static analysis tools generally takes the sources directly and do their stuff. But that is not how Clang Static Analyzer works. It works as a kind of monitor in top of building the program, using scan-build. When you are analyzing a program, you are also building the program.

For instance, if you are compiling a source file like that:

clang [clang-options] source_file.cpp

you can perform static analysis like that:

scan-build [scan-build-options] clang [clang-options] source_file.cpp

scan-build works by replacing calls to the compiler by calls to ccc-analyzer . This works generally well, but there are some cases where that things get a bit more complicated. That is the case of CMake where the paths to the compiler are hardcoded in the generated makefiles.

For that, you have to run cmake and make with scan-build:

export CCC_CC=clang
export CCC_CXX=clang++
scan-build cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang .
scan-build make

This can take a very long time. On eddic, it is about three times slower than a normal compilation. An important point to note about performance, is that you can run compilations in parallel (-j option of make) and that it is supported by scan-build quite well.

Once analysis is performed, the found bugs are put into an HTML report. By default, the HTML report is created in /tmp/, but you can specificy the folder with -o option of scan-build.

You can enable or disable checker with the -enable-checker and -disable-checker options of scan-build.

Results on eddic

Several versions of Clang ago, I tried the static analyzer on eddic, but it failed on several source files without producing any results. Moreover, at this time, I don't think there was any nice HTML report at this time.

I ran it again on eddic with the last versions. Here is a picture of the generated report:

As you can see, 14 bugs have been found. Unfortunately, none of them is a real bug on my code, but they are not all false positives neither. For instance, here is some unreachable code report:

It is indeed an unreachable statement, but it is expected, since it is an assert to ensure that the code is unreachable. But that proves that the analysis works ;)

Even if it didn't found anything, this time it worked much better than the last time I checked and the HTML results are just really good.

I hope you found this article interesting. If you happen to have interesting results on your codebase with the CLang static analyzer, I'd be glad to hear about them ;)

budgetwarrior 0.2.1 - Minor changes and Gentoo ebuild

Baptiste Wicht — Thu, 16 Jan 2014 11:41:00 GMT

I've released a new version of budgetwarrior, the release 0.2.1. budgetwarrior is a simple command line application to manage a personal budget.

The version 0.2.1 contains several bug fixes about archived accounts and bug fixes for budget across several years.

The application as well as the source code is available online: https://github.com/wichtounet/budgetwarrior

I've created Gentoo ebuilds for this application. They are available on my Portage overlay: https://github.com/wichtounet/portage-overlay

Gentoo Installation

Edit overlays section of /etc/layman/layman.cfg. Here's an example:

overlays: http://www.gentoo.org/proj/en/overlays/repositories.xml http://github.com/wichtounet/portage-overlay/raw/master/repository.xml

Sync layman

layman -S

Add the overlay:

layman -a wichtounet

Install budgetwarrior

emerge budgetwarrior

Conclusion

If you find any issues with the tool, don't hesitate to post an issue on Github. If you have comments about it, you can post a comment on this post or contact me by email.

CMake Testing - Rerun the last failed tests with CTest

Baptiste Wicht — Thu, 06 Dec 2012 07:47:12 GMT

Some time ago, we saw how to use CMake to run Boost Tests in paralel, now it is time for another tip.

A feature that I think is lacking in CMake/CTest is a way to launch only the last failed tests. As it is not possible to do that directly, I posted the question on StackOverflow and got a great answer from Fraser. I wanted to share its answer.

CTest has -I option to select a list of tests to run. The idea here is to convert the log of CTest in format readable by CTest. What I think is great in its answer is that the solution is a CMake script:

set(FailedFileName FailedTests.log)
if(EXISTS "Testing/Temporary/LastTestsFailed.log")
  file(STRINGS "Testing/Temporary/LastTestsFailed.log" FailedTests)
  string(REGEX REPLACE "([0-9]+):[^;]*" "\\1" FailedTests "${FailedTests}")
  list(SORT FailedTests)
  list(GET FailedTests 0 FirstTest)
  set(FailedTests "${FirstTest};${FirstTest};;${FailedTests};")
  string(REPLACE ";" "," FailedTests "${FailedTests}")
  file(WRITE ${FailedFileName} ${FailedTests})
else()
  file(WRITE ${FailedFileName} "")
endif()

This test just transforms one file into another.

You can then run the last failing tests using:

cmake -P 
ctest -I FailedTests.log

Very easy, isn't it ?

There is a limitation to this solution. It won't work when CTest is running in dashboard mode, but it wouldn't take too long to adapt it for that.

Hope you found that tip useful.

Use CMake to easily compiles Latex documents into PDF

Baptiste Wicht — Mon, 24 Sep 2012 07:07:33 GMT

Everyone who compiles Latex documents by hand knows that it is not a panacea. You have to compile the file several times to handle the references. Moreover, if you have a glossary or an index, you have to run others commands between Latex commands so that everything is correctly resolved. The better way to handle Latex compilation is to write a MakeFile compiling each part. However, writing a Latex MakeFile by hand is not easy and especially not interesting.

Using CMake for most of my development projects, I tried to find a CMake script to generates a MakeFile easily. I did found a good script for that, but I wanted to add some features and change some things, so I forked it to Github: The CMakeLatex repository.

Usage

Here is an example using all the features of the script for one of my Latex documents.

PROJECT(master_project NONE)
cmake_minimum_required(VERSION 2.8)
SET(LATEX_OUTPUT_PATH build)
INCLUDE(UseLATEX.cmake)

file(GLOB_RECURSE contents_files RELATIVE ${CMAKE_SOURCE_DIR} contents/*.tex)

ADD_LATEX_DOCUMENT(
    master.tex
    INPUTS ${contents_files}
    IMAGE_DIRS images
    BIBFILES bibliography.bib
    USE_INDEX
    USE_GLOSSARY
    FILTER_OUTPUT
    )

To use it, you have to download the files of the repository and put them aside your Latex files (or just make symlinks to the files in a clone of the repository for easy update). Then, the UseLATEX.cmake file has to be included in your CMakeLists.txt file.

I think that it is a good practice to generates the Latex files in another directory. This directory can be set using the LATEX_OUTPUT_PATH variable.

Then, to add a latex document, you can use the ADD_LATEX_DOCUMENT function. The first parameter is the name of the main Latex file. After that, you have to give several parameters:

INPUTS: It needs the list of Latex files that are included in master file. I use the GLOB_RECURSE function to find all of them in a contents subfolder.
IMAGE_DIRS: The directory where the image are stored. They will be copied to the build folder and automatically converted if necessary.
BIBFILES: If you have a bibliography, you just have to list all the .bib files of your project.
USE_INDEX: Necessary only if your document use an index.
USE_GLOSSARY: Necessary only if your document use a glossary.
FILTER_OUTPUT: This option activates the filtering of pdflatex output to the console. For now, the option is quite limited, but it allows you to have a smoother output. It has to be taken into account that this option hides the overflow and underflow warnings.
CONFIGURE: You can use the CMake configuration feature on some of your files if you want CMake variables to be replaced in the documents.

Once your Latex document is configured, you can just run cmake on your project. After that, you can use targets to generate pdf:

make pdf: This will generate the Latex file using several passes and running all the necessary commands.
make fast: This will generate a pdf in only one pass. This can be useful if you want to see a rough draft of your document quickly.

I already use this script for several of my documents. I hope that it will be useful for some of you. If you want any problem in the script or in the generate make file or if you have an idea for improvement, don't hesitate to let a command or to publish an Issue or a Pull Request in the CMakeLatex repository.

This script only support pdflatex and can only generates pdf directly. If you want latex support with dvi/ps/pdf generation, you should take a look at the original project: CMakeUserUseLATEX

Architexa is available for free - Understand your code base

Baptiste Wicht — Wed, 22 Aug 2012 03:00:37 GMT

Architexa is a tool suite that helps a team to document collaboratively a large Java code base. The tool is made for a whole team to understand a code base. The tool is available as an Eclipse plugin.

When several developers are working on a large application, it is not always simple to have a whole view of the application. Even with some documentation of the application code. It is even harder for a new developer that joins the project to know what the code base is about. In all these cases, Architexa will help your team. It can also be useful when you inherit an application.

Starting from today, Architexa is available for free for individuals and for teams of up to three developers. You can read the official announce at the end of the article.

My Review of Architexa

I tried Architexa on several of my current Java Projects, but never in team. So perhaps my point of view is not very accurate regarding to general users of the tool. I made my tests using Eclipse Juno.

However, even when working alone on a project, I think that this tool is very useful.

The installation is very straightforward, you just have to use the update site directly in Eclipse. Then, you have several new options in the EDI to use Architexa features.

Three diagrams are available in the Architexa tool suite:

Class Diagram : This diagram can be automatically generated for a package, or several packages.
Sequence Diagram : You can create Sequence Diagrams for some of your program actions.
Layered Diagram : This diagram allows you to represent the architecture of your application. The system allows you to represent several levels of details.

You can easily have several diagrams of each type in your project. You can store them as local files, in a server or in the community server to make them available for everyone.

You can add comment in each diagram. In each diagram you can also access the Javadoc of each class. Of course, you can also access any piece of code from your diagrams.

Advantages

Architexa is very simple to use. The tool have access to very good guides directly inside the IDE.
The Real-Time Code analysis is awesome. Once something is in a diagram, it is always kept up to date.
The sharing features are also great.
Even if there are fews diagrams, I think that there are largely enough to have a very good understanding of a code base.
All the graphs looks very nice, there are very readable

Drawbacks

No support for generics and enums.
The tool is only available as an Eclipse plugin. I'm especially using IntelliJ Idea and NetBeans.
The tool is only available for Java. There is a prototype for C/C++ that is available on demand, but I didn't tried it at the current time.
Sometimes, the creation of a very simple diagram takes a bit long time for my feeling. Creating a diagram with three elements can take several seconds. Perhaps, it is better with larger diagrams. I haven't had the occasion to test it with large code .

Conclusion

To conclude, Architexa is a great tool suite. It is useful for any Java developers that works in a large application. It allows them to have better understanding of its code base.

The official announce: Architexa Tool suite is Now Available for Free

More information on the official site: http://www.architexa.com/

taskwarrior-php 0.1 : A simple PHP Frontend for Taskwarrior

Baptiste Wicht — Mon, 09 Jul 2012 07:21:14 GMT

I released the version 0.1 of taskwarrior-php.

This project is a simple PHP Frontend for Taskwarrior. For now, the frontend is quite basic. All the tasks are displayed and sorted by projects. The completion of each project is also computed.

You can also insert a new task. For now, only the project and the description of the task can be modified.

Download

The application is available on the Git repository : https://github.com/wichtounet/taskwarrior-php

The installation is simple, you just have to put all the files in a folder of a PHP server. Then, you have to edit the config.php to set the location of your Taskwarrior files.

It is necessary that the Taskwarrior files are on the PHP server as well. For that, you can use the FTP pull and push commands of Taskwarrior.

Don't hesitate to contact me if you have some ideas for this project or if you find some bugs.

Install Valgrind on Gentoo Linux

Baptiste Wicht — Fri, 13 Apr 2012 06:57:38 GMT

Valgrind is very powerful suite of software for dynamic analysis of binary programs. Valgrind is available in an ebuild on the Gentoo portage tree, but if you want to install valgrind on your Gentoo distribution, there is a problem with the build with the standard library. On Gentoo, the standard C library (glibc) is stripped and Valgrind needs the debug symbols to work. If you try to launch valgrind without the debug symbols, you will get the following error:

valgrind:  Fatal error at startup: a function redirection
valgrind:  which is mandatory for this platform-tool combination
valgrind:  cannot be set up.  Details of the redirection are:
valgrind:  
valgrind:  A must-be-redirected function
valgrind:  whose name matches the pattern:      strlen
valgrind:  in an object with soname matching:   ld-linux-x86-64.so.2
valgrind:  was not found whilst processing
valgrind:  symbols from the object with soname: ld-linux-x86-64.so.2
valgrind:  
valgrind:  Possible fixes: (1, short term): install glibc's debuginfo
valgrind:  package on this machine.  (2, longer term): ask the packagers
valgrind:  for your Linux distribution to please in future ship a non-
valgrind:  stripped ld.so (or whatever the dynamic linker .so is called)
valgrind:  that exports the above-named function using the standard
valgrind:  calling conventions for this platform.
valgrind:  
valgrind:  Cannot continue -- exiting now.  Sorry.

So first, you have to activate the debug symbols for the libraries in your /etc/make.conf:

FEATURES="splitdebug"

Then, you can emerge again the glibc:

sudo emerge glibc

If you already had emerged valgrind, there is no need to emerge it again, it should work now.

And finally, you can emerge valgrind:

sudo emerge valgrind

And everything will work fine.

Install the Insight Debugger on Linux Mint (works for Ubuntu too)

Baptiste Wicht — Thu, 26 Jan 2012 08:28:41 GMT

Insight is a very good debugger based on gdb. I prefer it over ddd or kdbg as I find it clearer and easier to use. Moreover, this debugger is also the one used in the book Assembly language Step by Step, for Linux. However, Insight has been removed from Debian packages already more than a year ago.

But, thanks to SevenMachines, a PPA repository is available to install it on Linux Mint (works also on Ubuntu and Ubuntu-based Linux distributions).

To add the repository to your apt sources, add the following lines to the /etc/apt/sources.list file:

deb http://ppa.launchpad.net/sevenmachines/dev/ubuntu natty main 
deb-src http://ppa.launchpad.net/sevenmachines/dev/ubuntu natty main

and update your apt sources:

sudo apt-get update

Then you can install insight:

sudo apt-get install insight

And now you are ready to use Insight as your debugger.

If you don't trust this PPA repository, you can also try it to install it from the sources (http://sources.redhat.com/insight/), but doesn't seem to very simple to install it. I wasn't able to build it on my Linux Mint 12.

Google+ is now open to all

Baptiste Wicht — Thu, 22 Sep 2011 12:50:53 GMT

After about 90 days of trial on invitation-only mode, Google+ is now open to everybody.

For those who don't know, Google+ is the social network platform of Google, with several interesting features like Circles, Hangouts, ...

For example, you can see my page on Google+.

Personally, I find this social network very interesting, but there are not enough people on it to concurrence really Facebook and the others networks. Don't hesitate to give it a try, it's worth it!

How to profile C++ application with Callgrind / KCacheGrind

Baptiste Wicht — Thu, 01 Sep 2011 06:25:10 GMT

I have shown before how to profile a C++ application using the Linux perf tools. In this post, we will see how to profile the same kind of application using Callgrind. Callgrind is a tool in part of the Valgrind toolchain. It is running in Valgrind framework. The principle is not the same. When you use Callgrind to profile an application, your application is transformed in an intermediate language and then ran in a virtual processor emulated by valgrind. This has a huge run-time overhead, but the precision is really good and your profiling data is complete. An application running in Callgrind can be 10 to 50 times slower than normally.

The output of Callgrind is flat cal graph that is not really usable directly. In this post, we will use KCachegrind to display the informations about the profiling of the analyzed application.

How to compute metrics of C++ project using CCCC

Baptiste Wicht — Tue, 02 Aug 2011 07:18:53 GMT

CCCC (C and C++ Code Counter) is a little command-line tool that generates metrics from the source code of a C or C++ project. The output of the tool is a simple HTML website with information about all your sources.

CCCC generates not only information about the number of lines of codes for each of your modules, but also complexity metrics like the McCabe Cyclomatic Complexity level of your modules and functions, design metrics like the coupling between the modules or object oriented metrics like the depth of inheritance tree for each of your classes, ...

How to profile your applications using the Linux perf tools

Baptiste Wicht — Mon, 18 Jul 2011 07:30:45 GMT

When an application encounters some performance issues, we have to find the code that causes the problem to optimize only what really matters.

To find the code we have to optimize, the profilers are really useful. In this post, we'll use the Linux perf tools to profile a simple C++ application.

The perf tools are integrated in the Linux kernel since the 2.6 version. The perf tools are based on the perf events subsystem. The perf profiler uses hardware counters to profile the application. The result of this profiler are really precise and because it is not doing instrumentation of the code, it is really fast.

IntelliJ Idea 10 Early Acces Program is here

Baptiste Wicht — Wed, 28 Jul 2010 15:59:40 GMT

Today, the Early Access Program (EAP) has been opened for IntelliJ Idea 10.

There is a lot of changes, here are some of them (the most interesting in my point of view) :

Faster indexing speed
Java - Faster method usage search
Spring Web Services support
Spring testing support
New Spring live templates (patterns and frameworks)
intelligent resources support (smart completion and warnings for resource types)
Maven Refactorings
Maven pom.xml editor new features
Maven Dependencies Diagram
Discover & download libraries from maven repositories even in plain java projects
XML editing improvements in IntelliJ IDEA X
Git support improvements, including 'Git Log' view.
Mercurial support
Hibernate Criteria API supported
JDBC Console UI revised
Initial support for AspectJ and Spring Roo frameworks
A lot of other interesting features

To consult the full list of new features, read the release notes. And if you want to test this version, download it here.

Discover Java VisualVM 1.3

Baptiste Wicht — Wed, 28 Jul 2010 05:10:31 GMT

2 days ago, Java VisualVM 1.3 has been released. We'll see what's new with this release.

At this time, there is no package for Ubuntu, so we'll install it manually :

wget https://visualvm.dev.java.net/files/documents/7163/151728/visualvm_13.zip
unzip visualvm_13.zip

And then we can directly launch it :

cd visualvm_13/bin
sh visualvm

You'll be asked for the license agreement and after that the tool is launched :