Automated Testing Performed by Developers

I finished my thesis about a month ago and it was checked and graded just recently. Title of the thesis is “Automated Testing Performed by Developers”. It consists of a literary review that forms a theoretical basis for the action research. The thesis is available online.

I’m pretty happy how it turned out and I learned a lot while doing it. There are of course many things that I would want to fix, change or expand, but there is only a limited amount of time to write it. The thesis could be summed up as “testing fun, testing good, testing hard”.

Now I just have to finish one more course before graduating.

TeamCity 7 Continuous Integration Essentials

I got a review copy of TeamCity 7 Continuous Integration Essentials by Volodymyr Melymuka recently and I must say that it was pretty captivating to read (finished it in an evening and planning to read it again after digesting a bit). The book is about setting up TeamCity 7 for continuous integration (a subject close to my heart). While it does not go really deep into technical details, it serves as a good starting point for getting started with TeamCity 7.

The book starts by describing continuous integration and some other basic terms and concepts and continues into architecture of TeamCity and build lifecycle. These form the foundation for rest of the book as the author takes us through setting up the TeamCity and a sample project that is used to showcase how unit tests are handled and how code coverage metrics can be collected.

Setting up, configuring and maintaining the TeamCity server and build agents is described in sufficient detail and pointers are given to where to find more information. The end of the book is used to wrap up with more advanced techniques, like remote run, where TeamCity can be used to verify changes before committing them into source control.

I was really happy how down the earth the book is. The author does not feel need to make things complicated or complex, but explains them in a way that is easy to understand. This does not mean that he would be glossing over the details though. Everything is explained in sufficient detail.

The book is good purchase if you don’t have previous experience with setting up and administering TeamCity server. The book also covers TeamCity 7 from point of view of Windows, Linux and Macintosh, making it a good source of information regardless of the operating system in use. The book is relatively short, slightly over 100 pages, so reading through it will not take a long time.

TeamCity and dotCover

I recently had an opportunity to play around with TeamCity and dotCover in while trying to measure test coverage. Basic idea is explained well in blog posting, but I found couple gotchas (reading manual is what I do as a last resort, as usual).

First was that analysed dll-files need to be from a debug build. pdb-files alone probably might do the trick, but easiest way to get those from all of our projects was to make a debug build. This of course can lead to funny situations, if your build has been configured to output files to same directory, regardless them being release or debug configuration. So this needs to be taken into account when collecting files automatically and packaging them for release (we don’t want to ship pdb-files along the release dlls)

Another gotcha is when you have configured different output directories (bin/debug and bin/release). If your build includes testing task that matches by wildcards (*test*.dll for example), changes are that you end up with debug and release binaries in unit testing. That is fast way of doubling your reported test cases though.

Third one is that when you finally get everything up and running, you need to know how to read the results. It seems that total coverage reported is only for those dll-files that actually got picked up while testing (well, that actually makes sense if you think about it). Those dll-files that were not used by the tests, are not reported and are not included in the total statistics.