Testing with Hy and Nose

I have been using Nose for the longest time and it’s my go-to tool for running tests. Nose is very no-nonsense and has very sensible defaults (to me at least).

Writing a test in Hy that gets discovered and executed with Nose is pretty straightforward. One just needs a correct naming convention and everything happens more or less automatically. However, by applying some standard lisp practices, one can cut down amount of the code needed quite a bit. In this post, I’ll present how my style of writing tests have evolved over time and what kind of measures I took in order to write minimum amount of code.

Continue reading

End of Vacation

The blog has been rather silent recently. Partly because I have been busy learning new things (Haskell mainly) and haven’t had anything to blog about, partly because of spending the vacation to go various places with my family. I haven’t worked on pyherc in a while, but I haven’t forgotten it. Next big thing it needs is a new partitioner that can split a level into different sized squares. After that I can reimplement the old and rather ugly looking catacombs level.

I started a side project, partly in order to learn Haskell and made a first release just recently. Space Privateers is now available in Hackage: https://hackage.haskell.org/package/SpacePrivateers. The game is sci-fi themed roguelike, set in distant future and it is built using the excellent LambdaHack engine. I learned some things about packaging and things like that from it, but actual Haskell coding I haven’t done much. I really would like to contribute to the engine, but first I really have to learn quite a bit more Haskell.

One thing I sort of miss is working with Hy. So I’m going to make more effort and catch up with the great community and see where the language is currently headed.

Learning NumPy Array

I recently got a review copy of Ivan Idris’ “Learning NumPy Array” from Pact Publishing. I have read his earlier book “NumPy Cookbook” and found that useful, so I had my expectations quite high when I started.

The book is not huge brick, but still has enough content for almost 150 pages. As usual, first chapter is dedicated for installing NumPy, Matplotlib, SciPy, and IPython in various operating systems. While the information is good, I think just pointing to online resources would have been sufficient.

The second chapter is reserved for NumPy basics. This is where things are starting to get interesting if you haven’t worked with NumPy and arrays before. It is a good idea to read this chapter carefully, if you aren’t familiar with NumPy. Later chapters are built on top of the foundation laid here and are easier to understand when you understand the basics.

Starting from the 3rd chapter, the book dives into details of NumPy arrays and tools that are available to work with them. I like the fact the each subsequent chapter is built on a theme (basic data analysis, simple predictive analytics and signal processing techniques) with concrete examples. Mostly examples are built around various kinds of weather data, but there’s a little bit of stocks thrown into the mix too. Mathematical foundations are only explained in briefly because of the limited amount of the pages the book has. There’s enough detail for reader to understand what is going on and more information is readily available on internet.

Near the end of the book, there is short chapter about profiling, debugging and testing. Especially the part about testing I found very brief and not that useful, but this is book about NumPy after all and not about testing. This is probably the weakest part of the book and could have been left out. The pages used for this chapter could have been used to explain NumPy in more detail.

The last chapter of the book touches other related libraries briefly. It’s good to know how NumPy relates to for example SciPy and scikit-learn.

All in all I found the book very enjoyable to read and easy to follow. Sometimes graphics was getting a bit on the way, like when textual output was shown as an image of text instead of text (so font differed just slightly or the output had different coloured background). The author is already working on the next book, called “Learning Python Data Analysis” which also sounds quite interesting and is expected to come out 2015.

New learning project

I learn by doing and I have never really learned anything about web development. So in order to fix that, I started a new hobby project (as if I didn’t have enough of them already) unexpected-raptor.

Unexpected-raptor is a web site written with Python and Django. The goal is to write a tool that helps me to manage our group’s BattleTech games. BattleTech is really great and fun game, but running a full mercenary company by hand can get a bit tedious because of all the calculations that need to be taken care of in-between of games.

Since writing a straight conversion of the rules into digital form would be breaching the IP rights of the respective owners (I seem to recall that Microsoft actually owns the rights for digital games), I’m writing a more general system that can be configured to take care of BattleTech too.

This is not the first of it’s kind, there exists multiple products that do various parts of the process. But since I’m doing this to learn web development and not to game (although that’s a nice bonus), I’ll be rolling out yet-another-tool.

satin-python is now available at PyPi

PyPi is a Python package index (also known as a cheeseshop). Satin is a UI testing library that I have been tinkering with on and off while developing Herculeum. I have now released version 0.1.0 of Satin in PyPi, in order to make it easier for others to download it.

The change is reflected in requirements-dev file of Herculeum. Now it is possible to download almost all dependencies of the game in just a single command:


pypi install -r requirements-dev

Running nosetests for Hy

Getting nosetest to work with Hy was easier than I thought in the end. The trick I used was to create a Python module that will act as an entry point. In that module I first import Hy, which in turn lets me to import my test modules:

import hy
from .test_person_repository import *

After that I can run nosetests and end up with:

tuukka@new-loft:~/programming/python/sandvalley/src/sandvalley/tests$ nosetests
..
----------------------------------------------------------------------
Ran 2 tests in 0.015s

OK

Pretty nifty I must say. Having to import everything from modules in a entry point module is a bit hassle, but that’s something that I should be able to automate.

The same method can be used to call Hy code from Python program.

hy

I stumbled upon a new programming language called hy. Basically it is for Python what Clojure is for Java; a LISP implementation. I have played around with it a bit, mainly for my Sandvalley project and at least so far I find it easier to use than Clojure. I think this probably is because Python standard library is much more familiar to me than Java’s.

If I have understood everything correctly, interfacing hy and Python is extremely simple. hy programs get turned into Python abstract syntax trees and the interpreter does not see any difference at all. There are some conventions how for example function names are mapped (some-function gets turned into some_function).

Hy seems to have quite active development community. What I lack currently is the documentation, but eventually that’ll pick up too.

(sorry, no code samples this time, still have to figure out how things mesh together :))

Automated Testing Performed by Developers

I finished my thesis about a month ago and it was checked and graded just recently. Title of the thesis is “Automated Testing Performed by Developers”. It consists of a literary review that forms a theoretical basis for the action research. The thesis is available online.

I’m pretty happy how it turned out and I learned a lot while doing it. There are of course many things that I would want to fix, change or expand, but there is only a limited amount of time to write it. The thesis could be summed up as “testing fun, testing good, testing hard”.

Now I just have to finish one more course before graduating.

Refactoring tests

Work with the magic system continues. Last time I added ability to create effects for spells. This time I work on creating those effects. Since the test for this step is really close to the previous step and the code required to make it pass is just couple of lines, I’ll concentrate on writing about something else: refactoring tests.

The test I wrote is as follows:

def test_triggering_effect(self):
    """
    Casting a spell should trigger the effect
    """
    effects_factory = mock(EffectsFactory)
    effect = mock(Effect)
    dying_rules = mock(Dying)
    when(effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(effect)

    effect_handle = EffectHandle(trigger = 'on spell hit',
                                 effect = 'healing wind',
                                 parameters = {},
                                 charges = 1)

    spell = (SpellBuilder()
                .with_effect_handle(effect_handle)
                .with_target(self.character)
                .build())

    spell.cast(effects_factory, 
               dying_rules)
    
    verify(effect).trigger(dying_rules)

You probably notice how similar it is to the code I wrote in the previous step. It didn’t bother me when I was working on making it pass, but as soon as I was done with that, it was time to clean things up.

First step was to move these two similar tests to a separate test class called TestSpellEffects. The maintenance is easier when similar tests or tests for certain functionality are close to each other.

Next I extracted the duplicate code from each test and moved it into a setup function:

def setup(self):
    """
    Setup test cases
    """
    self.character = (CharacterBuilder()
                            .build())

    self.effects_factory = mock(EffectsFactory)
    self.effect = mock(Effect)
    self.dying_rules = mock(Dying)
    when(self.effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(self.effect)

    self.effect_handle = EffectHandle(trigger = 'on spell hit',
                                      effect = 'healing wind',
                                      parameters = {},
                                      charges = 1)

    self.spell = (SpellBuilder()
                    .with_effect_handle(self.effect_handle)
                    .with_target(self.character)
                    .build())

The setup function is run once for each test and it is perfect place for code that would be duplicated otherwise. It’s good idea to pay close attention what actually ends into setup and try to keep it as small as possible. The more there is code in setup function the harder to read the tests seem to me.

After this change the actual tests were much smaller:

def test_creating_effect(self):
    """
    Casting a spell should create effects it has
    """
    self.spell.cast(self.effects_factory,
                    self.dying_rules)

    verify(self.effects_factory).create_effect(key = 'healing wind',
                                               target = self.character)

def test_triggering_effect(self):
    """
    Casting a spell should trigger the effect
    """
    self.spell.cast(self.effects_factory, 
                    self.dying_rules)
    
    verify(self.effect).trigger(self.dying_rules)

Much nicer than the first version. This highlights the fact that the test code is equally important as the production code and deserves equal attention and care.

Changes for this step are in two commits: ad2bdda513bdbd170ac7927fabcf0632b7e8658a and adfa2540bc90ae28187bb09caeba62a4f16adc50.

Effects of spells

The next step with spell casting takes me to effects. There is already a effect subsystem in place that is being used for healing potions and spider poisonings for example. Effects can be one off or they can have a duration (finite or even infinite). So it seemed like a good idea to reuse those effects for spells. It gives the possibility to have immediate spell effects (like magic missiles and fire balls) or longer lasting ones (blessings and curses).

First step is to create some effects, so I will start with a test:

def test_creating_effect(self):
    """
    Casting a spell should create effects it has
    """
    effects_factory = mock(EffectsFactory)
    effect = mock(Effect)
    dying_rules = mock(Dying)
    when(effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(effect)

    effect_handle = EffectHandle(trigger = 'on spell hit',
                                 effect = 'healing wind',
                                 parameters = {},
                                 charges = 1)

    spell = (SpellBuilder()
                .with_effect_handle(effect_handle)
                .with_target(self.character)
                .build())

    spell.cast(effects_factory, 
               dying_rules)

    verify(effects_factory).create_effect(key = 'healing wind',
                                          target = self.character)

Essential parts are where we tell the effect_factory to return a mocked effect when it is being called with correct parameters. EffectHandles are sort of specifications that tell when to create what kind of effect. In this case, when ‘on spell hit’ event happens, we want to create a healing wind. EffectFactory on the other hand can be used to create effect by name, which in this case is ‘healing wind’.

I’m using a SpellBuilder to create instance of Spell object. The reason for this is to decouple tests and the code they are testing to a degree. Obviously there has to be some sort of connectivity, but I try to limit it a bit. If internal representation of spells change, it is enough to change the builder and all the tests are hopefully up and running again (at least if the verification part of the test works).

Executing tests at this point gives an error:

Traceback (most recent call last):
  File "C:\Python32\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\programming\pyHack\src\pyherc\test\unit\test_spells.py", line 85, in test_triggering_effect
    target = self.character)
  File "C:\Python32\lib\site-packages\mockito\invocation.py", line 98, in __call__
    verification.verify(self, len(matched_invocations))
  File "C:\Python32\lib\site-packages\mockito\verification.py", line 51, in verify
    raise VerificationError("\nWanted but not invoked: %s" % (invocation))
mockito.verification.VerificationError:
Wanted but not invoked: create_effect()

One noteworthy thing about how mockito-python verifies interactions between objects. If I have a verification like:

verify(effects_factory).create_effect(key = 'healing wind',
                                      target = character)

It does not match to a call like this:

effects_factory.create_effect('healing wind',
                              character)

If the verification is specified with keyword arguments, then the call must be made with keyword arguments:

effects_factory.create_effect(key = 'healing wind',
                              target = character)

This is a limitation in mockito-python and so far I have not found a workaround for it. It is something that needs to be remembered when working with keyword arguments.

Another advantage of reusing effects subsystem is that I already have an EffectsCollection class ready that is used to store and query Effects and EffectHandles. Instead of writing all that logic again, I can just reuse it in Spell class:

@logged
def get_effect_handles(self, trigger = None):
    """
    Get effect handles

    :param trigger: optional trigger type
    :type trigger: string

    :returns: effect handles
    :rtype: [EffectHandle]
    """
    return self.effects.get_effect_handles(trigger)

In order to make the test pass, I made following modifications to Spell class:

@logged
def cast(self, effects_factory):
    """
    Cast the spell

    :param effects_factory: factory for creating effects
    :type effects_factory: EffectsFactory
    """
    handles = self.effects.get_effect_handles('on spell hit')
    effects = []

    for target in self.target:
        for handle in handles:
            effects.append(effects_factory.create_effect(
                                            key = handle.effect,
                                            target = target))

At this point the test passes. The test verifies only that the EffectsFactory was called with parameters that it should be called with. It does not verify what is being done with the results of the call. That is left for the next step, when I want to write the code to actually trigger the created effects.

Code for this change (including the SpellBuilder) can be found in commit 62512fb694a93e81e792ec2cf41c1d10901f4c32.

Does it make sense to take this tiny steps? Couldn’t I have just written a test that checks that the effects being created are being triggered and hit two birds with a single stone? I could have done that and I think that would have been as valid approach as dividing it to two separate tests. I prefer to work with very small steps and small tests. While they help me to pinpoint the exact spot where the system is misbehaving, the drawback is the amount of code I have to write.