Tinkering with testing in Hy

I have been playing around with Hy more recently and started liking it a lot. I’m nowhere productive in coding with it, but I’m sure learning all kinds of new things.

Recently I decided to write some tests for Sandvalley project and came up with the following:

(import [sandvalley.tests.helpers [get-in-memory-connection]])
(import [sandvalley.person [save-person load-person]])
(import [sandvalley.database.schema [create-schema]])
(import [hamcrest [assert-that is- equal-to]])

(defn test-save-person []
  (let [[person {"id" None "name" "Pete"}]
        [connection (create-schema (get-in-memory-connection))]
        [saved-person (save-person person connection)]
        [loaded-person (load-person (get saved-person "id") connection)]]
    (assert-that (get loaded-person "name") (is- (equal-to "Pete")))))

(defn test-update-person []
  (let [[person {"id" None "name" "Pete"}]
        [connection (create-schema (get-in-memory-connection))]
        [saved-person (save-person person connection)]
        [loaded-person (load-person (get saved-person "id") connection)]]
    (assoc loaded-person "name" "Uglak")
    (save-person loaded-person connection)
    (let [[updated-person (load-person (get saved-person "id") connection)]]
      (assert-that (get updated-person "name") (is - (equal-to "Uglak"))))))

(if (= __name__ "__main__")
  (test-save-person)
  (test-update-person))

Pretty basic things, we create an empty in-memory database and do some saving, loading and updating on it. Nice thing is that I can use Hamcrest with asserts and the API does not stand out from rest of the code. The part I don’t like is

(assoc loaded-person "name" "Uglak")

, since there I’m changing loaded-person. I’ll probably end up writing a helper function that can combine two dictionaries and use it instead.

However, I haven’t been able to figure out how to run tests with Nose. If I could compile the code to bytecode, it should be possible. Hy project has tests written in Hy and apparently I can even run them with nose. However, when I tried to imitate what they are doing, I ended up getting funny errors deep inside hy2py. So in the meantime, I have settled on running tests from __main__. The drawback of course is that as soon as one of them fails, the execution halts.

Summer of Testing

Yesterday I attended at GeekCollision event about testing to talk about my roguelike project and how I have been testing that. There were other presentations too, unfortunately I could not stay to watch them all.

One presentation particularly caught my attention. Asko Soukka was demoing a system for writing tests for Plone. The main idea was to allow tests used as part of user documentation. The first demo showed how Robot framework could take screen shots as tests are running in order them to be included as a part of documentation. The second demo showed how it is possible to capture video of Robot doing testing and having a voice synthesizer to explain what is going on. Essentially, you end up with a test that verifies a particular feature, screen shots for documentation and a tutorial video. How cool is that?

Automated Testing Performed by Developers

I finished my thesis about a month ago and it was checked and graded just recently. Title of the thesis is “Automated Testing Performed by Developers”. It consists of a literary review that forms a theoretical basis for the action research. The thesis is available online.

I’m pretty happy how it turned out and I learned a lot while doing it. There are of course many things that I would want to fix, change or expand, but there is only a limited amount of time to write it. The thesis could be summed up as “testing fun, testing good, testing hard”.

Now I just have to finish one more course before graduating.

Refactoring tests

Work with the magic system continues. Last time I added ability to create effects for spells. This time I work on creating those effects. Since the test for this step is really close to the previous step and the code required to make it pass is just couple of lines, I’ll concentrate on writing about something else: refactoring tests.

The test I wrote is as follows:

def test_triggering_effect(self):
    """
    Casting a spell should trigger the effect
    """
    effects_factory = mock(EffectsFactory)
    effect = mock(Effect)
    dying_rules = mock(Dying)
    when(effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(effect)

    effect_handle = EffectHandle(trigger = 'on spell hit',
                                 effect = 'healing wind',
                                 parameters = {},
                                 charges = 1)

    spell = (SpellBuilder()
                .with_effect_handle(effect_handle)
                .with_target(self.character)
                .build())

    spell.cast(effects_factory, 
               dying_rules)
    
    verify(effect).trigger(dying_rules)

You probably notice how similar it is to the code I wrote in the previous step. It didn’t bother me when I was working on making it pass, but as soon as I was done with that, it was time to clean things up.

First step was to move these two similar tests to a separate test class called TestSpellEffects. The maintenance is easier when similar tests or tests for certain functionality are close to each other.

Next I extracted the duplicate code from each test and moved it into a setup function:

def setup(self):
    """
    Setup test cases
    """
    self.character = (CharacterBuilder()
                            .build())

    self.effects_factory = mock(EffectsFactory)
    self.effect = mock(Effect)
    self.dying_rules = mock(Dying)
    when(self.effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(self.effect)

    self.effect_handle = EffectHandle(trigger = 'on spell hit',
                                      effect = 'healing wind',
                                      parameters = {},
                                      charges = 1)

    self.spell = (SpellBuilder()
                    .with_effect_handle(self.effect_handle)
                    .with_target(self.character)
                    .build())

The setup function is run once for each test and it is perfect place for code that would be duplicated otherwise. It’s good idea to pay close attention what actually ends into setup and try to keep it as small as possible. The more there is code in setup function the harder to read the tests seem to me.

After this change the actual tests were much smaller:

def test_creating_effect(self):
    """
    Casting a spell should create effects it has
    """
    self.spell.cast(self.effects_factory,
                    self.dying_rules)

    verify(self.effects_factory).create_effect(key = 'healing wind',
                                               target = self.character)

def test_triggering_effect(self):
    """
    Casting a spell should trigger the effect
    """
    self.spell.cast(self.effects_factory, 
                    self.dying_rules)
    
    verify(self.effect).trigger(self.dying_rules)

Much nicer than the first version. This highlights the fact that the test code is equally important as the production code and deserves equal attention and care.

Changes for this step are in two commits: ad2bdda513bdbd170ac7927fabcf0632b7e8658a and adfa2540bc90ae28187bb09caeba62a4f16adc50.

Effects of spells

The next step with spell casting takes me to effects. There is already a effect subsystem in place that is being used for healing potions and spider poisonings for example. Effects can be one off or they can have a duration (finite or even infinite). So it seemed like a good idea to reuse those effects for spells. It gives the possibility to have immediate spell effects (like magic missiles and fire balls) or longer lasting ones (blessings and curses).

First step is to create some effects, so I will start with a test:

def test_creating_effect(self):
    """
    Casting a spell should create effects it has
    """
    effects_factory = mock(EffectsFactory)
    effect = mock(Effect)
    dying_rules = mock(Dying)
    when(effects_factory).create_effect(key = 'healing wind',
                                        target = self.character).thenReturn(effect)

    effect_handle = EffectHandle(trigger = 'on spell hit',
                                 effect = 'healing wind',
                                 parameters = {},
                                 charges = 1)

    spell = (SpellBuilder()
                .with_effect_handle(effect_handle)
                .with_target(self.character)
                .build())

    spell.cast(effects_factory, 
               dying_rules)

    verify(effects_factory).create_effect(key = 'healing wind',
                                          target = self.character)

Essential parts are where we tell the effect_factory to return a mocked effect when it is being called with correct parameters. EffectHandles are sort of specifications that tell when to create what kind of effect. In this case, when ‘on spell hit’ event happens, we want to create a healing wind. EffectFactory on the other hand can be used to create effect by name, which in this case is ‘healing wind’.

I’m using a SpellBuilder to create instance of Spell object. The reason for this is to decouple tests and the code they are testing to a degree. Obviously there has to be some sort of connectivity, but I try to limit it a bit. If internal representation of spells change, it is enough to change the builder and all the tests are hopefully up and running again (at least if the verification part of the test works).

Executing tests at this point gives an error:

Traceback (most recent call last):
  File "C:\Python32\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\programming\pyHack\src\pyherc\test\unit\test_spells.py", line 85, in test_triggering_effect
    target = self.character)
  File "C:\Python32\lib\site-packages\mockito\invocation.py", line 98, in __call__
    verification.verify(self, len(matched_invocations))
  File "C:\Python32\lib\site-packages\mockito\verification.py", line 51, in verify
    raise VerificationError("\nWanted but not invoked: %s" % (invocation))
mockito.verification.VerificationError:
Wanted but not invoked: create_effect()

One noteworthy thing about how mockito-python verifies interactions between objects. If I have a verification like:

verify(effects_factory).create_effect(key = 'healing wind',
                                      target = character)

It does not match to a call like this:

effects_factory.create_effect('healing wind',
                              character)

If the verification is specified with keyword arguments, then the call must be made with keyword arguments:

effects_factory.create_effect(key = 'healing wind',
                              target = character)

This is a limitation in mockito-python and so far I have not found a workaround for it. It is something that needs to be remembered when working with keyword arguments.

Another advantage of reusing effects subsystem is that I already have an EffectsCollection class ready that is used to store and query Effects and EffectHandles. Instead of writing all that logic again, I can just reuse it in Spell class:

@logged
def get_effect_handles(self, trigger = None):
    """
    Get effect handles

    :param trigger: optional trigger type
    :type trigger: string

    :returns: effect handles
    :rtype: [EffectHandle]
    """
    return self.effects.get_effect_handles(trigger)

In order to make the test pass, I made following modifications to Spell class:

@logged
def cast(self, effects_factory):
    """
    Cast the spell

    :param effects_factory: factory for creating effects
    :type effects_factory: EffectsFactory
    """
    handles = self.effects.get_effect_handles('on spell hit')
    effects = []

    for target in self.target:
        for handle in handles:
            effects.append(effects_factory.create_effect(
                                            key = handle.effect,
                                            target = target))

At this point the test passes. The test verifies only that the EffectsFactory was called with parameters that it should be called with. It does not verify what is being done with the results of the call. That is left for the next step, when I want to write the code to actually trigger the created effects.

Code for this change (including the SpellBuilder) can be found in commit 62512fb694a93e81e792ec2cf41c1d10901f4c32.

Does it make sense to take this tiny steps? Couldn’t I have just written a test that checks that the effects being created are being triggered and hit two birds with a single stone? I could have done that and I think that would have been as valid approach as dividing it to two separate tests. I prefer to work with very small steps and small tests. While they help me to pinpoint the exact spot where the system is misbehaving, the drawback is the amount of code I have to write.

Intermission: wiring things for behave

After reflecting the status of the building pieces for spell casting, I realised that I can wire things a bit further on behave side.

I started by fixing a step definition for behave:

@when('{caster_name} casts {spell_and_target}')
def impl(context, caster_name, spell_and_target):
    # Simon casts magic missile on Uglak
    # Simon casts healing wind
    
    caster = get_character(context, caster_name)
    spell = spell_and_target

    make(caster, cast_spell(spell_name = spell))

Since healing wind spell will heal only the caster, I make some assumptions here (like not caring about who I’m casting the spell at). I used cutesy here too, just like in the melee combat, in order to keep the step definition short and decoupled from the actual implementation details.

Implementation for cast_spell – function is as follows:

class CastSpell():
    """
    Class representing casting a spell
    """
    def __init__(self, spell_name):
        """
        Default constructor
        
        :param spell_name: name of the spell to cast
        :type spell_name: string
        """
        super(CastSpell, self).__init__()
        self.spell_name = spell_name
    
    def __call__(self, caster):
        """
        Performs the casting
        
        :param caster: character doing the casting
        :type caster: Character
        """
        caster.old_values = {}
        caster.old_values['hit points'] = caster.hit_points

        action_factory = (ActionFactoryBuilder()
                                    .with_move_factory()
                                    .with_attack_factory()
                                    .with_drink_factory()
                                    .with_inventory_factory()
                                    .with_dying_rules()
                                    .with_spellcasting_factory()
                                    .build())

        caster.cast(direction = 1,
                    spell_name = self.spell_name, 
                    action_factory = action_factory)

def cast_spell(spell_name):
    """
    Cast a spell

    :param spell_name: name of the spell to cast
    """
    action = CastSpell(spell_name)
    return action

This hopefully decouples behave steps enough from actual implementation, in case they change later. CastSpell class is used to cast spells and it is callable (__call__ method has been implemented). This allows me to store context about the action in the object’s attributes and still call it like it would be a function. Another option would of course be to create a function and handle context parameters with closures.

cast_spell – function is just a convenient way of constructing CastSpell object with given parameters.

I’m keeping track of casters hit points, because further down the line I want to verify that they change after the spell has been cast. Later on I should add other attributes like mana points for tracking.

I’m also creating a new instance of ActionFactory. That is an object that is used to create actions, like moving, fighting and casting spells. Having a builder object makes it easy to construct the ActionFactory.

Final step was to implement one more behave step:

@then('{character_name} should be in full health')
def impl(context, character_name):
    character = get_character(context, character_name)

    assert_that(character.hit_points, equal_to(character.max_hp))

The step is short and it verifies that the character in question is in full health.

After running the behave I got following output:

  @wip
  Scenario: Healing                       # features\magic.feature:29
    Given Simon is Wizard                 # features\helpers\events.py:35
    And Uglak is Goblin                   # features\helpers\events.py:35
    And Uglak is almost dead              # features\helpers\events.py:35
    And Uglak is standing in room         # features\helpers\context.py:35
    And Simon is standing away from Uglak # <string>:50
    Given Simon is almost dead            # features\helpers\events.py:35
    When Simon casts healing wind         # <string>:24
    Then Simon should be in full health   # <string>:109
      Assertion Failed: 
      Expected: <5>
           but: was <1>

So Uglak and Simon, both in weak condition, are standing in a room. Simon casts healing wind, but does not have full health. Every step should have been implemented at this point, meaning I can keep coding and running the test to see when the code satisfies it.

Code for this change can be found from commit 3a32c172c65f19e7fcee9360eda4c301994ad8e8.

Targetting spells

My next goal was to start working with actual spell objects. I sketched my ideas on a paper and decided that there would be a single Spell class that could be parametrized to handle various spells. And the first step to creating that class would be targeting characters with spells.

I started writing following test:

    def test_target_single(self):
        """
        Targeting a single character should be possible
        """
        level = LevelBuilder().build()
        
        character = (CharacterBuilder()
                        .with_level(level)
                        .with_location((5, 5))
                        .build())

        spell_generator = SpellGeneratorBuilder().build()

        spell = spell_generator.create_spell(spell_name = 'healing wind', 
                                             target = character)

        assert_that(character, is_in(spell.target))

Nothing too complicated. My focus will be on healing wind spell and after I get it working, I can start from the beginning and see what needs to be changed to get magic missile working. You may notice that target attribute is a list of some sort, instead of just a single entity. I’m jumping forward a bit here, but I know already that I want spells that can target multiple characters and felt that it makes sense to write the system to support that from the beginning.

The code needed to make the test pass is simple:

class Spell():
    """
    Class to represent spells
    
    .. versionadded:: 0.9
    """
    
    def __init__(self):
        """
        Default constructor
        """
        self.target = []

class SpellGenerator():
    """
    Factory for creating spells
    
    .. versionadded:: 0.9
    """
    @logged
    def __init__(self):
        """
        Default constructor
        """
        pass
     
    @logged
    def create_spell(self, spell_name, target):
        """
        Create a spell
         
        :param spell_name: name of the spell
        :type spell_name: string
        :param target: target of the spell
        :type target: Character
        :returns: ready to use spell
        :rtype: Spell
        """
        new_spell = Spell()
        new_spell.target.append(target)
 
        return new_spell

SpellGenerator will be the most used interface for the spell subsystem. It can be used to create all the different spells.

Code for this can be found from commit 112e9a77654490e0c8875c23572aa561bf35462d

Testing screencast

As part of YIIP1400 course I created a short screencast about how testing is done in pyherc/Herculeum. It is available at YouTube. Making it was rather interesting even when the end result is quite rough and unpolished. I probably will make more of these later.

When the magic system has something substantial to show I’m going to make a short demo where I play the game and talk about the new features.

Getting closer to the source of magic

The current plan is to have spell casting use effects sub-system and make Spell object a composite that has a list of effects. In order to make construction of Spells easy, I’m planning on using a factory and call it while creating the action. Following code captures this in the form of a test:

def test_spell_is_created_with_a_factory(self):
    """
    When creating a spell casting action, spell should be created
    """
    spell_factory = SpellFactoryBuilder().build()
    when(spell_factory).create_spell('healing wind').thenReturn(mock())
    spellcasting_factory = (SpellCastingFactoryBuilder()
                                        .with_spell_factory(spell_factory)
                                        .build())
    
    spellcasting_factory.get_action(
                              SpellCastingParameters(self,
                                                     direction = 1, 
                                                     spell_name = 'healing wind'))
    
    verify(spell_factory).create_spell('healing wind')

Essentially, when SpellCastingFactory is used to create a spell called “healing wind”, the SpellFactory should receive a call to make a spell called “healing wind”. I’m creating spell_factory with SpellFactoryBuilder (so I have a real instance and not just a mock object), but then I’m using mockito to tell it to return mock when create_spell – method is called. I would like to use mock, but for some reason I couldn’t get it working with the code in SpellCastingFactoryBuilder:

def with_spell_factory(self, spell_factory):
    """
    Configure spell factory to use
    """
    if spell_factory == None:
        self.use_real_spell_factory = True
    else:
        if hasattr(spell_factory, 'build'):
            self.spell_factory = spell_factory.build()
        else:
            self.spell_factory = spell_factory
    return self

When creating a SpellCastingFactory with the builder, I can supply it with a spell_factory to use. I the spell_factory has method “build”, it will be called and the result is used as a factory, otherwise the supplied object is used. For mock object, hasattr will always return True, so this approach does not work.

Following modification to code makes the test to pass:

@logged
def get_action(self, parameters):
    """
    Create a spell casting action

    :param parameters: parameters used to control creation
    :type parameters: SpellCastingParameters
    """
    spell = self.spell_factory.create_spell(parameters.spell_name)
    
    return SpellCastingAction(caster = parameters.caster, 
                                       direction = parameters.direction,
                                       spell = spell)

Again, really small step forward, but the progression is steady and the direction looks good. I’m going to leave details about finding who the spell hits and how much the mana is subtracted later (both probably ending up in the SpellCastingAction) and concentrate on getting the spell part done.

Code for the changes can be found in the following commit.

Continuing with unit tests

In the previous step we added Cast method to Character class in order to make spellcasting possible. While the method itself works, the actual SpellCastingFactory does not do anything useful. As you can see, I decided that MagicFactory is a poor choice of name in this case and changed it to SpellCastingFactory.

First we need to make the SpellCastingFactory to create us an Action. Following test shows how I envision it working:

def test_creating_action(self):
    """
    Test that action can be created
    """
    action_factory = (ActionFactoryBuilder()
                                .with_spellcasting_factory()
                                .build())
    
    action = action_factory.get_action(
                                       SpellCastingParameters(self,
                                                              direction = 1, 
                                                              spell_name = 'healing wind'))
    
    assert_that(action, is_not(None))

Nothing too fancy. I’m using ActionFactoryBuilder to create me a configured ActionFactory that has a real implementation of SpellCastingFactory. Code for that is simple and not really interesting.

Following code in SpellCastingFactory makes the test pass:

@logged
def get_action(self, parameters):
    """
    Create a spell casting action

    :param parameters: parameters used to control creation
    :type parameters: SpellCastingParameters
    """
    return SpellCastingAction(caster = parameters.caster, 
                                        direction = parameters.direction,
                                        spell = None)

Very simple again, with no checks or constraints. I’m focusing on developing the simplest and easiest case of spellcasting first. After that has been done, I can go back and start adding the complexity.

SpellCastingAction has an instance attribute spell that I’m setting to None. This attribute will hold the actual Spell that is cast. Most likely I’m going to use the effects subsystem for implementing effects of spells. Next step in implementing the magic system is probably going to deal with spell attribute, so that we can have something else than None in place there.

Code in this step can be found in commit: https://github.com/tuturto/pyherc/commit/adb96f6720ac521039ce6240f9b5242fa678d0ee