Jupyter + Archimedes

I have been playing with Jupyter and Calysto Hy recently. When it comes to testing (especially with the tools I’m used to), there’s couple hitches: nosetests aren’t executed automatically and any exception that a cell throws stops execution of whole notebook. Like before, this blog post was originally written with Jupyter, because it just makes so easy to interleave code and prose, while making sure that the code is actually runnable.

Continue reading

Jupyter Lab, Calysto Hy, Archimedes… oh my

I recently came across a really nifty looking set of tools, namely: Jupyter Lab and Calysto Hy. Former is continuation of IPython, a notebook based Python authoring tool. It lets you write notebooks online and mix in Python. As such it’s really nice for writing reports, while doing compations required for that report at the same time. Calypso Hy is metakernel, that allows Jupyter Lab (and Jupyter in general) use Hy.

Here I’m going to give Jupyter Lab and Calysto Hy a little test drive, using some familiar libraries: Hypothesis and Archimedes.

Continue reading

Testing Hy macros

I like my code tested and working. I like it even better when it’s done automatically. But recently I was faced with a new kind of code that proved to be rather different beast to tackle: macros. Problem with testing them is that macros usually capture some common, general pattern and give it a name. If one were to use the same approach as with testing function, they would feed specific parameters to macro and assert that the generated code looks like what it should look like. This could soon lead into brittle, hard to maintain tests, since tests would specify very precisely what the generated code should look like. Even minor changes to implementation would break most of the tests, even if the functionality of the generated code wouldn’t change.

Better option (depending on the case again) would be to write code that uses that macro and assert that the code has correct functionality. Depending on the complexity of the macro this might require more than single test in order to cover all possibilities. This is the way I usually test my macros, when I bother testing them at all. Most of the time I just assume that if there are any problems with the macro implementation, tests covering code where it is used will catch problems.

Continue reading

Archimedes

I decided to split off macros used for Hypothesis into their own library called Archimedes. The package isn’t in PyPi yet and might never be there, since it’s not particularly useful as a Python library. One major thing that is still lacking is support for settings, but I don’t foresee that to be too difficult to implement. After that I’ll probably do a little bit cleaning up and call the library finished for now.

Hy, Nose and Hypothesis

I recently came across a tool called Hypothesis that immediately sparked my interest. Their documentation describe it as:

Hypothesis is a Python library for creating unit tests which are simpler to write and more powerful when run, finding edge cases in your code you wouldn’t have thought to look for. It is stable, powerful and easy to add to any existing test suite.

Stable, powerful and easy to add are all words that I like. And if you know me, you quickly guessed that I wanted a Hy interface for this new shiny tool.

Continue reading

Python, Behave and Mockito-Python

This article was originally published in November 2012 issue of Open Source for You. It is republished here under Creative Commons Attribution-Share Alike 3.0 Unported license.

Here’s an article that explores behaviour-driven development with Python for software developers interested in automated testing. Behaviour-driven development approaches the designing and writing of software using executable specifications that can be written by developers, business analysts and quality assurance together, to create a common language that helps different people communicate with each other.

This article has been written using Python 2.6.1, but any sufficiently new version will do. Behave is a tool written by Benno Rice and Richard Jones, and it allows users to write executable specifications. Behave has been released under a BSD licence and can be downloaded from http://pypi.python.org/pypi/behave. This article assumes that version 1.2.2 is being used. Mockito-Python is a framework for creating test doubles for Python. It has been released under an MIT licence, and can be downloaded from: https://bitbucket.org/szczepiq/mockito-python/. This article assumes the use of version 0.5.0, but almost any version will do.

Our project
For our project, we are going to write a simple program that simulates an automated breakfast machine. The machine knows how to prepare different types of breakfast, and has a safety mechanism to stop it when something goes wrong. Our customer has sent us the following requirements for it:

“The automated breakfast machine needs to know how to make breakfast. It can boil eggs, both hard and soft. It can fry eggs and bacon. It also knows how to toast bread to two different specifications (light and dark). The breakfast machine also knows how to squeeze juice from oranges. If something goes wrong, the machine will stop and announce an error. You don’t have to write a user interface; we will create that. Just make easy-to-understand commands and queries that can be used to control the machine.”

Layout of the project
Let’s start by creating a directory structure for our project:

mkdir -p breakfast/features/steps

The breakfast directory will contain our code for the breakfast machine. The features directory is used for storing specifications for features that the customer listed. The steps directory is where we specify how different steps in the specifications are done. If this sounds confusing, don’t worry, it will soon become clear.

The first specification
In the case of Behave, the specification is a file containing structured natural language text that specifies how a given feature should behave. A feature is an aspect of a program—for example, boiling eggs. Each feature can have one or more scenarios that can be thought of as use cases for the given feature. In our example, the scenario is hard-boiling a single egg. First, we need to tackle boiling the eggs. That sounds easy enough. Let’s start by creating a new file in the features directory, called boiling_eggs.feature, and enter the following specification:

Feature: Boiling eggs
  as an user
  in order to have breakfast
  I want machine to boil eggs

Background:
  Given machine is standing by

Scenario: boil hard egg
  Given machine has 10 eggs
  And egg setting is hard
  And amount of eggs to boil is set to 1
  When machine boils eggs
  Then there should be 1 boiled egg
  And eggs should be hard

It does not look like much, but we are just getting started. Notice how the specification reads almost like a story. Instead of talking about integers, objects and method calls, the specification talks about machines, eggs and boiling. It is much easier for non-coders to read, understand and comment on this kind of text. At this point, we can already try running our tests, by moving into the breakfast directory and issuing the command behave. Behave will try to run our first specification, and will fail because we have neither written steps or implemented the actual machine. Because Behave is really helpful, it will tell us how to proceed:

You can implement step definitions for undefined steps with these snippets:

@given(u'machine is standing by')
def impl(context):
    assert False

Now we need to define what is to be done when the breakfast machine is standing by and has 10 eggs ready to be boiled. Create the file eggs.py in the steps directory and add the following code in it:

from breakfast import BreakfastMachine
from behave import *

@given(u'machine is standing by')
def impl(context):
    context.machine = BreakfastMachine()

@given(u'machine has {egg_amount} eggs')
def impl(context, egg_amount):
    context.machine.eggs = int(egg_amount)

Steps are the bridge between the specification and the system being tested. They map the natural-language sentences into function calls that the computer can understand.

The first function defines what will happen when there should be a breakfast machine standing by. Let’s create a new instance of BreakfastMachine and store it in context, which is a special object that Behave keeps track of. It is passed from step to step, and can be used to relay information between them. Eventually, we will use it to assert that the specification has been executed correctly.

This defines code that is executed when there is a step ‘machine has x eggs’, where x can be anything (in our example it is 10). {egg_amount} is automatically parsed and passed as a parameter to the function, which has to have an identically named parameter. Note that the parameters are Unicode strings, and thus need to be converted to integers in our example.

If we were to run Behave at this point, we would get an error message that BreakfastMachine cannot be imported. This, of course, is because we have not yet written it. It might feel strange to start coding from this end of the problem (specifications and tests), instead of diving headlong into coding BreakfastMachine. The advantage of approaching the task from this direction is that we can first think about how we would like our new object to behave and interact with other objects, and write tests or specifications that capture this. Only after we know how we would like to use the new object do we start writing it.

In order to continue, let’s create BreakfastMachine in the file breakfast.py and save it in the breakfast directory. This is the class our client asked us to write and which we want to test:

class BreakfastMachine(object):

    def __init__(self):
        super(BreakfastMachine, self).__init__()
        self.eggs = 0
        self.egg_hardness = 'soft'
        self.eggs_to_boil = 0
        self.boiled_eggs = 0
        self.boiled_egg_hardness = None

    def boil_eggs(self):
        pass

Steps to implement a hardness setting for eggs and the amount of eggs to boil are quite similar to setting the total amount of eggs available. The difference is that we are not creating a new BreakfastMachine, but using the one that has been stored in context earlier. This way, we configure the machine step by step, according to the specification. You can keep running Behave after each addition to eggs.py to see what kind of reports it will output. This is a good way of working, because Behave guides you regarding what needs to be done next, in order to fulfil the specification. In eggs.py, add the following:

@given(u'egg setting is {egg_hardness}')
def impl(context, egg_hardness):
    context.machine.egg_hardness = egg_hardness

@given(u'amount of eggs to boil is set to {amount}')
def impl(context, amount):
    context.machine.eggs_to_boil = int(amount)

Up to this point, we have been configuring the breakfast machine. Now it is time to get serious and actually instruct the machine to boil our eggs, and verify afterwards that we got what we wanted. Add the following piece of code to eggs.py:

@when(u'machine boils eggs')
def impl(context):
    context.machine.boil_eggs()

@then(u'there should be {amount} boiled egg')
def impl(context, amount):
    assert context.machine.boiled_eggs == int(amount)

@then(u'eggs should be {hardness}')
def impl(context, hardness):
    assert context.machine.boiled_egg_hardness == hardness

The first step is to instruct our breakfast machine to boil eggs. The next two are to ascertain that the results of boiling are what we wanted. If you run Behave at this point, an error will be displayed, because the machine does not yet know how to boil eggs. Change breakfast.py to the following final version, and the tests should pass:

class BreakfastMachine(object):

    def __init__(self):
        super(BreakfastMachine, self).__init__()
        self.eggs = 0
        self.egg_hardness = 'soft'
        self.eggs_to_boil = 0
        self.boiled_eggs = 0
        self.boiled_egg_hardness = None

    def boil_eggs(self):
        if self.eggs_to_boil <= self.eggs:
            self.boiled_eggs = self.eggs_to_boil
            self.boiled_egg_hardness = self.egg_hardness
            self.eggs = self.eggs - self.eggs_to_boil
        else:
            self.boiled_eggs = 0
            self.boiled_egg_hardness = None

Now we have a passing scenario and some code; so what’s next? You could experiment with what we have now, and see if you can write another scenario and try to soft-boil an egg, or boil multiple eggs. Just add a new scenario in boiling_eggs. feature after the first one, leaving an empty line in between. Do not repeat the feature or background sections—those are needed only once. After experimenting for a bit, continue to the second specification.

The second specification
Let’s start by reviewing the customer’s requirements listed at the beginning of this article.

Boiling varying numbers of eggs to different specifications, i.e., hard or soft, is taken care of. Frying and toasting should be easy to implement, since it is similar to boiling eggs. “Just make easy-to-understand commands and queries that can be used to control the machine,” catches our attention, and we ask for a clarification from our customer, who sends us an API specification that tells us how their user interface is going to interact with our machine. Since they are still working on it, we’ve got only the part that deals with eggs. The following is for ui.py in the breakfast directory:

class BreakfastUI(object):

    def __init__():
        super(BreakfastUI, self).__init__()

    def eggs_boiled(self, amount, hardness):
        pass

    def error(self, message):
        pass

Their user interface is not ready yet, and their API specification is not completely ready either. But we got some of it, and it is good enough for us to start working with.

We can first tackle sending error messages to the user interface. Let’s add the following code at the end of boiling_eggs.feature:

Scenario: boiling too many eggs should give an error
  Given machine has 1 eggs
  And amount of eggs to boil is set to 5
  When machine boils eggs
  Then there should be error message "not enough eggs"

The next step, like before, is to implement new steps in
eggs.py:

from breakfast import BreakfastMachine
from ui import BreakfastUI
from mockito import verify, mock
from behave import *

@given(u'machine is standing by')
def impl(context):
    context.ui = mock(BreakfastUI)
    context.machine = BreakfastMachine(context.ui)
    ...

@then(u'there should be error message "{message}"')
def impl(context, message):
    verify(context.machine.ui).error(message)

We also need to modify our BreakfastMachine to connect to the user interface when it starts up. We do this by modifying the __init__ method in breakfast.py as per the following:

def __init__(self, ui):
    super(BreakfastMachine, self).__init__()
    self.ui = ui
    self.eggs = 0
    self.egg_hardness = 'soft'
    self.eggs_to_boil = 0
    self.boiled_eggs = 0
    self.boiled_egg_hardness = None

There are two interesting bits in eggs.py. The first is in the method where we set the machine to stand by. Instead of using a real BreakfastUI object (which wouldn’t have any implementation anyway), we create a test double that looks like BreakfastUI, but does not do anything when called. However, it can record all the calls, their parameters and order.

The second interesting part is the function where we verify that an error message has been delivered to the UI. We call verify, pass the UI object as a parameter to it, and then specify which method and parameters should be checked. Both verify and mock are part of Mockito, and offer us tools to check the interactions or the behaviour of objects.

If we run Behave after these modifications, we are going to get a new error message, as shown below:

Then there should be error message "not enough eggs" #
features\steps\eggs.py:35
Assertion Failed:
Wanted but not invoked: error(u'not enough eggs')

This tells us that the specification expected a call to the method error, with the parameter ‘not enough eggs’. However, our code does not currently do that, so the specification fails. Let’s fix that and modify how the machine boils eggs (breakfast.py):

def boil_eggs(self):
    if self.eggs_to_boil <= self.eggs:
        self.boiled_eggs = self.eggs_to_boil
        self.boiled_egg_hardness = self.egg_hardness
        self.eggs = self.eggs - self.eggs_to_boil
    else:
        self.boiled_eggs = 0
        self.boiled_egg_hardness = None
        self.ui.error('not enough eggs')

Let’s add a call to the UI object’s error method in order to let the user interface know that there was an error, and that the user should be notified about it. After this modification, Behave should run again without errors and give us a summary:

1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
12 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.0s

There is a distinction between these two approaches. In the first one, we wanted to verify the state of the system after it boiled the eggs. We checked that the amount of eggs boiled matched our specification, and that they were of the correct hardness.

In the second case, we did not even have a real user interface to work with. Instead of writing one from scratch in order to be able to run tests, we created a test double: an object that looks like a real user interface, but isn’t one. With this double, we could verify that the breakfast machine calls the correct methods with the correct parameters.

Now we have the basics of boiling eggs. If you want, you can continue from here and add more features to the breakfast machine, like frying eggs and bacon, and squeezing juice out of oranges. Try and add a call to the user interface after the eggs have been boiled, and verify that you are sending the correct number of eggs and as per the specified hardness.

After getting comfortable with the approaches and techniques shown here, you can start learning more about Behave and Mockito-Python. Good places to start are their download pages, as both offer short tutorials.

In this brief example, we had a glimpse of several different ways of writing executable specifications. We started our project from a natural-language file that specified how we wanted our machine to boil eggs. After this specification, we wrote an implementation for each step, and used that to help us to write our actual breakfast machine code. After learning how to verify the state of a system, we switched our focus to verifying how objects behave and communicate with each other. We finished with a breakfast machine that can boil a given number of soft or hard-boiled eggs, and that will issue a notification to the user interface in case there are not enough eggs in the machine.

Note: Example code in this article is available as a zip archive at https://github.com/tuturto/breakfast/zipball/master.

new rat AI in works

AI routines for enemies in Herculeum are nothing fancy, mostly just random movement until the player happens to wander too close. The plan is to have each monster to behave in a semi-predictable, distinct way (think monster in Super Mario Bros.). First guinea pig for AI experimentation will be rat. The plan is to make them search for closest wall and then walk alongside the wall, randomly turning back when they arrive a corner. If player happens to come too close, the rat will attack. I have not yet decided how will they react on ranged attacks, but probably they are going to flee.

To make things more interesting, I’m going to write AI routines using Hy.

As always, I started with little bit doodling and then writing couple of tests:

(import [pyherc.test.builders [LevelBuilder CharacterBuilder]]
	[pyherc.ai.rat [next-to-wall?]]
	[hamcrest [assert-that is- is-not :as is-not- none has-items]])

(defn test-empty-space-is-detected [] 
  "test that an empty space is not reported as wall"
  (let [[character (-> (CharacterBuilder)
		               (.build))]
	[level (-> (LevelBuilder)
               (.with-floor-tile :floor)
		       (.with-wall-tile :empty-wall)
		       (.with-empty-wall-tile :empty-wall)
		       (.with-solid-wall-tile :solid-wall)
		       (.with-character character (, 5 9))
		       (.build))]
	[wall-info (next-to-wall? character)]]
    (assert-that wall-info (is- (none)))))

(defn test-wall-is-detected [] 
  "test that a wall can be detected next to a given point"
  (let [[character (-> (CharacterBuilder)
		               (.build))]
	[level (-> (LevelBuilder)
               (.with-floor-tile :floor)
		       (.with-wall-tile :empty-wall)
	    	   (.with-empty-wall-tile :empty-wall)
    		   (.with-solid-wall-tile :solid-wall)
		       (.with-wall-at (, 4 10))
		       (.with-wall-at (, 5 10))
    		   (.with-wall-at (, 6 10))
	    	   (.with-character character (, 5 9))
		       (.build))]
	[wall-info (next-to-wall? character)]]
    (assert-that wall-info (is-not- (none)))
    (let [[wall-direction (get wall-info :wall-direction)]]
      (assert-that wall-direction has-items [:east :west]))))

The threading macro (->) makes it breeze to use all those builders that I have coded in Python. Without it I would have to write very deeply nested function calls. I’m also very happy about how clean and readable Hamcrest verifications are. It’s really hard to get closer to natural language in a standard programming language.

satin-python is now available at PyPi

PyPi is a Python package index (also known as a cheeseshop). Satin is a UI testing library that I have been tinkering with on and off while developing Herculeum. I have now released version 0.1.0 of Satin in PyPi, in order to make it easier for others to download it.

The change is reflected in requirements-dev file of Herculeum. Now it is possible to download almost all dependencies of the game in just a single command:


pypi install -r requirements-dev

Running nosetests for Hy

Getting nosetest to work with Hy was easier than I thought in the end. The trick I used was to create a Python module that will act as an entry point. In that module I first import Hy, which in turn lets me to import my test modules:

import hy
from .test_person_repository import *

After that I can run nosetests and end up with:

tuukka@new-loft:~/programming/python/sandvalley/src/sandvalley/tests$ nosetests
..
----------------------------------------------------------------------
Ran 2 tests in 0.015s

OK

Pretty nifty I must say. Having to import everything from modules in a entry point module is a bit hassle, but that’s something that I should be able to automate.

The same method can be used to call Hy code from Python program.