Testing with Hy and Nose

I have been using Nose for the longest time and it’s my go-to tool for running tests. Nose is very no-nonsense and has very sensible defaults (to me at least).

Writing a test in Hy that gets discovered and executed with Nose is pretty straightforward. One just needs a correct naming convention and everything happens more or less automatically. However, by applying some standard lisp practices, one can cut down amount of the code needed quite a bit. In this post, I’ll present how my style of writing tests have evolved over time and what kind of measures I took in order to write minimum amount of code.

Following is an example of two tests written with Python. They’re used to test that unknown items are named after their appearance and that character can give names to groups of items (calling all healing potions to doozer potions).

class TestItems():
    """
    Testing more advanced features of item class
    """
    def __init__(self):
        """
        Default constructor
        """
        self.character = None
        self.item = None

    def setup(self):
        """
        Setup test case
        """
        self.character = (CharacterBuilder()
                            .build())

        self.item = (ItemBuilder()
                         .with_name('healing potion')
                         .with_appearance('blue potion')
                         .build())

    def test_appearance_of_unknown(self):
        """"
        Test that appearance is reported for an unknown item
        """
        name = self.item.get_name(self.character)

        assert(name == 'blue potion')

    def test_appearance_of_generic_named_item(self):
        """
        Test that given name is reported for a generally named item
        """
        self.character.item_memory['healing potion'] = 'doozer potion'

        name = self.item.get_name(self.character)

        assert_that(name, is_(equal_to('doozer potion')))

If this code is translated one to one on Hy, result is something like following:

(defclass TestItems []
  "Testing more advanced features of item class"
  [[--init-- (fn [self]
                "Default constructor"
                (setv self.character nil)
				(setv self.item nil)
				nil)]
   [setup (fn [self]
             "Setup test case"
             (setv self.character (-> (CharacterBuilder)
			                          (.build)))
			 (setv self.item (-> (ItemBuilder)
			                     (.with-name "healing potion")
								 (.with-appearance "blue potion")
								 (.build))))]
   [test-appearance-of-unknown 
    (fn [self]
       "Test that appearance is reported for an unknown item"
       (assert (= (.get-name self.item self.character) "blue potion")))]

   [test-appearance-of-named-item
    (fn [self]
       "Test that given name is reported for a generally named item"
       (assoc self.character.item-memory "healing potion" "doozer potion")
       (assert (= (.get-name self.item self.character) "doozer potion")))]])

Nothing particularly fancy or groundbreaking here, but well functioning set of tests with a setup and test phases.

Since I’m not a huge fan of writing classes where they aren’t absolutely needed, I quickly learned that Nose can operate on plain functions as easily as with test classes. Having a setup phase is not automatic, but one needs to manually do that at the beginning of each test. Following code is the next step of evolution:

(defn setup []
  "Setup test case"
  (let [[character (-> (CharacterBuilder)
                       (.build))]
        [item (-> (ItemBuilder)
		          (.with-name "healing potion")
				  (.with-appearance "blue potion")
				  (.build))]]
     {:character character
	  :item item}))

(defn test-appearance-of-unknown []
  "Test that appearance is reported for an unknown item"
  (let [[context (setup)]
        [character (:character context)]
		[item (:item context)]]
    (assert (= (.get-name item character)
			   "blue potion"))))
			   
(defn test-appearance-of-named-item []
  "Test that given name is reported for a generally named item"
  (let [[context (setup)]
        [character (:character context)]
		[item (:item context)]]
    (assoc character.item-memory "healing potion" "doozer potion")
	(assert (= (.get-name item character)
			   "doozer potion"))))

The idea is that while Nose can automatically execute each function starting with word “test”, those functions can call setup function. This setup function will create data that is needed for test and return them in a dictionary. Test function can then retrieve the data and place it in local scope, before starting the execution. No need to define a separate class, but we still can use setup function to share data between multiple tests. Advantage here is that if the setup function doesn’t muck about with global data, tests should be more isolated from each other.

There’s still some duplication going on here of course. Each test is a function with a doc string (the doc string is used by nose to report if the test passed or not) and there’s an explicit call to setup function (only in those tests that need setup of course). Managing the local scope requires let form that pulls values from the context dictionary.

This is the point where some macros entered in play. I identified three major parts that kept repeating: test function with a doc string, setup and setting local variables to values provided by setup.

First I tackled the easiest one, which is making a test function:

(defmacro fact [desc &rest code]
  (let [[fn-name (HySymbol (.join "" ["test " desc]))]]
    `(defn ~fn-name []
       ~desc
       ~@code)))

Fact macro takes description of the test and one or more lines of actual test code. It then outputs a function, which is named to “test ” followed by the description (funny thing, Python doesn’t mind if you have a function with spaces in its name) and body of code.

Second step was to write a macro that can crate setup functions:

(defmacro background [context-name &rest code]
  (let [[symbols (ap-map (first it) code)]
        [fn-name (HySymbol (.join "" ["setup_" context-name]))]]
    `(defn ~fn-name []
       ~(.join "" ["setup context " context-name])
       (let [~@code]
         ~(dict-comp (keyword x) x [x symbols])))))

This is quite a bit more complex and took a while to get (mostly) correct. This macro takes name of the setup function and arbitrary amount of two element lists. Each list has symbol as first item and value as second. Macro will then generate a function that creates a dictionary, with keywordified symbols as keys and their values as values (pretty much same as the earlier setup function did).

Having setup function without a way to call it isn’t useful, so the third macro takes care of that:

(defmacro/g! with-background [context-name symbols &rest code]    
  (let [[fn-name (HySymbol (.join "" ["setup_" context-name]))]]
    `(let [[~g!context (~fn-name)]
           ~@(ap-map `[~it (get ~g!context ~(keyword it))] symbols)]
       ~@code)))

with-background macro creates a let binding, calls provided setup function and pulls specified symbols in the local scope. After that the test code can be executed (funny thing, one can have a test with multiple backgrounds. I haven’t found a reason to do so, but it’s possible.)

After having these macros at my disposal, previous code was modified to look like this:

(background potions
            [potion (-> (ItemBuilder)
                        (.with-name "healing potion")
                        (.with-appearance "blue potion")
                        (.build))]
            [character (-> (CharacterBuilder)
                           (.build))])

(fact "unknown items are named after their appearance"
      (with-background potions [potion character]
        (assert-that (.get-name potion character) (is- (equal-to "blue potion")))))

(fact "items named by character are reported by their given name"
      (with-background potions [potion character]
        (assoc character.item-memory "healing potion" "doozer potion")
        (assert-that (.get-name potion character)
                     (is- (equal-to "doozer potion")))))

Functionality of the tests is pretty much the same, but there’s a lot less of technical details of testing framework visible. They could be even more conscise, if I were to write some hamcrest matchers, but I haven’t gotten around that yet. This probably isn’t the final form yet, but will evolve a bit in the future, when I learn more about coding and writing tests.

Advertisements

One thought on “Testing with Hy and Nose

  1. Pingback: Hy, Nose and Hypothesis | Engineer's Journey

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s