Busted or LuaUnit? Answered

I didn't have the time to complete a thorough evaluation of Busted and LuaUnit but have come to the conclusion that I will use Busted. Both Busted and LuaUnit work well and, given Lua's dynamic nature, I found it easy to write tests with them.

I came to my decision after going back to the Stockfetch tests after a gap of two or three months. It was much easier to work out what was being tested from the Busted tests than it was from the LuaUnit ones. It was because of the names that I had given the tests not any failing on LuaUnit's part.

So my choice was not made on a technical basis but because Busted encourages me to write more understandable tests.

I also found, in some cases, Busted's spies, stubs and mocks made for shorter test code than using Lua's flexibility with LuaUnit. On the other hand, LuaUnit tests seem to run much quicker than Busted's. Around ten times faster in my case.

Learning Lua - Test Doubles

I was able to spend some time learning Lua this week and made a little progress with my stockfetch project which am using to answer Busted or LuaUnit?. 

This week I needed to mock some functions to write some tests. Lua, as I had expected, makes it easy to replace functions with test "doubles". I was expecting that Busted would make it easy to "mock" a function. What I found is that Busted definitions of "stub" and "mock" are not the ones that I understand. 

My understanding was based on four types of test "doubles" - fakes, stubs, mocks and spies. Fakes are functions that provide an implementation just for testing that would not be used in production.  Stubs return a known value or perform a known side effect. Mocks are like stubs with the added functionality of being able to query interaction with them. (Such as what arguments they were called with, how often they were called and what they returned.) Spies could be considered as mocks which additionally call the actual function.

From my current understanding, in Busted spies are like the spies that I understand. It seems that Busted stubs appear to be like the mocks with which I am familiar but without the ability to specify a return value (or values). From the docs, Busted mocks are tables whose functions have been wrapped in spies or stubs.

The different terminology isn't a problem but not being able to specify a return value from a test "double" leads to additional work. (It might actually be possible to do so in Busted but I haven't been able to find out how just yet.)

I had expected to "roll my own" test doubles with LuaUnit but having to do part of the job myself with Busted means that there still isn't much to choose between them.

In one test, that I wrote I wanted to check that a function was called five times and have it return 0 each time it was called.

This is how I achieved that using Busted:

        saveProcessTicker = stockfetch.processTicker

    stockfetch.processTicker = function () 

      return 0 

    end 

    spy.on(stockfetch, "processTicker")

    assert.equal(0, stockfetch.run('fixtures/mixedTickers.txt'))

    assert.spy(stockfetch.processTicker).was.called(5)

    stockfetch.processTicker:revert()

    stockfetch.processTicker = saveProcessTicker

This is the LuaUnit equivalent:

    saveProcessTicker = stockfetch.processTicker

  local processTickerCalled = 0

  stockfetch.processTicker = function ()

    processTickerCalled = processTickerCalled + 1 

    return 0 

  end 

  luaunit.assertEquals(stockfetch.run('fixtures/mixedTickers.txt'), 0);

  luaunit.assertEquals(processTickerCalled, 5)

  stockfetch.processTicker = saveProcessTicker

The main difference being needing to "manually" count the number of times that the function was called with LuaUnit.

I will be looking at using test doubles with builtin modules such as IO next.

Preparing to Learn Lua

I took my second step in preparing to learn Lua. I've set up a simple training project through which I can get a feeling of the workflow and practices that will work for me when programming in Lua. I've based the project on the StockFetch example from Test Driving Javascript Applications by Dr. Venkat Subramaniam. (A very worthwhile read).

Before I set up the project, I wanted to see how easy it would be to read the contents of a web page in Lua. It turned out to be very easy once I had installed luasocket.

    sudo luarocks-jit install luasocket

Reading a webpage highlighted, for me, the ability to return multiple values from a Lua function:

   http = require 'socket.http'

    body,code,headers,status = http.request('http://bbc.co.uk')

I also noticed that much Lua is written defensively in the sense that care is taken to minimise opportunities for crashes through the use of the assert function and other such techniques. Clearly, being able to return a status code and a value from a function will help to write robust code.

Knowing how to read websites in Lua, I proceeded to set up the project. I adopted a directory structure that will be familiar to many people but I don't know how common it is in Lua circles.

The basic structure is:

    stockfetch/                           -- main directory

        stockfetch-cli.lua                -- command line interface

        src/                              -- lua modules

            stockfetch.lua                -- main app module

        test/                             -- tests

            stockfetch-busted.lua         -- busted tests

            stockfetch-unit.lua           -- Lua unit tests

After setting up the project, I wanted to work out how best to test its main program, stock watch-cli.lua. It's basically a three line script to initiate the main stockfetch module.

      local stockfetch = require('src/stockfetch');

      local rc = stockfetch.run()

      os.exit(rc)

I'm glad I did as I found out a few things about file paths in Lua that were useful to know from the off:

• Lua retains the current working director setting from the OS. So relative paths calculated in a script can change depending on the directory from which the script was launched
• Another way of looking at that is the Lua does not change the current working directory to that from which a script is loaded
• The dofile function only accepts absolute file paths
• There is no builtin function to "clean" a file path limiting the use of setting a base-file path and a relative path including ../

It soon became apparent that I wasn't going to be able to test stock watch-cli by using docile. (I would be delighted to learn of such a way if there is one.) Instead, I used Lua's command line interfaces to test it.

I found two different ways to run command line programs from Lua - os.execute and io.popen. The first returns the return code from the command executed, the second returns the output from the command. I wanted both, so I chose to use os.execute and redirect the command output to a file. That way I'm able to get both the output and the return code.

I successfully wrote tests in both Busted and LuaUnit using this approach. 

Finally, I loaded the project into a GitHub repository.

Next, I will complete the stockfetch app following a test-driven approach and start to learn how to code in Lua.

Busted or LuaUnit?

Having decided to learn Lua and testing Lua at the same time, I need to decide on a testing framework. There seem to be quite a large number available. I was guided to look at Busted and LuaUnit. Both seem active, professional and have good documentation. They have quite different approaches and it is hard to chose between them without any experience of them. 

I have decided to use them both whilst learning Lua. That decision will, no doubt, slow my learning a little but it should give a greater depth to what I learn.

To get started quickly, a took a function from a Prag Prog Magazine  article "a-functional-introduction-to-lua" and created a module to put it in and wrote both Busted and LuaUnit tests for it.

Here is the module and function:

    local learnLua = {}

    function learnLua.namesOf (things)

      local names = {}

        for index, thing in pairs(things) do

    names[index] = thing.name

  end

      return names

    end

    return learnLua

Here's the Busted test:

    require 'busted.runner'()

    local ll = require 'learnLua'

    describe("Flames with Lua", function()

      describe("namesOf", function()

        it("extract string names", function()

          local cats = {

            { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

          }

          assert.equal("meg", ll.namesOf(cats)[1])

          assert.equal("mog", ll.namesOf(cats)[2])

          assert.equal(2, # ll.namesOf(cats))

        end)

      end)

    end)

Here's the LuaUnit test:

    luaunit = require 'luaunit'

    ll = require 'learnLua'

    local cats = {

      { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

    }

    function testNameExtract() 

      local cats = {

        { name = "meg", breed = "persian" },

  { name = "mog", breed = "siamese" }

      }

      local names = ll.namesOf(cats)

      luaunit.assertEquals(names[1], "meg")

      luaunit.assertEquals(names[2], "mog")

      luaunit.assertEquals(# names, 2)

   end

  os.exit( luaunit.LuaUnit.run() )

Both tests pass:

    $ luajit learn-unit.lua            
    . 
    Ran 1 tests in 0.000 seconds, 1 success, 0 failures    OK

    $ luajit learn-busted.lua    ●
    1 success / 0 failures / 0 errors / 0 pending : 0.000617 seconds

Not a lot to chose between them when testing such a simple function. I need to work up to something more complex to really see the differences between them.

I'd be happy to learn about improvements in style or form that I could make in writing Lua, Busted tests and LuaUnit tests.

There is some early feedback on Busted and LuaUnit in Test Doubles and you can read my conclusion in Busted or LuaUnit? Answered

Installing LuaJIT and Luarocks on OS X

Thanks to Homebrew, installing Lua on OS X is easy. I wanted to install the LuaJIT version as that is the one that I'm likely to be using in the future. It was just a matter of:

    $ brew install luajit

I also wanted to install the Luarocks package manager. My first attempt failed.

Then I found the very helpful instructions at Mesca's Homebrew-Luarocks page. 

Sadly, it is no longer available via Homebrew. I need to find a new method to install Luarocks to use with Luajit