Busted or LuaUnit? Answered

I didn't have the time to complete a thorough evaluation of Busted and LuaUnit but have come to the conclusion that I will use Busted. Both Busted and LuaUnit work well and, given Lua's dynamic nature, I found it easy to write tests with them.

I came to my decision after going back to the Stockfetch tests after a gap of two or three months. It was much easier to work out what was being tested from the Busted tests than it was from the LuaUnit ones. It was because of the names that I had given the tests not any failing on LuaUnit's part.

So my choice was not made on a technical basis but because Busted encourages me to write more understandable tests.

I also found, in some cases, Busted's spies, stubs and mocks made for shorter test code than using Lua's flexibility with LuaUnit. On the other hand, LuaUnit tests seem to run much quicker than Busted's. Around ten times faster in my case.

HTTPS with Lua Sockets

I noticed that a couple of people a week are reading Busted or LuaUnit?. If they were looking for some comparison, they would have been disappointed. Especially as they didn't look at Lua Test Doubles which does give some feedback on Busted and LuaUnit.

I've started out to complete the test-driven development of the stockfetch program using both Busted and LuaUnit. The next step in the project is to read the stock prices from Yahoo. On the surface this didn't seem to be difficult as when I tried GETting a webpage using Lua Socket, it was pretty simple.

But I found that the Lua Socket HTTP "simple" form is unable to handle https. To GET data from an https source you need to use the Lua Socket HTTP "generic" form and the Lua Sec module. Lua Socket HTTP generic form uses LTN12 sinks to collect the content rather than returning a simple string.

Here is a simple example of GETting the content of the Lua Home Page. I hope it is sufficiently self-explanatory.

local https = require('ssl.https')

local ltn12 = require('ltn12')

local responseContent = {}

local returnCode, httpResponseCode, httpHeaders, htttpStatus = https.request{

  url = 'https://www.lua.org',

  sink = ltn12.sink.table(responseContent)

}

if returnCode then

  print('httpResponseCode: ' .. httpResponseCode)

  print('date: ' .. httpHeaders.date)

  print('server: ' .. httpHeaders.server)

  print('last-modified: ' .. httpHeaders['last-modified'])

  print('content-length: ' .. httpHeaders['content-length'])

  print('content-type: ' .. httpHeaders['content-type'])

  print(table.concat(responseContent));

else

  print(httpResponseCode)       -- this will contain a lua socket error message

end

Note: I used Lua Rocks to install the Lua Sec module. It requires OpenSSL to be installed. If you running Lua under macOS, you might find these instructions helpful. I did. 

Finding a Hyphenated Sub-String of a String in Lua

The Lua string module has a find function which locates a specified sub-string with a longer string. It seems intuitive that the function looks for a string within a string.

    > str = 'A string containing abc-def amongst other things'

    > =string.find(str, 'amongst')

    29    35

However, what isn't clear from that example that the second argument to Lua's find function is not a simple string but a pattern. A '-' is a "reserved" symbol in a Lua pattern so if you simply search for a string containing one, you will most probably not find it.

    > =string.find(str, 'abc-def')

    nil

The '-' character  in a pattern means "Match the previous character (or class) zero or more times, as few times as possible." It can be "escaped" with the '%' character.

    > =string.find(str, 'abc%-def')

    21   27

Alternatively, the find function accepts a fourth parameter which turns off pattern matching and your search will return the expected result. (In order to use it, you must also supply the third parameter which tells the find function where to start its search).

    > =string.find(str, 'abc-def', 1, true)

    21        27

Lua Error Handling

Like many things in Lua, error handling is simple and straightforward and a little different from many languages. I am writing these simple notes to act as a reminder of those differences.  

There are two functions in Lua that allow you to trap errors, pcall and xpcall. The 'p'  seems to stand for protected, I haven't been able to make a good guess as to the 'x'. The difference between the two is that xpcall allows you to supply your own error handling function.

The main difference is that pcall and xpcall are functions rather than statements. You need to pass the code you wish to protect to them as an argument. You can't just wrap your code in try blocks as you can in many languages. You can't pass code blocks (chunks in Lua terminology) as function arguments in Lua. You need to wrap the code in a function. (It can be anonymous.)

A consequence of this is that you need to remember to pass the function and not the result of the function. A mistake that I found easy to make:

    > function f() return 1 end

    > =pcall(f())

    false attempt to call a number value

It should have been:

    > =pcall(f)

    true 1

You have probably noticed that pcall returns multiple values, the first is a status (true for okay, false if an error occurred) and then either the values returned from your function or from the error function. For example:

    > function fe() error('there was an error') end

    > =pcall(fe)

    false stdin:1: there was an error

You pass arguments to your function by supplying them as additional arguments to pcall:
    function errDemo (i, j) 
      if 'number' ~= type(i) or 'number' ~= type(j) then 
        error{msg='catch this'}
      end
      return i, j
    end

    > =pcall(errDemo, 1, 2)

    true       1 2

    > status, err = pcall(errDemo, "1", 2)

    > =status

    false

    > =err.msg

    catch this

Lastly, xpcall takes an additional function, passed as second argument, that will be called when an error occurs. It is passed object that is returned by the error function. Here is a simple example:

    > function handleError (err) return 'caught you' end

    > =xpcall(errDemo, handleError, "1", 2)
    false caught you

Installing Lua LGI on macOS

I wanted to see how easy it would be to build some very simple GUI apps in Lua. As I already have GTK+3 installed on my machine, I thought it would be a good idea to try to use it from Lua. LGI is gobject-introspection based dynamic Lua binding to GObject based libraries including GTK+3. It can be installed via Luarocks.

A simple attempt at installing LGI results in an error as Luarocks can't find the libffi library that LGI requires. It is a similar problem to the one I encountered with installing the Ruby GTK+3 Gem.  Applying a similar solution to the one kindly supplied by the Ruby Gnome 2 team did the trick.

This is the command that worked:


  sudo PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig luarocks-jit install lgi

Notes:

I installed GTK+3 using Homebrew

I am using LuaJIT 2.0.4

I am using luarocks-jit 2.3.0

Parsing Rebol Data with Lua PEG

I diverted myself from learning lua by working through the stockfetch program as I wanted to test how tricky it would be to read a Rebol data file into a Lua table using Lua's Parsing Expression Grammars (LPeg). I didn't attempt to write an actual conversion tool, just something that would give a good feeling for how difficult it would be to write such a tool.

It is necessary to understand that there really isn't such a thing as a Rebol data file. In Rebol, code is data and data is code. Any syntactically valid Rebol file can be evaluated, if a Rebol file contains functions they will be run. (The same is true of the languages descended from Rebol - Red, Boron and World.) 

Whilst functions are first class values in Lua, I don't think that variables can be treated so in Lua (though I could well be wrong). 

If I am correct, this means that it will only ever be possible to convert a subset of Rebol files to Lua tables.

For my purposes, I felt it was sufficient to confirm that both Rebol objects and blocks could be converted into Lua tables. 

When considering objects, I excluded the possibility of them containing functions or code executed as the object was loaded. For this test, a Rebol object was restricted to be akin to a keyword:value store.

Rebol blocks are ordered series of values, somewhat similar to a Lua table using integer keys.

I also applied the following restrictions for my "proof of concept":

• Words can only be a single letter - a to z
• Values can either be a block, an object or a single digit string - 0 to 9
• Blocks and Objects cannot be empty

Once I started to understand LPeg, I found writing the trial convertor to be quite straightforward. I'm happy with the solution that I came up with as it appears very readable to me (especially compared with regex). I'm sure that it could be improved by someone more familiar with Lua and LPeg than I currently am.

Here's my function:

function reboldata.import (reb)
  local bind = function (...)
    local args = {...}
    local t = {}
    for i = 1, #args, 2 do
      t[args[i]] = args[i + 1] 
    end
    return t
  end
  local End = lpeg.P(-1)
  local Space = lpeg.S(' \n\t')^0
  local OpenBlock = Space * '['
  local CloseBlock = ']' * Space
  local EndString = lpeg.S(' \n\t') + CloseBlock
  local String = Space * lpeg.C(1 - EndString)^1 * Space
  local NotCloseBlock = (1 - CloseBlock)^0
  local Word = lpeg.R('az')
  local GetWord = lpeg.C(Word) * ':' * Space
  local Value = lpeg.R('09')
  local Element = (String + lpeg.V'Block') + lpeg.V'Object'
  local GetWordValue = GetWord * 
                       ((lpeg.C(Value) + lpeg.V'Block') + lpeg.V'Object') * Space
  local ObjectContent = (GetWordValue^0 / bind) + CloseBlock
  local ParseRebol = lpeg.P{
    'PR';
    PR = lpeg.Ct(lpeg.V'BO'),
    BO = (lpeg.V'O' + lpeg.V'B') + End,
    O = (lpeg.V'Object' * lpeg.V'B') + lpeg.V'Object',
    B = lpeg.V'Block' * lpeg.V'BO',
    Object = lpeg.P('make object!' * Space * 
             OpenBlock * ObjectContent * CloseBlock),
    Block = lpeg.Ct(OpenBlock * Element^0 * CloseBlock)
  }
  return lpeg.match(ParseRebol, reb)
end

Full source and tests

Lua Patterns, Red Parse, and Regex

In my Learning Lua project, I needed to extract a four character code from a string. Many languages use a builtin Regex implementation to provide the capability to match and extract patterns from strings. I am also familiar with Red, which like Rebol before it, provides the Parse Dialect (a Domain Specific Language (DSL)) which not only has excellent string matching and extraction features but makes it easy to write further DSLs. 

I found that Lua has another approach. It has a built-in lightweight pattern language, known as Patterns. From what I understand, it was designed to be simpler than Regex both in its "grammar" and its capability. (Lua does have Regex modules available. It also has the very interesting LPeg pattern-matching library, based on Parsing Expression Grammars (PEGs).)

I thought it would be interesting to compare the three approaches, patterns, parse and regex. Here are three versions of a short function to extract the first four consecutive alphabetic characters from a string. The functions return an empty string if there is no match.

First Regex, courtesy of Ruby:

    def parse_line line

      ticker = line.match(/[A-Za-z]{4,4}/)

      if ticker then

        ticker[0]

      else

        ''

      end

    end

Note: 

    I'm no Regex expert, there may well  be a better way, and probably a more Rubyish way too.

Second Parse, courtesy of Red

parse-line: function [line [string!] /local ticker][

alpha: charset [#"A"-#"Z" #"a"-#"z"] parse line [to copy ticker 4 alpha] any [ticker ""] ]

Notes: 

    1. The charset allows you to efficiently define set of characters to be matched.

    2. The parse function apply the rules provided between the []s to the input.

    3. A parse rule can have any number of sub-rules.

    4. Red Parse details.

Finally Patterns, courtesy of Lua

    function stockfetch.parseLine (line)

      local match = string.match(line, '(%a%a%a%a)')

      if match then 

        return match

      else

        return ''

      end

    end

There doesn't seem much to chose between the approaches for such a simple task. The Lua pattern is very readable but you do have to know (or lookup) that %a matches an alphabetic character. The Regex is self-explanatory once you understand Regex syntax. The Red parse rule may seem a little more complicated because it also specifies to where the data are extracted (copy ticker) and has to ensure that the rules cover the whole supplied string (to end).

It will be interesting to see how the three approaches compare with more challenging requirements. Perhaps, I'll be able to add LPeg into the equation too.

Note: I am a strong supporter of and contributor to Red and am highly biased towards its Parse DSL. 

Learning Lua - Stubbing, Mocking and Spying on Functions in Testing

I was very pleased to find that it is fully possible to overwrite existing functions in Lua.  It even applies to those built-in functions that can feel as though they are part of the language. I guess that they are actually functions that are defined in the global scope.

Here's a quick example:

    > a = 1

    > sp = print

    > sp(a)

    1

    > print = function () a = 2 end

    > print()

    > sp(a)

    2

    > print = sp

    > print(a)

    2

This allows full stubbing, mocking and spying on functions during testing, albeit with a little work. You simply save the function you want to stub, mock or spy on and write a function to act as a stub, mock or spy. You can make use of Lua's upvalues to be able to count how many times the function was called in a test.

Here's an example of mocking a function in a test using LuaUnit:

    function testMocking ()

        saveParseLine = t.parseLine

        local parseLineCalled = 0

        t.parseLine = function (line)

            parseLineCalled = parseLineCalled + 1

            luaunit.assertEquals(line, 'AAPL')

            return 'AAPL' 

        end

        luaunit.assertEquals(funcThatCallsParseLine(data), 0);

        luaunit.assertEquals(parseLineCalled, 1) 

        t.parseLine =  saveParseLine

    end

Learning Lua - Test Doubles

I was able to spend some time learning Lua this week and made a little progress with my stockfetch project which am using to answer Busted or LuaUnit?. 

This week I needed to mock some functions to write some tests. Lua, as I had expected, makes it easy to replace functions with test "doubles". I was expecting that Busted would make it easy to "mock" a function. What I found is that Busted definitions of "stub" and "mock" are not the ones that I understand. 

My understanding was based on four types of test "doubles" - fakes, stubs, mocks and spies. Fakes are functions that provide an implementation just for testing that would not be used in production.  Stubs return a known value or perform a known side effect. Mocks are like stubs with the added functionality of being able to query interaction with them. (Such as what arguments they were called with, how often they were called and what they returned.) Spies could be considered as mocks which additionally call the actual function.

From my current understanding, in Busted spies are like the spies that I understand. It seems that Busted stubs appear to be like the mocks with which I am familiar but without the ability to specify a return value (or values). From the docs, Busted mocks are tables whose functions have been wrapped in spies or stubs.

The different terminology isn't a problem but not being able to specify a return value from a test "double" leads to additional work. (It might actually be possible to do so in Busted but I haven't been able to find out how just yet.)

I had expected to "roll my own" test doubles with LuaUnit but having to do part of the job myself with Busted means that there still isn't much to choose between them.

In one test, that I wrote I wanted to check that a function was called five times and have it return 0 each time it was called.

This is how I achieved that using Busted:

        saveProcessTicker = stockfetch.processTicker

    stockfetch.processTicker = function () 

      return 0 

    end 

    spy.on(stockfetch, "processTicker")

    assert.equal(0, stockfetch.run('fixtures/mixedTickers.txt'))

    assert.spy(stockfetch.processTicker).was.called(5)

    stockfetch.processTicker:revert()

    stockfetch.processTicker = saveProcessTicker

This is the LuaUnit equivalent:

    saveProcessTicker = stockfetch.processTicker

  local processTickerCalled = 0

  stockfetch.processTicker = function ()

    processTickerCalled = processTickerCalled + 1 

    return 0 

  end 

  luaunit.assertEquals(stockfetch.run('fixtures/mixedTickers.txt'), 0);

  luaunit.assertEquals(processTickerCalled, 5)

  stockfetch.processTicker = saveProcessTicker

The main difference being needing to "manually" count the number of times that the function was called with LuaUnit.

I will be looking at using test doubles with builtin modules such as IO next.

Lua inf, -inf and nan

The Lua number datatype has the conventional three special values inf (positive infinity), -inf (negative infinity) and nan (not a number). It seems not to be so conventional when it comes to comparing these values. It returns true when comparing inf with inf, -inf with -inf, and nan with nan. I believe that mathematically this is not quite correct. However, I can see the advantages of being able to compare them from a programming perspective.

Preparing to Learn Lua

I took my second step in preparing to learn Lua. I've set up a simple training project through which I can get a feeling of the workflow and practices that will work for me when programming in Lua. I've based the project on the StockFetch example from Test Driving Javascript Applications by Dr. Venkat Subramaniam. (A very worthwhile read).

Before I set up the project, I wanted to see how easy it would be to read the contents of a web page in Lua. It turned out to be very easy once I had installed luasocket.

    sudo luarocks-jit install luasocket

Reading a webpage highlighted, for me, the ability to return multiple values from a Lua function:

   http = require 'socket.http'

    body,code,headers,status = http.request('http://bbc.co.uk')

I also noticed that much Lua is written defensively in the sense that care is taken to minimise opportunities for crashes through the use of the assert function and other such techniques. Clearly, being able to return a status code and a value from a function will help to write robust code.

Knowing how to read websites in Lua, I proceeded to set up the project. I adopted a directory structure that will be familiar to many people but I don't know how common it is in Lua circles.

The basic structure is:

    stockfetch/                           -- main directory

        stockfetch-cli.lua                -- command line interface

        src/                              -- lua modules

            stockfetch.lua                -- main app module

        test/                             -- tests

            stockfetch-busted.lua         -- busted tests

            stockfetch-unit.lua           -- Lua unit tests

After setting up the project, I wanted to work out how best to test its main program, stock watch-cli.lua. It's basically a three line script to initiate the main stockfetch module.

      local stockfetch = require('src/stockfetch');

      local rc = stockfetch.run()

      os.exit(rc)

I'm glad I did as I found out a few things about file paths in Lua that were useful to know from the off:

• Lua retains the current working director setting from the OS. So relative paths calculated in a script can change depending on the directory from which the script was launched
• Another way of looking at that is the Lua does not change the current working directory to that from which a script is loaded
• The dofile function only accepts absolute file paths
• There is no builtin function to "clean" a file path limiting the use of setting a base-file path and a relative path including ../

It soon became apparent that I wasn't going to be able to test stock watch-cli by using docile. (I would be delighted to learn of such a way if there is one.) Instead, I used Lua's command line interfaces to test it.

I found two different ways to run command line programs from Lua - os.execute and io.popen. The first returns the return code from the command executed, the second returns the output from the command. I wanted both, so I chose to use os.execute and redirect the command output to a file. That way I'm able to get both the output and the return code.

I successfully wrote tests in both Busted and LuaUnit using this approach. 

Finally, I loaded the project into a GitHub repository.

Next, I will complete the stockfetch app following a test-driven approach and start to learn how to code in Lua.

Busted or LuaUnit?

Having decided to learn Lua and testing Lua at the same time, I need to decide on a testing framework. There seem to be quite a large number available. I was guided to look at Busted and LuaUnit. Both seem active, professional and have good documentation. They have quite different approaches and it is hard to chose between them without any experience of them. 

I have decided to use them both whilst learning Lua. That decision will, no doubt, slow my learning a little but it should give a greater depth to what I learn.

To get started quickly, a took a function from a Prag Prog Magazine  article "a-functional-introduction-to-lua" and created a module to put it in and wrote both Busted and LuaUnit tests for it.

Here is the module and function:

    local learnLua = {}

    function learnLua.namesOf (things)

      local names = {}

        for index, thing in pairs(things) do

    names[index] = thing.name

  end

      return names

    end

    return learnLua

Here's the Busted test:

    require 'busted.runner'()

    local ll = require 'learnLua'

    describe("Flames with Lua", function()

      describe("namesOf", function()

        it("extract string names", function()

          local cats = {

            { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

          }

          assert.equal("meg", ll.namesOf(cats)[1])

          assert.equal("mog", ll.namesOf(cats)[2])

          assert.equal(2, # ll.namesOf(cats))

        end)

      end)

    end)

Here's the LuaUnit test:

    luaunit = require 'luaunit'

    ll = require 'learnLua'

    local cats = {

      { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

    }

    function testNameExtract() 

      local cats = {

        { name = "meg", breed = "persian" },

  { name = "mog", breed = "siamese" }

      }

      local names = ll.namesOf(cats)

      luaunit.assertEquals(names[1], "meg")

      luaunit.assertEquals(names[2], "mog")

      luaunit.assertEquals(# names, 2)

   end

  os.exit( luaunit.LuaUnit.run() )

Both tests pass:

    $ luajit learn-unit.lua            
    . 
    Ran 1 tests in 0.000 seconds, 1 success, 0 failures    OK

    $ luajit learn-busted.lua    ●
    1 success / 0 failures / 0 errors / 0 pending : 0.000617 seconds

Not a lot to chose between them when testing such a simple function. I need to work up to something more complex to really see the differences between them.

I'd be happy to learn about improvements in style or form that I could make in writing Lua, Busted tests and LuaUnit tests.

There is some early feedback on Busted and LuaUnit in Test Doubles and you can read my conclusion in Busted or LuaUnit? Answered

Installing LuaJIT and Luarocks on OS X

Thanks to Homebrew, installing Lua on OS X is easy. I wanted to install the LuaJIT version as that is the one that I'm likely to be using in the future. It was just a matter of:

    $ brew install luajit

I also wanted to install the Luarocks package manager. My first attempt failed.

Then I found the very helpful instructions at Mesca's Homebrew-Luarocks page. 

Sadly, it is no longer available via Homebrew. I need to find a new method to install Luarocks to use with Luajit

Learning Lua

It looks as though I may be working on a project using Lua and I thought it would be a good idea to start learning the language ahead of time. I know JavaScript and Red which like Lua are influenced, either directly or indirectly, by Scheme. I'm hoping my knowledge of them will help me in understanding both the concepts and philosophy of Lua.

The most common way of learning a new language, especially one with a command line interpreter, seems to be to experimenting with coding whilst reading or going through a tutorial. Once you've learnt the language, you go on to learn about its environment - modules (or packages), testing and the like. 

I'm going to adopt a different approach to learning Lua and try to use things  such as modules and testing from the outset. Over the next few months, we'll see how well it works for me.