Installing Lua LGI on macOS

I wanted to see how easy it would be to build some very simple GUI apps in Lua. As I already have GTK+3 installed on my machine, I thought it would be a good idea to try to use it from Lua. LGI is gobject-introspection based dynamic Lua binding to GObject based libraries including GTK+3. It can be installed via Luarocks.

A simple attempt at installing LGI results in an error as Luarocks can't find the libffi library that LGI requires. It is a similar problem to the one I encountered with installing the Ruby GTK+3 Gem.  Applying a similar solution to the one kindly supplied by the Ruby Gnome 2 team did the trick.

This is the command that worked:


  sudo PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig luarocks-jit install lgi

Notes:

I installed GTK+3 using Homebrew

I am using LuaJIT 2.0.4

I am using luarocks-jit 2.3.0

Swift Int32.divideWithOverflow doesn't always actually protect against overflows!!

Good news update.

I've started looking into Swift with the intention of using it to build personal applications running under macOS. I needed to generate some tests of Red/System's integer processing. I would normally generate such tests in Rebol or Ruby but as Red/System integers overflow, rather than raising exceptions, I needed to use a language that supports overflowing integers.

Swift has a collection of <operand>WithOverflow functions. So I went ahead and wrote a Swift program to generate the tests. Then I came across a problem, I got an overflow exception when using divideWithOverflow. It seemed strange. Here is the culprit:

Int32.divideWithOverflow(Int32.min, -1)

Playground execution failed: error:myPlayground.playground:31:7: 

error: division '-2147483648 / -1' results in an overflow

Int32.divideWithOverflow(Int32.min, -1)

I checked on the Swift-User mailing list if that is the expected behaviour. Apparently it is. Perhaps the name of the function needs to be updated to divideWithOverflowExeceptWhenDividingMinByMinusOne? ;-)

As a sanity check, I wrote a very simple C program to check what should be returned when dividing -2147483648 by -1.

Here's the code:

#include <stdio.h>

int main(int argc, const char * argv[]) {

    int32_t i = -2147483648 / -1;
    printf("-2147483648 / -1 = %d\n", i);

    return 0;
}

Here's the output:

-2147483648 / -1 = -2147483648

Based on that I went ahead and wrote the test generator in C. I got quite a surprise when performing the calculations, I got an overflow exception. It seems that the compiler is happy when using integer literals but not 32-bit integers. The following code gives an overflow exception. (Funnily enough it doesn't when multiplying.)

This code

  #include <stdio.h>

  int main(int argc, const char * argv[]) {

    

      int32_t i = -2147483648;

      int32_t j = -1;

      int32_t k = i * j;

      printf("-2147483648 * -1 = %d\n", k);

      int32_t m = i / j;

      printf("-2147483648 / -1 = %d\n", m);

    

      return 0;

  }

prints

  -2147483648 * -1 = -2147483648

and then fails with an overflow exception.

Thanks to Nenad Rakocevic, the creator of Red Language, I found that the compilers are delegating the division to the cpu. The behaviour seems to be cpu dependant. The Intel processor in my laptop raises an overflow error whilst the ARM processor in a Raspberry Pi doesn't. (It does give somewhat inconsistent results though).

So I've learnt that if there is a small chance that you could be dividing -2147483648 by -1 in a program with "overflowing" integers you need to protect against it.

Parsing Rebol Data with Lua PEG

I diverted myself from learning lua by working through the stockfetch program as I wanted to test how tricky it would be to read a Rebol data file into a Lua table using Lua's Parsing Expression Grammars (LPeg). I didn't attempt to write an actual conversion tool, just something that would give a good feeling for how difficult it would be to write such a tool.

It is necessary to understand that there really isn't such a thing as a Rebol data file. In Rebol, code is data and data is code. Any syntactically valid Rebol file can be evaluated, if a Rebol file contains functions they will be run. (The same is true of the languages descended from Rebol - Red, Boron and World.) 

Whilst functions are first class values in Lua, I don't think that variables can be treated so in Lua (though I could well be wrong). 

If I am correct, this means that it will only ever be possible to convert a subset of Rebol files to Lua tables.

For my purposes, I felt it was sufficient to confirm that both Rebol objects and blocks could be converted into Lua tables. 

When considering objects, I excluded the possibility of them containing functions or code executed as the object was loaded. For this test, a Rebol object was restricted to be akin to a keyword:value store.

Rebol blocks are ordered series of values, somewhat similar to a Lua table using integer keys.

I also applied the following restrictions for my "proof of concept":

• Words can only be a single letter - a to z
• Values can either be a block, an object or a single digit string - 0 to 9
• Blocks and Objects cannot be empty

Once I started to understand LPeg, I found writing the trial convertor to be quite straightforward. I'm happy with the solution that I came up with as it appears very readable to me (especially compared with regex). I'm sure that it could be improved by someone more familiar with Lua and LPeg than I currently am.

Here's my function:

function reboldata.import (reb)
  local bind = function (...)
    local args = {...}
    local t = {}
    for i = 1, #args, 2 do
      t[args[i]] = args[i + 1] 
    end
    return t
  end
  local End = lpeg.P(-1)
  local Space = lpeg.S(' \n\t')^0
  local OpenBlock = Space * '['
  local CloseBlock = ']' * Space
  local EndString = lpeg.S(' \n\t') + CloseBlock
  local String = Space * lpeg.C(1 - EndString)^1 * Space
  local NotCloseBlock = (1 - CloseBlock)^0
  local Word = lpeg.R('az')
  local GetWord = lpeg.C(Word) * ':' * Space
  local Value = lpeg.R('09')
  local Element = (String + lpeg.V'Block') + lpeg.V'Object'
  local GetWordValue = GetWord * 
                       ((lpeg.C(Value) + lpeg.V'Block') + lpeg.V'Object') * Space
  local ObjectContent = (GetWordValue^0 / bind) + CloseBlock
  local ParseRebol = lpeg.P{
    'PR';
    PR = lpeg.Ct(lpeg.V'BO'),
    BO = (lpeg.V'O' + lpeg.V'B') + End,
    O = (lpeg.V'Object' * lpeg.V'B') + lpeg.V'Object',
    B = lpeg.V'Block' * lpeg.V'BO',
    Object = lpeg.P('make object!' * Space * 
             OpenBlock * ObjectContent * CloseBlock),
    Block = lpeg.Ct(OpenBlock * Element^0 * CloseBlock)
  }
  return lpeg.match(ParseRebol, reb)
end

Full source and tests

Lua Patterns, Red Parse, and Regex

In my Learning Lua project, I needed to extract a four character code from a string. Many languages use a builtin Regex implementation to provide the capability to match and extract patterns from strings. I am also familiar with Red, which like Rebol before it, provides the Parse Dialect (a Domain Specific Language (DSL)) which not only has excellent string matching and extraction features but makes it easy to write further DSLs. 

I found that Lua has another approach. It has a built-in lightweight pattern language, known as Patterns. From what I understand, it was designed to be simpler than Regex both in its "grammar" and its capability. (Lua does have Regex modules available. It also has the very interesting LPeg pattern-matching library, based on Parsing Expression Grammars (PEGs).)

I thought it would be interesting to compare the three approaches, patterns, parse and regex. Here are three versions of a short function to extract the first four consecutive alphabetic characters from a string. The functions return an empty string if there is no match.

First Regex, courtesy of Ruby:

    def parse_line line

      ticker = line.match(/[A-Za-z]{4,4}/)

      if ticker then

        ticker[0]

      else

        ''

      end

    end

Note: 

    I'm no Regex expert, there may well  be a better way, and probably a more Rubyish way too.

Second Parse, courtesy of Red

parse-line: function [line [string!] /local ticker][

alpha: charset [#"A"-#"Z" #"a"-#"z"] parse line [to copy ticker 4 alpha] any [ticker ""] ]

Notes: 

    1. The charset allows you to efficiently define set of characters to be matched.

    2. The parse function apply the rules provided between the []s to the input.

    3. A parse rule can have any number of sub-rules.

    4. Red Parse details.

Finally Patterns, courtesy of Lua

    function stockfetch.parseLine (line)

      local match = string.match(line, '(%a%a%a%a)')

      if match then 

        return match

      else

        return ''

      end

    end

There doesn't seem much to chose between the approaches for such a simple task. The Lua pattern is very readable but you do have to know (or lookup) that %a matches an alphabetic character. The Regex is self-explanatory once you understand Regex syntax. The Red parse rule may seem a little more complicated because it also specifies to where the data are extracted (copy ticker) and has to ensure that the rules cover the whole supplied string (to end).

It will be interesting to see how the three approaches compare with more challenging requirements. Perhaps, I'll be able to add LPeg into the equation too.

Note: I am a strong supporter of and contributor to Red and am highly biased towards its Parse DSL. 

Learning Lua - Stubbing, Mocking and Spying on Functions in Testing

I was very pleased to find that it is fully possible to overwrite existing functions in Lua.  It even applies to those built-in functions that can feel as though they are part of the language. I guess that they are actually functions that are defined in the global scope.

Here's a quick example:

    > a = 1

    > sp = print

    > sp(a)

    1

    > print = function () a = 2 end

    > print()

    > sp(a)

    2

    > print = sp

    > print(a)

    2

This allows full stubbing, mocking and spying on functions during testing, albeit with a little work. You simply save the function you want to stub, mock or spy on and write a function to act as a stub, mock or spy. You can make use of Lua's upvalues to be able to count how many times the function was called in a test.

Here's an example of mocking a function in a test using LuaUnit:

    function testMocking ()

        saveParseLine = t.parseLine

        local parseLineCalled = 0

        t.parseLine = function (line)

            parseLineCalled = parseLineCalled + 1

            luaunit.assertEquals(line, 'AAPL')

            return 'AAPL' 

        end

        luaunit.assertEquals(funcThatCallsParseLine(data), 0);

        luaunit.assertEquals(parseLineCalled, 1) 

        t.parseLine =  saveParseLine

    end

Learning Lua - Test Doubles

I was able to spend some time learning Lua this week and made a little progress with my stockfetch project which am using to answer Busted or LuaUnit?. 

This week I needed to mock some functions to write some tests. Lua, as I had expected, makes it easy to replace functions with test "doubles". I was expecting that Busted would make it easy to "mock" a function. What I found is that Busted definitions of "stub" and "mock" are not the ones that I understand. 

My understanding was based on four types of test "doubles" - fakes, stubs, mocks and spies. Fakes are functions that provide an implementation just for testing that would not be used in production.  Stubs return a known value or perform a known side effect. Mocks are like stubs with the added functionality of being able to query interaction with them. (Such as what arguments they were called with, how often they were called and what they returned.) Spies could be considered as mocks which additionally call the actual function.

From my current understanding, in Busted spies are like the spies that I understand. It seems that Busted stubs appear to be like the mocks with which I am familiar but without the ability to specify a return value (or values). From the docs, Busted mocks are tables whose functions have been wrapped in spies or stubs.

The different terminology isn't a problem but not being able to specify a return value from a test "double" leads to additional work. (It might actually be possible to do so in Busted but I haven't been able to find out how just yet.)

I had expected to "roll my own" test doubles with LuaUnit but having to do part of the job myself with Busted means that there still isn't much to choose between them.

In one test, that I wrote I wanted to check that a function was called five times and have it return 0 each time it was called.

This is how I achieved that using Busted:

        saveProcessTicker = stockfetch.processTicker

    stockfetch.processTicker = function () 

      return 0 

    end 

    spy.on(stockfetch, "processTicker")

    assert.equal(0, stockfetch.run('fixtures/mixedTickers.txt'))

    assert.spy(stockfetch.processTicker).was.called(5)

    stockfetch.processTicker:revert()

    stockfetch.processTicker = saveProcessTicker

This is the LuaUnit equivalent:

    saveProcessTicker = stockfetch.processTicker

  local processTickerCalled = 0

  stockfetch.processTicker = function ()

    processTickerCalled = processTickerCalled + 1 

    return 0 

  end 

  luaunit.assertEquals(stockfetch.run('fixtures/mixedTickers.txt'), 0);

  luaunit.assertEquals(processTickerCalled, 5)

  stockfetch.processTicker = saveProcessTicker

The main difference being needing to "manually" count the number of times that the function was called with LuaUnit.

I will be looking at using test doubles with builtin modules such as IO next.

Lua inf, -inf and nan

The Lua number datatype has the conventional three special values inf (positive infinity), -inf (negative infinity) and nan (not a number). It seems not to be so conventional when it comes to comparing these values. It returns true when comparing inf with inf, -inf with -inf, and nan with nan. I believe that mathematically this is not quite correct. However, I can see the advantages of being able to compare them from a programming perspective.

Preparing to Learn Lua

I took my second step in preparing to learn Lua. I've set up a simple training project through which I can get a feeling of the workflow and practices that will work for me when programming in Lua. I've based the project on the StockFetch example from Test Driving Javascript Applications by Dr. Venkat Subramaniam. (A very worthwhile read).

Before I set up the project, I wanted to see how easy it would be to read the contents of a web page in Lua. It turned out to be very easy once I had installed luasocket.

    sudo luarocks-jit install luasocket

Reading a webpage highlighted, for me, the ability to return multiple values from a Lua function:

   http = require 'socket.http'

    body,code,headers,status = http.request('http://bbc.co.uk')

I also noticed that much Lua is written defensively in the sense that care is taken to minimise opportunities for crashes through the use of the assert function and other such techniques. Clearly, being able to return a status code and a value from a function will help to write robust code.

Knowing how to read websites in Lua, I proceeded to set up the project. I adopted a directory structure that will be familiar to many people but I don't know how common it is in Lua circles.

The basic structure is:

    stockfetch/                           -- main directory

        stockfetch-cli.lua                -- command line interface

        src/                              -- lua modules

            stockfetch.lua                -- main app module

        test/                             -- tests

            stockfetch-busted.lua         -- busted tests

            stockfetch-unit.lua           -- Lua unit tests

After setting up the project, I wanted to work out how best to test its main program, stock watch-cli.lua. It's basically a three line script to initiate the main stockfetch module.

      local stockfetch = require('src/stockfetch');

      local rc = stockfetch.run()

      os.exit(rc)

I'm glad I did as I found out a few things about file paths in Lua that were useful to know from the off:

• Lua retains the current working director setting from the OS. So relative paths calculated in a script can change depending on the directory from which the script was launched
• Another way of looking at that is the Lua does not change the current working directory to that from which a script is loaded
• The dofile function only accepts absolute file paths
• There is no builtin function to "clean" a file path limiting the use of setting a base-file path and a relative path including ../

It soon became apparent that I wasn't going to be able to test stock watch-cli by using docile. (I would be delighted to learn of such a way if there is one.) Instead, I used Lua's command line interfaces to test it.

I found two different ways to run command line programs from Lua - os.execute and io.popen. The first returns the return code from the command executed, the second returns the output from the command. I wanted both, so I chose to use os.execute and redirect the command output to a file. That way I'm able to get both the output and the return code.

I successfully wrote tests in both Busted and LuaUnit using this approach. 

Finally, I loaded the project into a GitHub repository.

Next, I will complete the stockfetch app following a test-driven approach and start to learn how to code in Lua.

Busted or LuaUnit?

Having decided to learn Lua and testing Lua at the same time, I need to decide on a testing framework. There seem to be quite a large number available. I was guided to look at Busted and LuaUnit. Both seem active, professional and have good documentation. They have quite different approaches and it is hard to chose between them without any experience of them. 

I have decided to use them both whilst learning Lua. That decision will, no doubt, slow my learning a little but it should give a greater depth to what I learn.

To get started quickly, a took a function from a Prag Prog Magazine  article "a-functional-introduction-to-lua" and created a module to put it in and wrote both Busted and LuaUnit tests for it.

Here is the module and function:

    local learnLua = {}

    function learnLua.namesOf (things)

      local names = {}

        for index, thing in pairs(things) do

    names[index] = thing.name

  end

      return names

    end

    return learnLua

Here's the Busted test:

    require 'busted.runner'()

    local ll = require 'learnLua'

    describe("Flames with Lua", function()

      describe("namesOf", function()

        it("extract string names", function()

          local cats = {

            { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

          }

          assert.equal("meg", ll.namesOf(cats)[1])

          assert.equal("mog", ll.namesOf(cats)[2])

          assert.equal(2, # ll.namesOf(cats))

        end)

      end)

    end)

Here's the LuaUnit test:

    luaunit = require 'luaunit'

    ll = require 'learnLua'

    local cats = {

      { name = "meg", breed = "persian" },

      { name = "mog", breed = "siamese" }

    }

    function testNameExtract() 

      local cats = {

        { name = "meg", breed = "persian" },

  { name = "mog", breed = "siamese" }

      }

      local names = ll.namesOf(cats)

      luaunit.assertEquals(names[1], "meg")

      luaunit.assertEquals(names[2], "mog")

      luaunit.assertEquals(# names, 2)

   end

  os.exit( luaunit.LuaUnit.run() )

Both tests pass:

    $ luajit learn-unit.lua            
    . 
    Ran 1 tests in 0.000 seconds, 1 success, 0 failures    OK

    $ luajit learn-busted.lua    ●
    1 success / 0 failures / 0 errors / 0 pending : 0.000617 seconds

Not a lot to chose between them when testing such a simple function. I need to work up to something more complex to really see the differences between them.

I'd be happy to learn about improvements in style or form that I could make in writing Lua, Busted tests and LuaUnit tests.

There is some early feedback on Busted and LuaUnit in Test Doubles and you can read my conclusion in Busted or LuaUnit? Answered

Installing LuaJIT and Luarocks on OS X

Thanks to Homebrew, installing Lua on OS X is easy. I wanted to install the LuaJIT version as that is the one that I'm likely to be using in the future. It was just a matter of:

    $ brew install luajit

I also wanted to install the Luarocks package manager. My first attempt failed.

Then I found the very helpful instructions at Mesca's Homebrew-Luarocks page. 

Sadly, it is no longer available via Homebrew. I need to find a new method to install Luarocks to use with Luajit

Learning Lua

It looks as though I may be working on a project using Lua and I thought it would be a good idea to start learning the language ahead of time. I know JavaScript and Red which like Lua are influenced, either directly or indirectly, by Scheme. I'm hoping my knowledge of them will help me in understanding both the concepts and philosophy of Lua.

The most common way of learning a new language, especially one with a command line interpreter, seems to be to experimenting with coding whilst reading or going through a tutorial. Once you've learnt the language, you go on to learn about its environment - modules (or packages), testing and the like. 

I'm going to adopt a different approach to learning Lua and try to use things  such as modules and testing from the outset. Over the next few months, we'll see how well it works for me.