HTTPS with Lua Sockets

I noticed that a couple of people a week are reading Busted or LuaUnit?. If they were looking for some comparison, they would have been disappointed. Especially as they didn't look at Lua Test Doubles which does give some feedback on Busted and LuaUnit.

I've started out to complete the test-driven development of the stockfetch program using both Busted and LuaUnit. The next step in the project is to read the stock prices from Yahoo. On the surface this didn't seem to be difficult as when I tried GETting a webpage using Lua Socket, it was pretty simple.

But I found that the Lua Socket HTTP "simple" form is unable to handle https. To GET data from an https source you need to use the Lua Socket HTTP "generic" form and the Lua Sec module. Lua Socket HTTP generic form uses LTN12 sinks to collect the content rather than returning a simple string.

Here is a simple example of GETting the content of the Lua Home Page. I hope it is sufficiently self-explanatory.

local https = require('ssl.https')

local ltn12 = require('ltn12')

local responseContent = {}

local returnCode, httpResponseCode, httpHeaders, htttpStatus = https.request{

  url = 'https://www.lua.org',

  sink = ltn12.sink.table(responseContent)

}

if returnCode then

  print('httpResponseCode: ' .. httpResponseCode)

  print('date: ' .. httpHeaders.date)

  print('server: ' .. httpHeaders.server)

  print('last-modified: ' .. httpHeaders['last-modified'])

  print('content-length: ' .. httpHeaders['content-length'])

  print('content-type: ' .. httpHeaders['content-type'])

  print(table.concat(responseContent));

else

  print(httpResponseCode)       -- this will contain a lua socket error message

end

Note: I used Lua Rocks to install the Lua Sec module. It requires OpenSSL to be installed. If you running Lua under macOS, you might find these instructions helpful. I did. 

Installing Lua LGI on macOS

I wanted to see how easy it would be to build some very simple GUI apps in Lua. As I already have GTK+3 installed on my machine, I thought it would be a good idea to try to use it from Lua. LGI is gobject-introspection based dynamic Lua binding to GObject based libraries including GTK+3. It can be installed via Luarocks.

A simple attempt at installing LGI results in an error as Luarocks can't find the libffi library that LGI requires. It is a similar problem to the one I encountered with installing the Ruby GTK+3 Gem.  Applying a similar solution to the one kindly supplied by the Ruby Gnome 2 team did the trick.

This is the command that worked:


  sudo PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig luarocks-jit install lgi

Notes:

I installed GTK+3 using Homebrew

I am using LuaJIT 2.0.4

I am using luarocks-jit 2.3.0

Preparing to Learn Lua

I took my second step in preparing to learn Lua. I've set up a simple training project through which I can get a feeling of the workflow and practices that will work for me when programming in Lua. I've based the project on the StockFetch example from Test Driving Javascript Applications by Dr. Venkat Subramaniam. (A very worthwhile read).

Before I set up the project, I wanted to see how easy it would be to read the contents of a web page in Lua. It turned out to be very easy once I had installed luasocket.

    sudo luarocks-jit install luasocket

Reading a webpage highlighted, for me, the ability to return multiple values from a Lua function:

   http = require 'socket.http'

    body,code,headers,status = http.request('http://bbc.co.uk')

I also noticed that much Lua is written defensively in the sense that care is taken to minimise opportunities for crashes through the use of the assert function and other such techniques. Clearly, being able to return a status code and a value from a function will help to write robust code.

Knowing how to read websites in Lua, I proceeded to set up the project. I adopted a directory structure that will be familiar to many people but I don't know how common it is in Lua circles.

The basic structure is:

    stockfetch/                           -- main directory

        stockfetch-cli.lua                -- command line interface

        src/                              -- lua modules

            stockfetch.lua                -- main app module

        test/                             -- tests

            stockfetch-busted.lua         -- busted tests

            stockfetch-unit.lua           -- Lua unit tests

After setting up the project, I wanted to work out how best to test its main program, stock watch-cli.lua. It's basically a three line script to initiate the main stockfetch module.

      local stockfetch = require('src/stockfetch');

      local rc = stockfetch.run()

      os.exit(rc)

I'm glad I did as I found out a few things about file paths in Lua that were useful to know from the off:

• Lua retains the current working director setting from the OS. So relative paths calculated in a script can change depending on the directory from which the script was launched
• Another way of looking at that is the Lua does not change the current working directory to that from which a script is loaded
• The dofile function only accepts absolute file paths
• There is no builtin function to "clean" a file path limiting the use of setting a base-file path and a relative path including ../

It soon became apparent that I wasn't going to be able to test stock watch-cli by using docile. (I would be delighted to learn of such a way if there is one.) Instead, I used Lua's command line interfaces to test it.

I found two different ways to run command line programs from Lua - os.execute and io.popen. The first returns the return code from the command executed, the second returns the output from the command. I wanted both, so I chose to use os.execute and redirect the command output to a file. That way I'm able to get both the output and the return code.

I successfully wrote tests in both Busted and LuaUnit using this approach. 

Finally, I loaded the project into a GitHub repository.

Next, I will complete the stockfetch app following a test-driven approach and start to learn how to code in Lua.

Installing LuaJIT and Luarocks on OS X

Thanks to Homebrew, installing Lua on OS X is easy. I wanted to install the LuaJIT version as that is the one that I'm likely to be using in the future. It was just a matter of:

    $ brew install luajit

I also wanted to install the Luarocks package manager. My first attempt failed.

Then I found the very helpful instructions at Mesca's Homebrew-Luarocks page. 

Sadly, it is no longer available via Homebrew. I need to find a new method to install Luarocks to use with Luajit