Friday, January 24, 2014

A Dockerfile for FixMyStreet Part 2

More days later...

Update: many of the bugs and glitches I mention below have been fixed in Docker version 8.

I was able to access the FMS server running in the VirtualBox image on my MBA but only from within the image. Couldn't access it from my browser running on my MBA.

I tried a variety of hacks. The first one was to ignore dvm/vagrant and use VB's console to change the networking setting for Adapter 1 to bridge => en2: Display Ethernet. This allowed me to set the IP address to one within my LAN and I was able to access FMS in my browser.

But this required manual intervention and I wanted the image to be as automatic as possible. So I tried editting ~/.dvm/Vagrantfile to created a public network and to set the adapter to bridge. But this caused a conflict error message to halt the startup. I tried a variety of changes and none remove the conflict.

Eventually I realised that the default setting (192.168.42.43) address is accessible to the browser running on my MBA. Not sure why I didn't check this first. Unfortunately it's not accessible by other browsers on my LAN (192.168.1.*) but it's only for development so I'm happy.

Then I used Intercity's note about flattening the Docker image. It dropped from approx. 2GB down to 750MB. Still quite large but better.

Then I tried to push the image to Docker's public repository.  And I'm still trying... The upload keeps timing out, or the Docker DNS's disappear or something else.

Intercity's flattening note also shows how to export the image to a tarball and reimport it and I've at least got a tarball with which I can re-import the image as needed.

So lots of "learning experiences". I still think Docker might be the best devel tool since the cloud appeared but still many rough edges for people who use Macs.

A Dockerfile for FixMyStreet.

This is going to be a long and rambling report of how I go about writing a Dockerfile to build FixMyStreet (FMS) as an (almost) stand-alone web app. I'm making this up as I go so there will probably be lots of false turns.

Some preliminaries

I'm developing using Docker on my MacBook Air and deploying the image on Google Compute Engine. I installed the following on my MBA:
In the fixmystreet repository there is a Debian install script: commonlib/bin/install-site.sh, which I "translated" into Dockerfile commands as much as I was able to.

Then I followed the FMS manual installation instructions but once again I translated them to the equivalent Dockerfile instruction.

Two days later...

Well, I've wasted another two days of my life in futility. It seems that, at least on my MBA, the Mac OS X client has a fatal flaw: the connection to the VirtualBox image seems to drop after about ten minutes. It took me a day to realise that it was the Mac client that was faulty. I must have run and re-run the docker build a few hundred times, trying to isolate the fault.

And then in desperation I tried dvm ssh to log in to the boot2docker image running in VirtualBox. From there I copied my Dockerfile into the login directory and ran docker build and docker run from the login shell. And it ran without dropping the connection once. D'oh!

However it wasn't plain sailing from there. The final manual instruction is to run a script which installs all the requisite Perl modules. The script uses Miyagawa's Carton module to manage all the dependencies of the 84 modules used to build FMS. I ran the script both as a Dockerfile command and by logging into the container and running the script from the commandline.

No matter what I tried, modules were failing to compile (i.e. the gcc compiler was fatally crashing). Once again I wasted many hours Googling for the error messages until I found out the unexplained gcc crashes are usually from lack of RAM. Sure enough the default RAM for the container was only 512MB. In ~/.dvm/dvm.conf on my MBA I was able to change the RAM to 1024MB (my MBA has 4GB, plenty to spare). And at last all the Perl modules compiled and installed without complaint.

Finally, there were some Postgresql initialisation commands. These can only be performed when the postgres daemon is running so I decided to echo the commandline instructions into ~/.bashrc (along with a couple of useful aliases) so that when I run the image and fire up a bash shell, the remaining initialisation is performed upon my first login. I can then manually delete the instructions and I have an initialised and running instance of FixMyStreet.

Update: .bashrc only needs to start Postgres and FMS, other initialisations are now in the Dockerfile.

Image and Dockerfile: https://index.docker.io/u/garyaj/fms/


Wednesday, January 8, 2014

One RN104 was a disaster. The perils of small sample sizes.

Netgear support over the days following my report of the problems I was having with the RN104 was awesome. They tried everything, kept me informed at all stages and eventually asked me to return the old one and get a replacement.

The replacement now has an uptime of 26 days and hasn't skipped a beat.

Moral: don't generalise based on a sample of one. (Of course I could quibble about how the faulty unit got past QA in the first place...)