Monday, May 19, 2014

KiCad on Mac OS X Mavericks

Eventually I want to design a new circuit that I would like to implement as a real gadget. This means, eventually, I will need to design a PCB (printed circuit board) to hold all the parts and wire them together. I looked at a few different products. There's the really expensive Altium (supposedly the best :); there's a few no cost but proprietary ones e.g. ExpressPCB; there's a few no cost for initial limited sizes (pay for unlimited) e.g. Eagle and there's a few FOSS products e.g. gEDA and KiCad. I can't explain why but I decided I liked the look of KiCad and I wanted a tool without artificial limitations. And it (supposedly) runs on Mac OS X.

So here's the story of how I spent nearly a week getting KiCad EDA (Electronic Design Automation) to work on my MacBook Air running 10.9 (Mavericks).

To give me motivation and direction for my efforts I used a series of video tutorials from Contextual Electronics called 'Getting to Blinky'. The videos seem (to me at least) to be perfectly paced so I didn't get bored or (often) lost. Obviously the first step was to install KiCad on my MBA.

Native

There are at least three different ways to run KiCad on Mac OS X. One is to run it as a native app either by compiling it yourself or downloading a compiled version. It quickly became apparent that the Mac version is very poorly supported despite a lot of community effort. The KiCad downloads page offers three different ways to install a native KiCad. The "official" way is to use the shell script (poorly) maintained on GitHub. It hasn't been updated for three months and doesn't work anymore but the comments in the Issues page do point to working solutions. In summary use this. BUT make sure you have all the prerequisites installed (using Homebrew) and then use brew to uninstall 'xz' (you can re-install it after build is complete). (Damned if I know why build crashes if 'xz' is there but it does.) And don't forget to install Doxygen which isn't mentioned in the prerequisite list. And don't use the multi-CPU option on Mac. Seems 'make' has a problem on Mac.

So after a successful install of KiCad in /usr/local/bin my troubles really began. Seems that KiCad changed it's library format sometime in the past year and most of the docos refer to the old library but all the code expects the new format. Also on Mac the "home" directory is all over the place. Sometimes it really is $HOME. Other times it is $HOME/Library. Libraries can supposedly be downloaded in realtime from GitHub but it never worked for me so then you have to install them locally. There's a script on EEVblog that will do this. Eventually I got a working version of KiCad on my MBA. Working through the Blinky videos quickly showed up the limitations of using an MBA trackpad instead of a 2-button mouse with track wheel. Zooming takes a long time to adjust to. In the end I tended to use function keys. And it takes a while to work out which keys correspond to the Linux keys. You can save yourself some grief by installing MiddleClick now. But I don't want to use a mouse. So I persisted with using the trackpad for zooming and eventually I started to get the hang of it. But it will never be as smooth as a mouse wheel.

You can also install a compiled version of KiCad from here or here but both are fairly old versions of KiCad and both use the old library format.

The deal breaker for me was when I was placing tracks on the PCB and wanted to change the grid because the steps were too coarse. You can't change the grid on the Mac version. Neither of the compiled versions worked either. End of story!

Linux VM

From all the comments I read in my searches, the Linux version appears to get the best support so a second approach is to install VirtualBox, install Ubuntu on VirtualBox then install the latest and greatest KiCad using apt-get. This is a really straightforward approach with no surprises. It simply worked. BUT Ubuntu in VirtualBox is painfully slow in swapping applications. It does a really tedious animation of drawing small windows of the running apps and then one clicks on one to display it. And the disk image takes up 7.4GB whereas the native app takes only a couple hundred MB. And VirtualBox obviously uses a lot of OS resources. Firefox in VB was particularly slow. So almost usable but not quite. An added bonus is that I subscribed to the nightly updater for KiCad so bugs were being fixed quickly.

Wine

The tediously slow Ubuntu on VB plus the sheer size of the disk image made me think that perhaps I could try running a Windows image of KiCad under Wine. I use Wineskin Winery to install Win apps on my MBA. The task in this case is to find a Win installer for KiCad. There's plenty of scripts to compile it but not many actual install .exe's. I could only find the one mentioned on KiCad's download page and it's nearly a year old which for a fast changing product like KiCad is really old. However it installed simply and it works (almost). The main problem once again is the trackpad on my MBA. I'm so used to using taps instead of clicks but this version of KiCad doesn't seem to handle the trackpad well. Sometimes I have to double tap, other times tapping won't work at all and then I have to click but sometimes I have to double-click all for the same task. Very confusing. Also when drawing the PCB I had to move components out of the way to get access to components underneath. Not sure if this is simply an old version problem or a trackpad issue. Other versions pop up a window asking me to choose which component or track I want to move/edit/etc. And sometimes this version does pop up the window. But not consistently. As for disk image size, the Wine version clocks in at 790MB. But the older version might not have used up as much space. Hard to tell.

But none works well

So I can't decide which version to go with. A recent Win installer would be worth a try if I can find one. Maybe running the Linux version under a smaller distribution would be worth a try. TinyCoreLinux in VirtualBox is currently using 2.4MB(!) on my MBA. (I've been using TCL as part of the boot2docker app.) However the nightly compiles of KiCad are done under Ubuntu so there is a lot of advantage in using the "standard" environment. Maybe I can get rid of Ubuntu's painful animations. It's the first thing I do on Mac OS X. Or maybe the Mac native app will get grid change working...

In summary, none of the Mac alternatives works well. If I were going to use KiCad seriously I think a separate machine running Ubuntu would be a good idea and a mouse with track wheel seems almost compulsory.

As for KiCad itself, I really like it. I found each of it's stages easy to follow and was (eventually) able to get to a set of Gerber files for Blinky ready for manufacture. Dave Jones of EEVblog's opinion notwithstanding, KiCad is a serious contender for Open Source hardware developers. I just wish I could get a usable version for Mac OS X.

Update

I tweaked Ubuntu using the tips here (especially the gnome-session-flashback tip) and now KiCad screams along! (Cannot understand how or why anyone would release such a crippled desktop version.) Really smooth use of all sections. Even the trackpad scrolling is usable now (barely) but I still prefer F1 and F2 keys. I'm looking forward to some productive KiCad use on my MBA now.


Monday, April 14, 2014

Sorting disk usage, Perl to the rescue.

My MacBook Air was close to full up after a lot of music editting (see results at http://www.stmaryssingers.com/recordings.html ). In the past I would run

du -s * | sort -rn

to identify the biggest users of disk space giving for example:
14104880        iTunes
683304  Noël Français
682592  sibs
221488  DancingDay
93664   MissaAlmePater
27480   ChristmasLullaby
17680   JesuJoy
13656   MissaBenedicamus
12256   MassInHonorOfSaintJoseph
8760    pdfs
3584    ChrissyCarols
2456    LookingAtTheStars
1352    BriggsMass
408     Bach-Jesu
264     40_The_First_Nowell.sib
64      Thou-knowest-Lord-Z-58b.pdf
24      StMS20140412
24      StMS20140322
24      StMS20140309
24      StMS20140208
24      StMS20131214
0       GarageBand


and the numbers are in 512-byte blocks. Nowadays that's a lot of digits to decipher in the listing. So I started using the 'h' ('human readable') option:

du -sh * | sort -rn

giving:
676K    BriggsMass
334M    Noël Français
333M    sibs
204K    Bach-Jesu
132K    40_The_First_Nowell.sib
108M    DancingDay
 46M    MissaAlmePater
 32K    Thou-knowest-Lord-Z-58b.pdf
 13M    ChristmasLullaby
 12K    StMS20140412
 12K    StMS20140322
 12K    StMS20140309
 12K    StMS20140208
 12K    StMS20131214
8.6M    JesuJoy
6.7M    MissaBenedicamus
6.7G    iTunes
6.0M    MassInHonorOfSaintJoseph
4.3M    pdfs
1.8M    ChrissyCarols
1.2M    LookingAtTheStars

  0B    GarageBand
Unfortunately sort doesn't know how to sort the unit suffixes. But Perl does. It's a while since I used the Schwartzian Transform but it seems perfect for the task. I copied the Wiki code into a file, dusort.pl, which I placed in a directory in my PATH variable (~/bin in this case) and modified the regex extraction to make it sort by unit suffix first and then by number giving this:

#!/usr/bin/env perl 
use 5.010;

my $size = {P => 6, T => 5, G => 4, M => 3, K => 2, B => 1};
print
  map { $_->[0] }
  sort {
  $size->{$b->[2]} <=> $size->{$a->[2]}
                  ||
        $b->[1] <=> $a->[1]
  }
  map { [$_, /^([ \.0-9]{3,4})([PTGMKB])\t/] }
  <>;

and now when I run the command(s):
du -sh * | dusort.pl
I get the result:
6.7G    iTunes
334M    Noël Français
333M    sibs
108M    DancingDay
 46M    MissaAlmePater
 13M    ChristmasLullaby
8.6M    JesuJoy
6.7M    MissaBenedicamus
6.0M    MassInHonorOfSaintJoseph
4.3M    pdfs
1.8M    ChrissyCarols
1.2M    LookingAtTheStars
676K    BriggsMass
204K    Bach-Jesu
132K    40_The_First_Nowell.sib
 32K    Thou-knowest-Lord-Z-58b.pdf
 12K    StMS20131214
 12K    StMS20140208
 12K    StMS20140309
 12K    StMS20140322
 12K    StMS20140412
  0B    GarageBand
Obviously the next thing to do is to make a shell alias:
alias dus='du -sh * | dusort.pl'
and now I even save a few keystrokes in my task to pinpoint the Biggest (L)User.



Monday, April 7, 2014

Threes! and Threesus A.I.

I decided to try the Threes! app/game on my iPhone. I got to around 21,000 just playing by myself but I went Googling to see if there were ways to improve my score. Found quite a few "tips" articles had discovered most of what I was doing. (No-one mentions how to speed up the score totalling at the end of the game. (Swipe the screen a second or third time. Saves seconds per game.))

But then I discovered Threesus, watched the video for nearly 30 minutes in total awe and decided I would download Threesus and use the Assistant to help me get my score higher.

And once again it became another exercise in yak-shaving...

Threesus is written in C#, Microsoft's attempt at Java+Objective-C+... and is apparently pretty-well compulsory if you are programming on Windows. The author states that one can install Visual Studio Express for Windows Desktop and Threesus should compile and run in commandline Assistant mode.

Well, I don't have a Win box sitting around but I do have Parallels on my MacBookAir so I downloaded and installed Express and was able to fire up ThreesusAssistant very quickly. I had to read the source code to work out the commands but they are simple enough.

What intrigued me was the mention of the 'bonus' cards and how they are indicated by a '+' sign on the card. I'd never seen the '+' sign. The bonus cards were just like the normal '3' card on my version of Threes. Walt Destler has also blogged a high-level overview of Threesus and he refers to a detailed description of the Threes! algorithm and that's when I realised my copy of Threes! was out of date. Version 1.0.3 introduced the '+' sign on bonus cards.

So after a quick update I was able to use the Assistant to get my score upto around 28,000.

But disaster hit! The Assistant failed with a null reference error before I could get a higher score. I didn't fancy attempting to debug a run-time error in a language I don't know using an operating system I don't know.

Destler also mentions Xamarin Studio as a possible alternative for Mac users. I duly installed it and spent quite a few hours attempting to get Threesus to compile and run. Lots of googling for obscure errors, settings etc. etc. The usual yak-shaving for any new software.

At last I got Threesus to start in Xamarin. I input the initial grid data and it commenced "thinking". About 5 minutes later it returned the next move. Then it took 10 minutes for the next move. No way is Xamarin/Mono usable on Mac OSX. The Assistant is already slow enough in Visual Studio on Parallels but it seems to be 100s of times slower in Xamarin on Mac.

So now I have a problem. Any sensible person would give up at this stage and find something useful to do.

But not me. I threw a few dollars into the RPerl Kickstarter project last year in the hope that they could get further than a faster bubblesort and it seems to be coming along nicely. Perhaps it's ready to take on a Threes! playing A.I.? Once again a lot of yak-shaving to get the source downloaded and installed but it appears to be working. Now to see if I can translate/recode C# code into working Perl and then possibly speeding it up with RPerl. RPerl uses the Perl Inline module which also allows raw C or C++ to be in-lined into Perl code. That might save a lot of hassle.

I also realised that this could be a nice project to try out OCRing the iPad screen of the Threes! game grid. I don't want to build a full game-playing robot but the slowest and most error-prone part of using the Assistant is inputting the changes. If I could add a screen-reading section so that the Assistant merely has to output Left, Right, Up or Down it would speed manual play greatly. And while I'm at it, why not use a text to speech converter so I don't even have to look at the Assistant's output. (Time to start lobbying the game developer to add speech input!)  I've got an old iPhone 3GS to supply video input and there are some great articles on OCRing Sudoku grids from newspapers etc. The Threes! grid is very similar. So this might turn out to be a very elaborate but maybe quite fun project. Or a total disaster and time-sink...


Thursday, February 20, 2014

A GA144, a 10MHz ceramic resonator and an LED.

I've found some time to continue my dabbling in the intricacies of the GA144 chip. I spent too many hours trying to copy a demo which used a 32KHz watch crystal to provide accurate clocking for a couple of nodes in the chip. In the end I was unable to get the crystal to sustain oscillation for longer than about 30 seconds. And the nature of the component meant I was unable to monitor the oscillations with an oscilloscope. So I switched to using a 10MHz ceramic resonator and there is a similar demo in the ArrayForth sample code. The main difference is that this demo works! (Ceramic resonators are apparently easier to drive.)

I was also experimenting with the PWM demo application in the User Guide Ch. 9 and there is a small side note to the effect that maybe the constant used to set the LED's brightness ought not to be simply a linear increase but perhaps a Fibonnaci sequence because we perceive brightness increases logarithmically.

So I wondered if I could make the oscillator node send a signal every 100ms to its neighbour to cause the brightness constant to increase in a Fibonacci sequence and thus cause the LED brightness to increase slowly in an apparently "linear" fashion.

A couple of minor details: I used node 417 as the oscillator and attached a Murata CST 10.00 MHz resonator between pin 417.17 and ground. I used node 317 to drive the LED with the anode connected to the 3.3V supply from USB port A and the cathode connected to pin 317.17. (On the EVB001 eval board 317.17 and 417.17 feed out to header J21.) (Refer User Guide Ch. 9 for details of LED connections.)

The following code is very similar to that given in Block 956 in the ArrayForth sample code. However I incorporated changes as suggested by Chuck Moore in his StrangeLoop 2013 talk. And I added bigger and bigger delays between signals to node 317 (via up port). In the end the delay is a million to one so that the 10MHz oscillation is reduced to a 10Hz signal.

Block 864
init waits for a commencement signal from node 317 via the up port, loads the stack with 8 numbers and sets them to become a repeating stack array.

go runs 10 hi-low pulses to the resonator then drops into an infinite loop which calls pump and ring to keep the resonator pulsing and periodically outputs an instruction to node 317.

864 list 
417 orig tx osc 417 node 0 org
drive
 00 4n drop !b !b for unext ;
pump
 03 4n !b @ drop !b !b drop ;
ring
 05 4n !b @,
.........drop drop drop drop ;,
,

go 07 9 for drive drive next 0c .,
...
begin 0d,
......
999 for 999 for,
.........
pump pump ring ring,
......
next next,
......
17 a up a!,
.........@p 19 ; .. 1a ! a!,
...end,
,
init 1c up a! @
xx 1e drop io b! left a!,
7 dup dup or 20000 800,
6 dup dup or 30000 over,
dup dup drop drop go ;,
2c reclaim

Block 866
init sets the a and b registers and loads the stack with 1 and 1, the first numbers in the Fibonacci sequence, then sends an instruction to node 417 causing it to commence the oscillation loop.

pol tests if the up port has a signal on it. If so, elements on the stack being used by the cyc routine are dropped and the next value in the Fib sequence is calculated. If the Fib value becomes "negative", the stack is reset to 1 and 1 otherwise the Fib number is left on the stack to be used by cyc.

cyc uses the PWM method explained in the User Guide. The stack contains a value between 0 and 20000 hex representing brightness i.e. 0 (1) means no light, 20000 means full on/max. brightness.

866 list 
pwm demo 317 node 0 org br

init io b! up a! 1 dup dup dup,
.....
@p . .. 06 ; 07 !,
pol 08 @b 200 uw and if,
...0b up b! @b drop drop drop,
...
+ -if 1 dup dup dup,
...
then over over dup dup dup or,
...
io b! then 15 drop br


cyc ie- 1ffff and over . + -if cr
...19 .....20000 !b pol ; cr
...
1c then 10000 !b pol ; 1f                  

Block 862
is the loader block.

862 list
loader template
host load loader load,
using default ide paths,
kill boots 0 708 hook 0 -hook,
path to 317 0 317 hook,
,
setup application,
417 +node 417 /ram 1c /p,
317 +node 317 /ram 0 /p,
panel pause 0 ship upd ?ram 


                  

Friday, January 24, 2014

A Dockerfile for FixMyStreet Part 2

More days later...

Update: many of the bugs and glitches I mention below have been fixed in Docker version 8.

I was able to access the FMS server running in the VirtualBox image on my MBA but only from within the image. Couldn't access it from my browser running on my MBA.

I tried a variety of hacks. The first one was to ignore dvm/vagrant and use VB's console to change the networking setting for Adapter 1 to bridge => en2: Display Ethernet. This allowed me to set the IP address to one within my LAN and I was able to access FMS in my browser.

But this required manual intervention and I wanted the image to be as automatic as possible. So I tried editting ~/.dvm/Vagrantfile to created a public network and to set the adapter to bridge. But this caused a conflict error message to halt the startup. I tried a variety of changes and none remove the conflict.

Eventually I realised that the default setting (192.168.42.43) address is accessible to the browser running on my MBA. Not sure why I didn't check this first. Unfortunately it's not accessible by other browsers on my LAN (192.168.1.*) but it's only for development so I'm happy.

Then I used Intercity's note about flattening the Docker image. It dropped from approx. 2GB down to 750MB. Still quite large but better.

Then I tried to push the image to Docker's public repository.  And I'm still trying... The upload keeps timing out, or the Docker DNS's disappear or something else.

Intercity's flattening note also shows how to export the image to a tarball and reimport it and I've at least got a tarball with which I can re-import the image as needed.

So lots of "learning experiences". I still think Docker might be the best devel tool since the cloud appeared but still many rough edges for people who use Macs.

A Dockerfile for FixMyStreet.

This is going to be a long and rambling report of how I go about writing a Dockerfile to build FixMyStreet (FMS) as an (almost) stand-alone web app. I'm making this up as I go so there will probably be lots of false turns.

Some preliminaries

I'm developing using Docker on my MacBook Air and deploying the image on Google Compute Engine. I installed the following on my MBA:
In the fixmystreet repository there is a Debian install script: commonlib/bin/install-site.sh, which I "translated" into Dockerfile commands as much as I was able to.

Then I followed the FMS manual installation instructions but once again I translated them to the equivalent Dockerfile instruction.

Two days later...

Well, I've wasted another two days of my life in futility. It seems that, at least on my MBA, the Mac OS X client has a fatal flaw: the connection to the VirtualBox image seems to drop after about ten minutes. It took me a day to realise that it was the Mac client that was faulty. I must have run and re-run the docker build a few hundred times, trying to isolate the fault.

And then in desperation I tried dvm ssh to log in to the boot2docker image running in VirtualBox. From there I copied my Dockerfile into the login directory and ran docker build and docker run from the login shell. And it ran without dropping the connection once. D'oh!

However it wasn't plain sailing from there. The final manual instruction is to run a script which installs all the requisite Perl modules. The script uses Miyagawa's Carton module to manage all the dependencies of the 84 modules used to build FMS. I ran the script both as a Dockerfile command and by logging into the container and running the script from the commandline.

No matter what I tried, modules were failing to compile (i.e. the gcc compiler was fatally crashing). Once again I wasted many hours Googling for the error messages until I found out the unexplained gcc crashes are usually from lack of RAM. Sure enough the default RAM for the container was only 512MB. In ~/.dvm/dvm.conf on my MBA I was able to change the RAM to 1024MB (my MBA has 4GB, plenty to spare). And at last all the Perl modules compiled and installed without complaint.

Finally, there were some Postgresql initialisation commands. These can only be performed when the postgres daemon is running so I decided to echo the commandline instructions into ~/.bashrc (along with a couple of useful aliases) so that when I run the image and fire up a bash shell, the remaining initialisation is performed upon my first login. I can then manually delete the instructions and I have an initialised and running instance of FixMyStreet.

Update: .bashrc only needs to start Postgres and FMS, other initialisations are now in the Dockerfile.

Image and Dockerfile: https://index.docker.io/u/garyaj/fms/


Wednesday, January 8, 2014

One RN104 was a disaster. The perils of small sample sizes.

Netgear support over the days following my report of the problems I was having with the RN104 was awesome. They tried everything, kept me informed at all stages and eventually asked me to return the old one and get a replacement.

The replacement now has an uptime of 26 days and hasn't skipped a beat.

Moral: don't generalise based on a sample of one. (Of course I could quibble about how the faulty unit got past QA in the first place...)