Tuesday, October 28, 2008

Objective-J is a seriously interesting language

I'm working my way through the Objective-J tutorials at the moment. OJ is absolutely brilliant. It has simply(!) transposed the Objective-C syntax into Javascript. And the result is stunning. In-browser apps which look and feel like Mac OS X apps. All the years of OC accumulated knowledge and the ubiquity of JS in every browser. Glad I spent some time learning OC. I thought it was wasted with my chances of getting work programming in OC being close to zero. But with the incredible popularity of the iPhone and the iTouch and now with OJ, OC(-like) programming is going to hit the charts with a bullet.

And I've already got a great (but big!) app I can develop in OJ.

A music OCR/OMR chip?

What if one were to program a SEAForth 40C18 with just the recogniser routines of Audiveris? There's some quite heavy lifting in Audiveris which could easily take advantage of the 40 cores in the chip. I've never met a 100 lines of Java that couldn't be reduced to one line in Perl. I wonder if one could get a similar reduction in Forth. I haven't looked inside Audiveris yet but I bet most of the 100,000 lines of code are the GUI and the object interfaces. Algorithms tend to be compact, for a couple of reasons: big ones are difficult to understand hence humans don't think them up; and big algorithms are slow; the fewer lines of code, the faster it runs. Anyway, now that I've received my 40C18 I will have to find something to do with it when I've got past the Hello World tutorials.

Monday, October 27, 2008

Lowering one's expenses on public transport

I was quite stunned to see a guy come through the electronic ticket barrier at my local station this morning and hand his ticket to another guy waiting there who immediately went back through the barrier to catch a train. Weekly tickets here in Sydney allow unlimited journeys through the rail system within the zone for the ticket.

I don't know how these guys linked up but I could easily see it happening via Facebook or whatever. If you could arrange your starting and stopping times at work or wherever you could share the cost of a weekly ticket amongst a crowd of people, all of it legal (well probably not within the T&C for the ticket but how are you going to check?). No doubt the rail management hate it but how are they going to stop this happening?

I wonder if there is already an online 'ticket pooling' service, like car-pooling but you share your weekly/monthly/yearly ticket. Let the computer work out how to allocate people to a ticket. It would require incredible discipline but if you are struggling for every dollar, you need such discipline to survive. (I remember the luxury when I first started to earn a pay-packet of deliberately not budgeting when I went supermarket shopping. Being from a single-parent family and not rich, my job each Wednesday night was to scan the newspaper for specials at the various supermarkets at the local shopping mall. I then worked out the shopping lists for each store and my mother, sister and I shared the shopping load. It halved our food expenses but it was very time consuming.)

Monday, October 20, 2008

Learning not to be paranoid

I worked out why colleague was so cool towards me when I started a new project. I thought he just didn't like me. But no, it's not about me. Turns out he was in the midst of job-hunting. He found the new job and is leaving this week.

I've seen this behaviour so many times in my working career but my basic paranoia always makes me think it's personal. It almost never is. Usually it's job-hunting, sometimes it's a sick or dying relative, once it was gambling.

The best time was when I was the IT manager for a company and I was fighting with the CFO who kept blaming me for the financial state of the company. (An independent auditor had blamed the GM and CFO and exonerated me.) But he decided to make me redundant (a 'nice' way of firing me) anyway. I went above his head and asked the GM to intervene but he acted as though it wasn't his problem. Once again my paranoia told me it was about me. But as I was walking to the exit on my last day there, the GM walked out with me. He'd been fired by his boss. The bastard CFO got away clean.

Sunday, October 19, 2008

A possible UBM

Better get this idea down before it gets lost in the haze. Louis Savain's UBM looks like the way computing is heading. IntellaSys' SEAForth chip looks very much like what Savain says a parallel processing CPU (MIMD) ought to look like. Are the processing elements of COSA implementable in 64 words of RAM and 64 words of ROM?

The problem is that the devel tools for the SEAForth are primitive. There's no way a non-specialist is going to write parallel programs for the SEAForth using circuit emulators and Forth. So the task, if I care to take it, is to implement a graphical COSA devel environ which targets the SEAForth. Or hosts the IDE itself? Is that feasible?

Tuesday, October 7, 2008

My continuing Forth education

To answer the question I asked in a previous post, the Perl source code is at least 10MB, on top of how many gigabytes are taken up by the OS.

A Forth interpreter could be as small as 8k. Even full-blown development versions are only a couple of hundred kilobytes. Hence Forth can go anywhere there is a CPU and some memory. One can even buy CPUs with the instruction set being Forth. So think pervasive and lots of bang per byte. Employing the principal of "learn once, use everywhere" (hence I use bash, Vim and Perl), Forth looks like a more useful tool to learn than say VHDL or Verilog for FPGAs.

And I like the look of the new SEAForth 40C18 chip: 40 CPUs(with RAM and ROM) on a chip, all with in-built Forth interpreter. $21 in quantity. Could be quite interesting.

Thoughts and feelings and House.

Was watching an episode of 'House' where House is accused of trying to biopsy a patient's spinal cord for his own benefit and not that of the patient. The whole premise of the House series is that House is constantly required to be completely objective despite his own life and that of his colleagues being in tatters.

It got me thinking about the nature of objectivity and human motivation. I remember John Macmurray's definition of objectivity being to act in terms of the thing or person one is considering. For example to act according to what someone needs, not in terms of one's own desires. An objective but tired parent will get out of bed at 3am to feed a crying baby even though every nerve and fibre wants to return to sleep. I also remember Macmurray pointing out in the light of Freud's discoveries about the unconscious, how impossible it is to know one's own motives. Aside from the simple fact that everything we do and think is the result of a whole spectrum of motivations, Macmurray pointed out that the motive(s) we are aware at any time are only the surface and almost certainly will not be the primary motivations. And I remember Macmurray's conclusion that to be a person is to be aware that every positive motivation contains within it it's own negative.

Hence the interesting House episode where we are never sure what House's motive is for seeking the biopsy. And in the end even House is not sure. But he still acts objectively. (I used to wonder why it is so hard to depict good people in drama but there obviously is no such thing as a 'good person', only people with more or less positive motivations. I'm sure there is some would-be PhD mapping out his/her thesis on 'The Christ-image in House M.D.' or whatever.)

What also triggered these thoughts was that yesterday afternoon I was walking home and a young couple passed by, hand in hand. How lovely! I thought. Then I overheard their conversation and realised both were mentally retarded or delayed learners or subnormal intelligence or whatever euphemism or politically correct term is used these days. And I then thought how difficult their lives together would be simply because of lack of understanding of how the world functions and their part in it. How would they/did they discover sex? Would anyone tell them or would they watch some awful X-rated video and get totally misled about what it's all about (speaking from experience here folks!). How would they manage a household budget?

Then I got to thinking, reminding myself that people do not need high intelligence to live well if their motivations are life seeking, not destructive. And how do people acquire life seeking motivations? The same way we acquire logical thoughts: we learn to do it. But we live in a society that is only just starting to discover that life affirming emotions are learnable and desirable. We actually live in an emotionally retarded society, intellectual giants but emotional retards.

And how do we learn life affirming emotions/motivations? Well to be honest I'm not sure. ('Emotional retards' includes me.) I do know that learning to trust one's own feelings is absolutely critical (just as being able to trust one's own thoughts is necessary for mature thinking). I also know that educating the senses is important. Seeking new experiences, discovering new tastes, hearing new sounds, seeing new colours and shapes. And so is feeling new feelings, meeting new people, loving intensely, crying in sadness and crying for joy, trusting even though frightened.

The aim of such education is to become aligned with one's life-affirming motivations and to learn to moderate one's destructive motivations. If one learns to habitually act affirmatively then we don't need to spend as much brain power trying to think our way out of dangerous, awkward, messy, unhappy situations and get on with enjoying the wonders of life on this beautiful planet. And we can still love life even if we don't think as fast or as clearly as we used to. Ultimately we can learn to feel objectively just as well as we can think objectively.

My conclusion though seems as wimpy as a wedding sermon. It just becomes more and more obvious to me that love is the only life-affirming motivation worth the effort of learning and teaching. All our other motivations are in the end self-seeking, albeit dressed up in the nicest terms. Only love acts in the interests of that which is not ourselves. Only love is truly objective.

Wednesday, July 2, 2008

Am I missing something about Forth?

As part of my "educate myself about FPGAs" curriculum, I've noticed Forth turning up in a lot of Google hits. Forth has always been on my long-standing list of languages I wanted to learn. I remember spending my last bit of hard-earned cash in the early 70s to buy an HP-35 programmable calculator. It got me through my final Electrical Circuit Theory exam so was worth the hundreds of dollars it cost me. It had just arrived here in Oz and the examiners didn't have time to change the rules about pocket-size programmable calculators (they didn't exist before the HP-35).

The HP-35 was a Reverse-Polish stack machine. I loved it. Stack machines seemed so obvious I couldn't understand why they never took off.

So when Forth came out I was intrigued to try it but never found a way to do so. There was no Internet then so one couldn't just download a copy and try it out. I saw Forth mentioned in lots of places by people I respected. There was even a microcontroller project (Elektor again?) with Forth being the built-in interpreter rather than BASIC.

So I downloaded gForth and worked my way through the included tutorial. OK, I can print "Hello world" and add some numbers. I also started reading Thinking Forth.

But what makes Forth so interesting to so many obviously intelligent people? Found a 1-hour movie where a skilled Forth developer writes some code. It left me quite cold. It seemed to take ages to write the simplest things. And in the end it looked so much like a recursive decent parser I kept asking myself why didn't he just use one to begin with. OK so that wasn't the point. He was showing people how he thinks in Forth and he succeeds really well in that and deserves praise for that.

But I couldn't see any advantage in using Forth for that particular task and it actually seemed really slow and tedious to use Forth compared to, say, Perl and RecDescent. Part of my task as a Perl programmer when I am about to write new code is to check CPAN for modules which already do what I need. "The best programmers don't" as they say. So there's no doubt a different mind-set between Forth programmers who appear to want to write everything themselves "because it's so easy" and say, Perl programmers, who tend to just re-use existing code from CPAN and rarely write code themselves at all.

Something to ponder a bit further.

Doing what I like

I occasionally browse through Elektor magazine. I've always been interested in gadgets especially the electronic ones and although I haven't earned a dollar building circuitry for a long time now, I've tried to keep up with what's been happening in electronic design. In the late 70s I wrote a BASIC program to convert circuit diagrams into wirewrap instructions and although I sold a copy to a small electronics firm I never pursued it as a marketable product (fool me!). I was never willing to spend the ten of thousands of dollars needed to buy an auto wirewrap machine and my vision of what I wanted to do never got to the "make a living" stage. So most of the time I was content to manually type in the circuit diagram to my program and let it calculate the length and order of the wires and then I could manually wrap my little circuits and get on with the fun of testing and debugging them.

I've referred in another post to Louis Savain's Universal Behaviour Machine. He's been getting heaps of flak in his comments posts, not the least because he doesn't seem willing/able to move beyond rhetoric about why a UBM (parallel processing) is a better paradigm for computing than a Universal Turing Machine (serial processing). One of the comments caught my attention. It suggested that Savain's UBM is "nothing more" (I love that phrase, it immediately alerts me that there is something to look at) than VHDL. "What's VDHL?" says I. Quick Google throws up a wealth of info and links including a series of articles in Elektor on FPGAs which I had glossed over. FPGAs were always attractive but the devel kits were typically thousands of dollars and not for doodlers like me. But then I saw an ad for a really cheap evaluation kit. For $39 I don't care if I blow it up, it's worth trying out.

So the web of ideas goes something like: no UBM yet -> make one -> how? -> modify an existing micro-core -> how? -> learn how to build/install/modify a micro-core in an FPGA -> how? -> buy a cheap eval kit -> where? -> aha! Avnet just released one -> order it -> learn about VHDL -> how? -> read on-line stuff -> note that some people are using Forth to program FPGAs -> how? -> learn Forth -> install a Forth micro-core on FPGA and try it out... and so on and so on.

But the best news for me is that I can use all those years of building logic systems once again. I already know about sync pulses and NOR/NAND logic and multiplexed buses etc. etc. It's quietly gotten a whole lot easier and cheaper to implement this stuff. My last attempt was an ASIC in the 80s, really expensive to design, test and make.

And it's something I like doing.

A replacement for Make

Bzr and Hg interesting but client wants Perforce so am moving client's repository from Clearcase to Perforce. Code base (mostly) written in C++ to run on various platforms including **nx and Win.

I'm taking this opportunity to see if build system could be improved.

First requirement is to replace Clearmake. Looked very closely at SCons and was about to make (pun!) it the build tool for new repository when I discovered an article about Makepp (awful name, sounds like a pre-schooler discussing urination), i.e. Make++ => MakePlusPlus => Makepp. My one major misgiving about SCons is that it is written in Python and although I like Python I don't have the years of experience in Python that I have in Perl. Makepp is written in Perl!. SCons even uses (a limited selection of) Python commands as its scripting language. Makepp OTOH uses GNU Make syntax for most of its scripting language but with the ability to insert Perl code when needed. Not sure if Make syntax is of much benefit to me. I haven't used anything but the simplest Make scripts for years. That's what MakeMaker.pm is for! But, like SCons, Makepp has some pretty intelligent default behaviour. It works out what language the source is in, looks for any required include files, knows the useful options for a host of compilers etc. etc. And, doesn't suffer from Make's recursion problem.

Makepp, again like SCons, has another very useful feature: the ability to correctly track intermediate files. One of the urgent problems, to me at least, with our existing build system is that it does a "clean" build every time. Nothing is kept from previous builds. I suspect this was not intended initially but is a consequence of how Clearmake/Make are being used. No wonder faulty branches are being checked in. I wouldn't wait 5 hours to confirm a build either if I were a developer. I'd much rather leave it to the overnight run and clean up the mess in the morning. So, hopefully, with quicker builds there will be less faulty code checked in.

Anyway, from my initial investigations, it looks like Makepp is the way to go.

Saturday, May 31, 2008

Three concepts which are shaking my foundations

1. HP's announcement of having made a memristor. I trained as an electrical engineer. The memristor is the 4th passive component (the others being the resistor, the capacitor and the inductor). One quarter of the fundamentals of electrical engineering was announced last week! A memristor can function as a memory cell but it's passive (i.e. no power to maintain state, no refresh cycles etc.). And to make matters worse the theory to explain how HP's memristor works requires a complete change in how electricity is understood (flux not voltage). It's been compared to Newton's discovery of F=ma.

2. Louis Savain's positing of a Universal Behaviour Machine rather than a Universal Computing Machine being the basis of computing theory. Parallelism is inherent in the UBM and serial processing has to be added as a constraint. Whereas for Turing's UCM, serialisation is inherent and parallelism is the constraint. Savain posits that we've got the whole theory of computing wrong. Thirty years as a programmer and now he tells me I don't know nuthin'. To make matters worse my training as an electrical engineer forces me to agree with him.

3. John Macmurray's positing that human beings are persons, not machines or animals but we don't have the philosophical apparatus to understand what a person is yet. Thinking is a by-product of what persons do. Blows the theoretical foundation out of most attempts at AI. A "thinking machine" is by definition an oxymoron.

This is God's/Allah's/Flying Spaghetti Monster's revenge for my complaint last week that there is nothing exciting happening in IT these days.

Tuesday, May 27, 2008

Making a boring job interesting

I found I needed to (re-)read a few online articles about making a job interesting because I was losing heart. (That's 'l-o-s-i-n-g' not 'l-o-o-s-i-n-g', God I hate it when people 'loose' something (as in set it free) when they mean 'lose' (as in misplace).)

Found a bit of inspiration in the bits that mention re-defining my job to include stuff I like doing. So I thought I'd better catalogue what I like doing. I like learning new things. I like learning things that other people are learning so I can talk about what I've learned, in other words, feel part of a community of learning. I like building new things. Usually that means new software artefacts, at least this is so with my skills (but my electrical engineering background means I still love the smell of solder resin and the ping of cut wires and the sheer delight of seeing the first LED light up). I also love teaching new stuff to people. I love preparing course materials (but hate PowerPoint).

Last night as I was about to leave I found a terabyte disk on the server full of VMWare images. Ooh! now that is interesting!

Yesterday I spoke to one of the developers about an error in one of the automatic builds. It was caused by a file having a different name to the one it had in the ClearCase Software Object declaration. That's a fatal flaw as far as I'm concerned. CC is out no matter what! I wouldn't touch it with a barge pole. A VCS is supposed to help not hinder software devel. And how did the developer's branch get into the main tree if it had errors in the compile? "Oh, we do that all the time. It takes too long to run a build in a branch so we just force the changed file into the main trunk and let the build engineer deal with it." (aka me!). Like bloody hell is that going to continue! Not sure if this contributed to the departure of the previous incumbent but it certainly will contribute to my departure if I can't get that changed in the next five months.

So here's the slowly forming picture of the future build system:
  • Set up virtualised copies of all the OS's we support. (That's probably done but I'm not sure if it's used yet.) Also need to set up some way of controlling the virtual server programmatically. (Need to learn about this! Yay!).
  • Setup a sandbox VM for myself. Probably RHEL (one of the supported OSs).
  • Install on my VM Apache2+mod_fcgid+Catalyst to allow web management of build system.
  • Speed-up display of build logs (gzip, caching etc.)
  • Install Bazaar(Bzr) or Mercurial(Hg) on VM and put in one of the product trunks
  • Install app to keep Bzr/Hg repository in step with Perforce repository (keep the mgmt happy)
  • Install SCons and see if it can handle a product build.
  • Prepare propaganda campaign: presentations, discussion papers, training courses. (But only if can get actual working example. Spin won't work here. Too many cynical developers :-)

Question: why is one product build taking 5 hours every day? It doesn't change that much. Why aren't previously compiled parts being used? Why does a full compile have to happen? (My guess is the VCS and build tool(s) haven't been reliable enough to risk using already compiled components. Another fatal flaw in CC (and P4?)?)


The good news about attempting this project is that if I fail I can walk in six months' time; no need to wear the current shit for any longer than necessary. The IT job market is so good at the moment I have to consider opportunity cost. I know I can earn 50% more at at least one job I was offered. I have a duty to my shareholders to maximise my company's profits :-). So if I can't get any committment from the mgmt here to my project there'll be no hard feelings on my part when I leave.

Sunday, May 25, 2008

The hard part of a new job

One of the hardest parts about starting a new job is when the previous incumbent has left with no handover. In this case my predecessor had spent 8 years in the role and built layers upon layers of "obvious" decisions which might or might not be still valid. How do I find out? I'm tired of pestering people who have spent decades in their job with "trivial" questions. Two weeks now of pestering questions and I don't feel any closer to understanding why they are doing things in such a complicated manner. The tangled interconnections are almost completely opaque.

For instance, the main system is written in Perl on a Unix box. But it fires off processes onto a variety of machines running various OS's and the critical one runs on a Win box. OK, simple enough. Just fire up Perl in a CMD window and step it through. OK, simple enough. First section opens up a Lotus database. Say what?!! Is Lotus still alive? Database is actually a text file but it's maintained via a Lotus Notes application running on IE6. Say what?!! It's only a friggin' text file. Why is Lotus Notes involved? OK, simple enough. Let's log in to the Lotus app. "You don't have permission!" I'm the owner of this system but I don't have permission to access a critical section of the app because Lotus Notes' permissions don't synch with LDAP.

So now I'm in the middle of my usual crisis of confidence. Do I really want to take on eight years or more of crap and try to clean out the Augean stables? Or do I blow the whole lot away and start again? And how do I convince the rest of this sorry crew that the new way is better than what they've been using for close to a decade?

And the really hard bit is that my immediate supervisor doesn't seem to care what happens but he won't give me root permission on the main box so I can't install any sandboxes to try things out. The other problem is that this organisation sells proprietary products and seems very reluctant to use Free/Open source stuff. Case in point: they can't afford to buy everyone an Adobe Acrobat licence so I can't install PDFMaker to allow me to output PDFs from my Word docs. But instead of installing a freebie (like PDFCreator) I've been told I have to take my Word document to one of the designers who has a licence and he/she will output the PDF for me. Madness! And it seems typical of the attitude here: we can't afford a proprietary licence so we have to do it manually because we don't want to use free tools.

Well too bad. I installed PDFCreator, installed Sumatra, removed Adobe Reader and now I can create and view PDFs quickly and easily without all the Adobe Reader's constant prompts asking me to update it.

The next problem is 2-fold. How to get rid of Lotus Notes dependency? My immediate feeling is: "Install Catalyst and simply replace the web app". The problem is my employers sell a proprietary web app (which took me over a week to config) which could be squeezed into handling this task. Do I want to spend a couple of weeks beating it into shape or should I just quietly install the Cat app and not tell anyone? They've been quite happy to use Lotus Notes so it's not as though I'd be throwing out one of their products.

The 2nd part of the problem is that the need for a Lotus Notes database is purely to handle their way of tracking ClearCase tasks. And that is because of CC's peculiar way of handling branches. (Well that's not strictly true, it's more because of the way they have chosen to use CC). So if we are moving to Perforce why should I bother with a task tracking system?

All this previous raving assumes I am willing to continue with this job. I'm asking myself whether I want to take on the weeks and months of effort needed to complete this conversion. I'm not sure.

I find myself thinking of that book: "The Three Signs of a Miserable Job" (i.e. anonymity, unmeasurability, irrelevence) and this job is very close to three strikes and I'm out. "Anonymous" because my boss seems to be unwilling to volunteer any help or assistance. I don't know if he resents my taking this job (he apparently remains a good friend of the previous incumbent) and my paranoia makes me think he wants me to fail. I don't have anyone to talk with here in my little alcove. "Unmeasurability" because I have no way of assessing what "good work" is and I certainly haven't been given the tools nor the authority to do what I think necessary. And "irrelevence" because the reality is the developers could quite easily handle their build tasks themselves, they resent having this conversion being forced on them by management. And it also looks like the build system is way too complicated with so many obscurities I get the impression the developers have given up trying to understand. I know I would have given up if I were in their position.

So I will let my doubts bubble away in the background for a bit longer. I go through this stage at every new job and it will pass if I stay. It took me over a month at the previous job so I'm not giving myself enough time.

Thursday, May 15, 2008

Converting choral music scores to MP3s

I sing in a choir. Apparently music and programming are closely related (can't find link) and nearly everywhere I've worked it didn't take long till I found other programmers who sang or played or wrote or painted or whatever. For many (like me) it's an absorbing hobby. For others programming has been the day job and their "real" life was the music etc.

I've come to classical choral music very late in life. I grew up with the popular music of the time, got into blues and jazz via guitar playing in my 20s and slowly my musical interests have expanded. But when I turned 50 I decided I wanted to learn to sing properly (tired of the sore throats) and my singing teacher suggested I join a choir to give me some focus. I'm kicking myself now for not having learned to sight read music and not persisting with my occasional attempts to learn keyboard. I insisted my children learned to read music as part of their readin', writin' and 'rithmetic but didn't apply the same logic to myself. D'oh!

So how do I learn to sing my part (tenor) in the choral pieces we perform? Software of course! I reasoned that there must by now be reasonable music OCRing software, Googled a bit and found only two products that continue to be supported. The various other attempts over the years seem to have died. Choral music tends to be "dead white man's music" and thankfully most of it is out of copyright. There is the wonderful Choral Public Domain Library which can often supply an electronic version of a work, often in MIDI format (my preference), sometimes in Sibelius or Finale format (which is useful with Finale's free player), but often only a PDF version is available.

Of course CPDL is only the tip of a vast repository of choral music sequestered in various libraries and archives around the world, probably never to be unearthed and performed. One of our choir members visited the Vatican and was allowed to hand transcribe a simple choral work he found in an archive there. Our choir was the first to perform the work outside of the Vatican. Another piece we perform annually was written 900 years ago. This is simply "mind-blowing baby" as Austin would say.

Back to music OCRing. The two products that seemed worth a look are PhotoScore and SharpEye. PhotoScore is the more heavily promoted and costs more but a Lite version is bundled with Sibelius so the cost is often hidden. The only problem is that when I tried PhotoScore a couple of years ago it was awful. I spent hours having to manually correct all the bits it failed to comprehend. In desperation I went looking again and found the wonderful SharpEye. It simply works. It's accurate and complete and quite inexpensive.

The main problem for me was that I am a Mac zealot and SharpEye only runs on Windoze. Initially I had to run it on Virtual PC and it was painfully slow but then Parallels was released and I ordered a MacBook the next day. SharpEye under Parallels is easy to use. So the procedure is to either download a PDF of the score or scan it in. Then I let SharpEye OCR the score. Usually there are some minor note value errors. (But it has even read hand-written scores and was reasonably accurate.) Compared to PhotoScore, this was heaven.

The final irony was to read a recent press release announcing that PhotoScore now uses the SharpEye engine. But the demo version I tried didn't seem any better than the older versions.

SharpEye exports in MusicXML format and both Finale and Sibelius can import MusicXML. MusicXML is actually the only 'full-content' format I can use to transfer files from SharpEye to Sibelius. The other common format, MIDI, doesn't handle lyrics.

So I import the MusicXML file into Sibelius and clean up the score a bit more. I simply play it and usually I can hear the errors (sharp/flats missing, sometimes (rarely) a note at the wrong pitch, more often timing errors (crotchets/quarter notes for minims/half notes or vice-versa). I switch from Sibelius' default instruments for voices (very soppy oohs and aah) to a nice piano and use a macro to set all the dynamics marks (mf, p, pp etc.) to 'f'. I want to hear the notes and ppp is usually too soft.

Then I export the file as a MIDI and import the MIDI into GarageBand. Now there ought to be scripts that would create four MP3 files for me, each one with a different part being played loudly and the other three parts being played quietly in the background. I tried a couple of Automator scripts but they hung. I tried manually creating an Automator script myself but it also hung at a crucial point so it makes me think the interface to GB might be faulty. Finally I FTP the MP3s to the website ready for choir members to listen and learn.

Now this whole tedious, time-consuming process ought to be largely automated. Obviously manual correction needs to be done as some point but I keep wondering what a more automated process would look like. I wonder if such a Score-to-MP3 system exists (more Googling required I think). Maybe I if learn a bit more about MusicXML I could use a Perl script to extract the four parts and push them through some sort of MP3 creator. It's a thought.

Wednesday, May 14, 2008

New job: surveying the land

So the job...

I don't know if it's just my way of wasting time because I don't want to be disappointed when I get to the nub of a task but I tend to look around a lot when I undertake a new task. Maybe it comes from using Perl so long. I usually Google for relevant topics. A couple of people mentioned the Unix/GNU make utility and I wanted to look at build/release systems to see what open-source and proprietary offerings are around. Found a good review on Freshmeat which led me to ClearCase and Perforce for proprietary offerings and cons and Scons for FOSS offerings. (Yes I know they address different problem domains, this is just my meanderings around the issue. I'm also aware of Subversion, Git, SVK, CVS, RCS and SCCS. And I'm aware of Ant and Maven. All of them address various aspects of the issue. Not all are relevant any more.)

Company uses ClearCase and Perforce for version control, wants to standardise on Perforce and wants legacy code moved from CC to P4. Current in-house build system uses make and Ant depending on code base. (And use of make would have to be mainly historical. Ant seems so much better, at least in my use of it.)

cons and Scons as replacements for make deserve a closer look. cons seems to have run out of interested developers. Written in Perl, parent of Scons, languishing from lack of publicity and support on an obscure Indian manufacturer's site, occasional pleas from users asking for such-and-such feature or bugfix but no one seems willing to put up hand to help. (Is this a job for EC?)

Scons on the other hand, while based on cons, is written in Python, thus gaining some flavour-of-the-month interest but more importantly, is visible to Google (and has thus qualified for Summer of Code support). Also seems to have an active and interested community behind it. And a web page designer! (Pleasant, clean, modern home page. Makes it look alive cf cons.)

Don't ever tell anyone :-), but I don't always obey my boss when told to develop something. I listen carefully to what he/she asks for and then I attempt to do two things. One is I try to work out what they really want/need. Not just in the political sense. I've worked with sociopathic bosses who think they can control me by not telling all the details of the requirements or who's paying for the job or who gets rewarded if it succeeds. One guy told me I had six weeks to complete a task but I accidentally (really!) read an email to his boss where he mentioned the 12 weeks he had allocated to the task. Turns out he was paid a bonus for every week under 12 he could extract. Bastard.

Politics aside, sometimes they can only think in terms of existing systems so the request will be 'one of them but this bit changed'. I have to look at the wider picture. 'One of them' might be so old and poorly maintained and so slow and resource hungry that a second one might destroy the machine or require a new one (not budgetted for of course).

There's also the much harder task: how do I stay interested? I can think of better things to do than spend 8 hours a day doing boring, repetitive, code monkey stuff. I like singing. I like playing guitar. I like talking to my wife and kids. I like walking the dog. I don't like boring shitwork. I'm also aware that there are a few million University graduates scattered around various "developing nations" perfectly capable of churning out a couple hundred lines of Java crap for less than half of what I get. So my code has to be compact but maintainable, precise but not obscure, correct but not overkill. And I have to stay interested. The only way I know how to do this is to find a way to turn a boring, churn-the-handle task into something exciting.

One way to do this is to use a new technology. This is how I discovered JQuery. A webpage needed some JavaScript but my first attempt clashed with the existing JS. And of course it didn't work correctly in IE6. Googled and found JQuery. Oh frabjous day! Suddenly JavaScript became interesting. So much power in such a compact notation. Sounds like Perl! And it automatically handled the PITA differences in IE6.

Succinct code

So the job...

Existing build system written in very clean and clear Perl. One of the advantages of Perl is that code is usually all the doco one needs if it's written by a reasonably competent Perl programmer (i.e. not a C or Java monkey in Perl clothing). Interestingly, a reasonably competent Perl programmer usually also adds useful/relevant comments. Java programmers OTOH love to put in comments like "the following adds 1 and 1 to make 2", followed by '$result = 1 + 1;'. Life is too short for this type of idiocy. I recently looked at a 600-line Perl script which did less than ten lines of xsh code. I also remember a few years ago a Java columnist demonstrating that now that Java had a regular expression library he could write his particular script in only 100 lines! The equivalent in Perl is a one-liner. 100 lines of code rots the brain if there's only one relevant line. Which is the relevant one? How much energy do I have to expend to process the other 99 lines to work out they are not the nub of the code? Of course, Java and many other languages aren't as succinct so one's thoughts are never as clear or focussed, they are always diffused over the other 99 lines.

New job

This blog is probably going to be more of a Twitter-like stream of consciousness rather than a considered, pondered assessment and evaluation of my situation (i.e. it's like every other blog).

Started new job as Build/release engineer for commercial organisation. (Spelled with an 's' immediately tells you which country. Job title immediately tells you the city and the organisation.) I'm a Perl programmer. Been one since 1990.

I spent 15 years prior to 1990 writing C code and then this new 'dynamic scripting' language Perl came along and said 'let me take care of the garbage (i.e. memory management) and you take care of the interesting part (i.e. writing code)'. Suddenly 90% of my coding time was freed. Oh, and Perl also let me forget about the intricacies of awk, sed, grep, join, cut, bash/csh/tcsh/ksh/sh (sure I occasionally use them but never for anything complicated). Suddenly one ring ruled them all.

Over the years since then I've looked long and hard at a lot of languages including VB, Java, C++, Smalltalk, Eiffel, Python, Ruby, Erlang, Haskell (the list goes on and on). All of them have attractive aspects. I remember reading once that each language is designed to solve a particular problem. (Maybe in that sense all programming languages are Domain-specific languages.) Fortran (Formula Translator) and COBOL (COmmon Business-Oriented Language) were early examples of this. Perl was invented supposedly because Larry Wall was too lazy to learn how to use all the Unix utilities and too impatient to wait for them even when he could get them to link together to process his report data.

Steve Yegge's talk on Dynamic Languages seems to confirm my own experience. It's just too hard to switch languages these days. In my case the Perl CPAN has over 10,000 downloadable plugin modules. It takes a lot of time to sift through 10,000 modules to find the ones which I am comfortable with. I can learn the syntax of a language in a couple of days but how long does it take to understand the subtleties of the Schwartzian Transform and why one would use it? Each language has such 'deep learning' aspects and they take a long time to grok.

I omitted JavaScript from my list above because I am finding I get more and more enjoyment from writing JS each day. JQuery has got to be something akin to manna from heaven.

Yegge's comments about Java compilers and language support tools effectively running the program and looking inside to see what it actually does rather than trust to the static declarations gives me great hope that a Perl programmer somewhere (won't be me: see blog title) will finally bite the bullet and run with the fact that only Perl can truly understand a Perl script and start writing IDEs and refactoring tools which are actually useful.