Monthly Archives: August 2012

Eight hundred years ago, a keyboard was a series of pegs or bars used to control a pipe organ, with each key opening a valve to admit air to a particular group of pipes. It started to take its modern shape in tandem with the development of musical notation. Both had become standards that we would recognise today by the middle of the Renaissance Era, around 1500. With the invention of escapement mechanisms, keys were united with strings, and the first spinets, clavichords and harpsichords appeared. The keyboard as a control interface continued to evolve as the instruments that it served proliferated and matured.

Early keyboard instruments were limited in ways that today’s instruments are not. Only a small class of them could be controlled by changing the speed at which a player’s fingers hit the keys, and these devices provided insufficient power to perform to a concert audience. Louder instruments required a mechanical plectrum to pluck the string from a fixed height, so that any manner of keypress resulted in the same sound.

To alter the tone and character of music, concert instruments started to resemble organ consoles, possessing two or more manuals and a large number of drawbars and stops. Octave doubling, Venetian swell, and auxiliary strings were variously employed to provide some dynamic versatility, but it was not until the invention of the fortepiano around 1720 that a keyboard instrument could combine the subtlety of finger-controlled dynamic range with the power of a concert instrument. Early pianos feel and play like development prototypes: they are quiet, feel insubstantial, and fall out of tune if somebody closes a door too quickly. Fortunately, the Industrial Revolution accelerated their development, mutating the fortepiano into a pianoforte by replacing the wooden frame with steel so that strings could be longer, tighter, louder, and maintain better tuning. This provided the strength and stability to withstand the additional tension of two or more octaves, and allowed second and third strings to be fixed to the higher notes to balance them with the power of the lower ones. The newer bass strings were overstrung with the others to make the resulting instrument more compact, and the grand piano took its distinctive, curvy shape. Pedals were added: one to lift the dampers, and one to soften the treble by shifting the hammers so that they could not contact the third strings. For keyboard musicians, however, an equally significant improvement was the invention of the double escapement in the 1830s.

Whereas keyboards had formerly required each key to be released to its starting position to sound again, double escapement allows a player to retreat the key of a sounded note by a few millimetres, and then strike again to repeat it. It permits a greater palette of playing styles, and accommodates figurations that are both quiet and fast. Certain elements of compositions, such as the note or chord tremolandi that are favoured by some modern composers, would be physically impossible without it. The double escapement is fiendishly complicated: it relies on a moving assembly poetically called the wippen. This couples the key to its hammer via a number of levers, moving linkages, and adjustable screws. It is sufficiently complex, and setting it up is such an art, that it would probably not have been invented or popularised had Victorian engineers had access to electronics. They didn’t, and their legacy is the brilliant and complicated key mechanism that accounts for much of the cost and labour of a modern concert piano.

Of course, the story doesn’t end there. Our forebears composed for many types of keyboard instrument, and so do we. The electronic era has bestowed upon us electroacoustic instruments such as the Rhodes and Wurlitzer pianos. These were designed, in the spirit of the clavichord, to be portable pianos. Because the electronics did some of the work, the hammers and dampers could be smaller, which made the escapements simpler and lighter. One such instrument, the Hohner Clavinet, is little more than an amplified clavichord, reminding us that innovation can be as retrospective as it is progressive. Add to this other classic electronic instruments with no finger-controlled dynamics – the Hammond organ, the Mellotron, and a plethora of classic analogue synthesisers – and it is clear that keyboard players can now choose from an incredible legacy of beautiful, but very different, instruments.

Today, our customers want the sound of these instruments without the liability of ownership. None of these instruments is simple to tune or maintain, and their scarcity, fragility and complexity makes them expensive. Part of Novation’s business is manufacturing MIDI controllers that allow an inexpensive key mechanism to be joined to synthesisers or samplers to reproduce these sounds. Our slogan, It’s The Feel, is also our mission, and it pays no small tribute to five hundred years of progress.

Selling MIDI controllers with such a statement is bold. We run the risk of being compared against not just the responsiveness of a vintage instrument, but its gestalt. A Rhodes piano isn’t just a sound, and its escapement isn’t just a feel. The joy of a Rhodes is just as much in the semiotics of its faded, off-white keys, the weight and rattle of each note as it’s deployed, and the way that the whole keyboard buzzes under the fingers as it is played. It’s the smell of dust and old solder flux, and the black fabric, rounded thermoplastic, and kitsch chrome detailing of a vintage instrument. And, of course, it’s just as much the smoke-filled photographs of jazz and rock legends of the Sixties and Seventies teasing immortal melodies from its keys. In all their flaws, instruments like the Rhodes are evocative and compelling because they are culture, and they are history. Many musicians refuse to play a sampled facsimile, no matter how indistinguishable it is from the original once their track is laid down, because if the performer doesn’t feel the same, the performance won’t be the same.

So, when we make electronic controllers, we find ourselves perched on the shoulders of giants, but sometimes wishing that they’d let us down for a few minutes so that we can take a walk and see the sights ourselves. Meanwhile, we contrive to stage re-enactments of the playing experience of a few favourite keyboards, and to elicit as much cultural meaning from them as possible. The Feel will never be the real thing, but our customer is buying a MIDI controller, rather than trawling classified adverts to become the custodian of an heirloom. What we provide must be an amalgam of everything they need to do, with a cost of ownership that they can afford and an ease of use and portability that mechanical instruments cannot touch. We remember that our art is forever shifting. Our mission is to make the most versatile and playable keyboard we can, and to discard those frustrations of real instruments that performers often forget.

Having set this stage, my next posting will discuss the design of MIDI controller keyboards, the choices we make about them, and what we can do to make them better.

A few colleagues and I were having difficulty accessing our SVN repository via https when working on our Macs at home. Specifically, an SVN checkout operation would hang. A subsequent SVN update operation would complete the checkout, but also hang prior to completion. We had just been living with this for a while but there comes a point when an engineer is annoyed so much that he or she feels compelled to do something about it! In this post I explain how I solved the problem.

Step 1: Enable Debug Logging

The first step of course is to use the web search engine of your choice (WSEOYC) to see if anyone else has encountered the problem. I tried this but to no avail. Another worthy port of call is to check your system logs for relevant information. On the Mac, I have lost count of the number of times I have forgotten to check the Console application for trace that ended up being the key to solving the problem, and it can even provide more useful text to feed back into the WSEOYC!

The next step is to vary the parameters of the problem. I found that I was successfully able to check out a codebase belonging to another company over https. But that didn’t help me fix our problem.

After much usage of the WSEOMC and checking of system logs I got no further. It was only then that I discovered from Dominic Mitchell’s Jabbering Giraffe blog that the command-line svn has a debug logging option for its network requests. Edit the file:


and you will find a line relating to the ‘neon debug mask’. Never one to do things in half-measures, I promptly updated this to enable all the logging features as follows:

neon-debug-mask = 511

Then when I examined the debug trace from the (now very chatty) SVN checkout operation that was hanging, I noticed the following:

    sess: Closing connection.
    sess: Connection closed.

The hanging was occurring just after “Closing connection” was output to the log file.

Step 2: Identify the Faulty Component

I guessed from the debug trace that there was a problem with the networking side of things. But how could I debug this?

My first search revealed a useful program called DTrace, an incredibly powerful tool to investigate almost any aspect of the way a system is running, using a language called D. MacTech’s article Exploring Leopard with DTrace is a great introduction. Unfortunately it looked overkill for the problem at hand so I will have to save that one for a thornier problem another day!

But the networking clue allowed me to find the following very helpful page from Davey Shafik: how-to-fix-svn-apache-ssl-breakage-on-os-x. Although relating to a different problem, the page was incredibly helpful so I definitely owe Davey a beer!

In particular, his problem related to a bug with libneon, a library bundled with OS X that handles HTTP and WebDAV requests. It occurred to me that maybe my problem also related to libneon. So I decided to try his first suggestion, “Upgrade the system libneon (bad idea, as OS X can overwrite it in any update)“. I figured that if OS X updates this library then that might even fix the issue; and if it doesn’t then I could always restore my version of the library after the update.

Step 3: Fix or Replace the Faulty Component

So how to update libneon? I didn’t much fancy compiling it from source myself, but again, Davey’s page introduced me to Homebrew, a package manager for OS X. After ensuring that Xcode or the Command Line Tools for Xcode are installed first, Homebrew can be installed just by running:

ruby <(curl -fsSk


After installing Homebrew I simply ran:

brew install neon

and then copied the newly compiled library over the system library:

cp -p /usr/local/Cellar/neon/0.29.6/lib/libneon.27.dylib /usr/lib/

after judiciously backing up the original library! This updated libneon from version 0.29.0 to 0.29.6 (see revision history). Then I made sure that the group, owner and permissions of the new library matched that of the old.

Step 4: Test the Fix

Et voilà! No more hangs when running svn from the command-line. Substituting the old library caused the issue to occur again, so it is seemingly definite that the libneon component was the issue.

The likely cause of the issue is the bug that was fixed in libneon 0.29.3: Change ne_sock_close() to no longer wait for SSL closure alert: fixes possible hang with IIS servers when closing SSL connection.

Now the problem is fixed once and for all which makes me a happy engineer as I have solved a technical problem that was causing us pain. It’s surprising what a little bit of applied effort can do even in the face of an initially completely mysterious problem!