I’ve just come back from EuroClojure 2012, where there were a number of Overtone talks and a number of tweets asking for music theory resources aimed at computer scientists. This will hopefully blog number 1 in a series on that theme.

Note that the code examples are designed to be pasted into a Clojure repl, so that you can take the code and play with it yourself.

The basic atoms of synthesizing sounds in Overtone are oscillators. An oscillator takes a frequency and makes a noise. Here are some examples:

(use 'overtone.live)
(demo (sin-osc 440))  ; sine wave
(demo (saw 440))      ; saw-tooth wave
(demo (square 440))   ; square wave

In each of these, we’re making an oscillator and giving it a frequency of 440 Hz — or equivalently, 440 cycles per second (cps). The problem we find is that most music isn’t defined in terms of frequency, it’s defined in terms of notes of the scale: C D E F G A B C and all the sharps and flats in between. How do we play a tune on a sine wave generator when the tune is made of notes rather than frequencies?

First, let’s write some helpers to play a sequence of frequencies through a sine wave:

(definst sine-wave [freq 440]
   (sin-osc freq))
;=> #<instrument: sine-wave>
(defn play-freqs
      ([freqs] (play-freqs freqs (now) (sine-wave (first freqs))))
      ([freqs time inst]
        (if-let [freqs (seq freqs)]
          (do (at time (ctl (:id inst) :freq (first freqs)))
              (apply-at (+ time 300) #'play-freqs [(rest freqs) (+ time 300) inst]))
          (at time (kill (:id inst))))))
;=> #'user/play-freqs
(play-freqs [440 660 330 440 220 880 220])
; *beautiful melody*
;=> #<ScheduledJob id: 76, created-at: Sat 09:32:26s, initial-delay: 0, desc: "Overtone delayed fn", scheduled? false>

We define a simple intrument, sine-wave, which has one parameter, freq. We can start the instrument with a particular frequency by writing (sine-wave 440); this returns a map of data about the particular oscillator instance which is generating the note. We can change the freq parameter of a running instrument by using (ctl (:id inst) :freq 660). Finally, we can stop a running inst with (kill (:id inst)). (And if it all goes horribly wrong, we can stop absolutely everything with (stop)).

Our goal is to be able to use play-freqs to play tunes made of keywords rather than frequencies: [:c4 :d4 :e4 :d4 :c4].

Frequency of notes from first principles

You can calculate frequency from pitch using only four fundamental axioms:

1. When an orchestra tunes up at the start of a rehearsal, they tune to A. A is normally defined to be 440 Hz.
2. Going up one octave is the same as doubling the frequency. That is, one octave above tuning A is 880 Hz, and one octave below is 220 Hz.
3. There are twelve semitones in an octave.
4. All semitones are equally sized.

If going up an octave doubles the frequency, then going up twelve semitones must also double the frequency. This means we must find the number semitone, where:

(nth (iterate #(* semitone %) 440) 12)
;=> 880

In other words, we want semitone^12 == 2, so semitone must be the twelfth root of 2:

; from the contrib library [org.clojure/math.numeric-tower "0.0.1"]
(require '([clojure.math.numeric-tower :as 'math]))
;=> nil
(def semitone (math/expt 2 1/12))
;=> #'user/semitone
(nth (iterate #(* semitone %) 440) 12)
;=> 880.0000000000003
; or alternatively:
(* (math/expt semitone 12) 440)
;=> 880.0000000000003

We can now define a function to find a frequency a given number of semitones from tuning A:

(defn semitones-from-a [semis]
  (* (math/expt semitone semis) 440))
;=> #'user/semitones-from-a
(semitones-from-a 0)
;=> 440.0
(semitones-from-a 12)
;=> 880.0000000000003
(semitones-from-a 3)
;=> 523.2511306011974
(semitones-from-a -9)
;=> 261.6255653005985

Normally, however, we use MIDI notes as a numerical representation of notes, rather than displacement from tuning A. In the MIDI note system, tuning A is defined to be 69, and going up or down one semitone increases or decreases the note value by one. So, for example, middle C is 60. We can get from midi notes to frequencies like this:

(defn midi-to-hz [midi-note]
  (semitones-from-a (- midi-note 69)))
;=> #'user/midi-to-hz
(midi-to-hz 69)
;=> 440.0
(midi-to-hz 81)
;=> 880.0000000000003
(midi-to-hz 72)
;=> 523.2511306011974
(midi-to-hz 60)
;=> 261.6255653005985

In fact, Overtone provides a function midi->hz to do exactly this transformation:

(midi->hz 69)
;=> 440.0
(midi->hz 81)
;=> 880.0
(midi->hz 72)
;=> 523.2511306011972
(midi->hz 60)
;=> 261.6255653005986

So with a melody as MIDI notes, we can play it as follows:

(play-freqs (map midi->hz [60 62 64 62 60 72 67 60]))

If you’ve got this far, the maths is over. The last mile is to be able to use keywords instead of raw MIDI values. Overtone provides a function for this called note:

(note :a4)
;=> 69
(note :a5)
;=> 81
(note :c5)
;=> 72
(note :c4)
;=> 60

So we can play our tune by chaining this with midi->hz:

(play-freqs (map (comp midi->hz note)
    [:c4 :c5 :e4 :f4 :g4 :f4 :e4 :d4 :c4 :c4]))
; *beautiful music*

Permalink | Leave a comment  »

I was reading the Clojure source code the other day when I noticed this curious snippet in LispReader.java:

static public int read1(Reader r){
    try
        {
        return r.read();
        }
    catch(IOException e)
        {
        throw Util.sneakyThrow(e);
        }
}

I immediately thought to myself: “what does sneakyThrow do?” It looks like it is a magic way to throw a checked exception without needing a throws declaration. But how does it work?

First, a little background. Checked exceptions are effectively a static analysis tool: there are no runtime checks on checked exceptions. Rather, javac will refuse to compile code where a checked exception is thrown with no catch block to catch it nor throws declaration to declare its propagation.

It’s possible to throw a checked exception using bytecode manipulation, or Thread.stop(Throwable), and these techniques have been known for at least a decade. However bytecode manipulation is messy, and Thread.stop(Throwable) has been deprecated for at least a decade too. Is there a pure-Java way to throw a checked exception sneakily?

C-family languages normally provide typecasts, a trapdoor to escape their static typing system when you think it is more hindrance than help. So a first attempt might go something like throw (RuntimeException) e;. However if you try this in the above code, you will get a ClassCastException at runtime, because IOException is not an instance of RuntimeException. It would seem that there is no pure-Java way to throw a checked exception.

So how does sneakyThrow work? Here it is, in all its glory:

/**
 * Throw even checked exceptions without being required
 * to declare them or catch them. Suggested idiom:
 * throw sneakyThrow( some exception );
 */
static public RuntimeException sneakyThrow(Throwable t) {
    // http://www.mail-archive.com/javaposse@googlegroups.com/msg05984.html
    if (t == null)
        throw new NullPointerException();
    Util.sneakyThrow0(t);
    return null;
}

@SuppressWarnings("unchecked")
static private  void sneakyThrow0(Throwable t) throws T {
    throw (T) t;
}

That link in the comments gives credit to Reinier Zwitserloot who, as far as I know, had the first mention of this technique in 2009 on the java posse mailing list.

What we have here is a severe abuse of Java. Util.sneakyThrow(t) calls Util.sneakyThrow0; then within sneakyThrow0() we cast to the parameterized type T. In this case that type is RuntimeException. At runtime, however, the generic types have been erased, so that there is no T type anymore to cast to, so the cast disappears.

In other words, we’ve managed to convince the compiler and the runtime that they’re seeing different things. The compiler sees the code with the cast:

throw (RuntimeException) t;

so it allows the now-unchecked exception to propagate. The runtime doesn’t see the generic types, so it sees no cast:

throw t;

and therefore it doesn’t complain about a ClassCastException.

There was one last nagging thought I had about the original code:

throw Util.sneakyThrow(e);

Given that Util.sneakyThrow(e) throws the exception itself, why does the calling code also use throw? The answer is, once more, to make the compiler happy. Without the throw, the compiler will demand a return statement afterwards.

Reinier Zwitserloot added this functionality to Project Lombok as the @SneakyThrows annotation, so now you can propagate checked exceptions sneakily with minimal boilerplate. The @SneakyThrows page also summarizes neatly some use-cases for why you would ever actually want to throw a checked exception:

• You are calling a method which literally can never throw the exception that it declares. The example given is new String(someByteArray, "UTF-8"), which declares that it throws UnsupportedEncodingException but UTF-8 is guaranteed by the Java spec to always be present.
• You are implementing a strict interface where you don’t have the option for adding a throws declaration, and yet throwing an exception is entirely appropriate — the canonical example is Runnable.run(), which does not throw any checked exceptions.

The first case is clear — the throws declaration is a nuisance and any solution to silence it with minimal boilerplate is welcome.

The second case has one common alternative: wrapping the checked exception in a RuntimeException so that you can throw it. Both approaches will have their critics. Wrapping an exception just to gain the privelege of throwing it results in a stacktrace with spurious exceptions which contribute no information about what actually went wrong. On the other hand, throwing checked exceptions may violate the principle of least surprise; it will no longer be enough to catch RuntimeException to be able to guarantee catching all possible exceptions.

It will be up to any given project to decide which is the lesser of two evils and establish a standard on their codebase.

Permalink | Leave a comment  »

I’ve been working with Ruby and Sinatra this week to write some RESTful interfaces. I’m new to Ruby and Sinatra, but I’m not new to dynamic languages, RESTful services, or the Rack / WSGI / PSGI / Ring view of web applications. I figured that this shouldn’t be a difficult task. Boy was I wrong.

Sinatra offers a very nice DSL for describing HTTP controllers. For example, a simple “Hello, world” controller looks like this:

require 'sinatra/base' # this is 'modular' style

class HelloWorld < Sinatra::Base
  get '/*' do
    'Hello, world!'
  end
end

Sinatra also conforms to the Rack specification: that is, Sinatra controllers expose a call method which (to a first approximation) takes an HTTP request and returns an HTTP response. We thought that having a nice simple interface to talk to the controller would make testing and test-driving very easy; and in fact Rack::Test::Methods provides a pleasing DSL for testing Rack applications through the call method:

require 'rspec'
require 'rack/test'
require 'hello-world'

describe 'Hello World application' do
  include Rack::Test::Methods

  def app
    HelloWorld
  end

  it 'should return Hello, world at root URL' do
    get '/'
    last_response.body.should match /Hello, world/
  end
end

The trouble started because we wanted to stub out dependencies of the controller. Our “Hello, world” controller grew to look something like this:

require 'sinatra/base' # this is 'modular' style

class HelloWorld < Sinatra::Base
  def repository
    raise NotImplementedException
  end

  get '/*' do
    repository.get('foo')
  end
end

We’re practising top-down TDD, where we don’t try to create dependencies until we’ve found a need for them at a higher level. At this point, we wanted a test to check this code. To do this, we’re going to have to stub out the repository with a test fake Here’s attempt #1:

require 'rspec'
require 'rack/test'
require 'hello-world'

describe 'Hello World application' do
  include Rack::Test::Methods

  def app
    HelloWorld
  end

  it 'should return Hello, world at root URL' do
    mock_repo = mock('repository')
    mock_repo.stub(:get).with('foo').and_return('bar')
    app.stub(:repository).and_return(mock_repo)
    get '/'
    last_response.body.should == 'bar'
  end
end

We figured that since app defines the application under test, we could just stub out the repository on the app and we’d be done. Sadly not; the app called the real repository method, not our stubbed version.

Surprise #1: HelloWorld and HelloWorld.new are both Rack applications!

Sinatra provides your controller classes with a call method, which internally implements the Singleton pattern and will new up a controller instance before delegating the call to the controller instance.

I have just one question: why?. Is it just so that I can save 4 characters by typing def app; HelloWorld; end rather than def app; HelloWorld.new; end? Sure, it saved typing, but it confused the hell out of us.

Okay, let’s leave the controller class magic behind. Here’s attempt two:

require 'rspec'
require 'rack/test'
require 'hello-world'

describe 'Hello World application' do
  include Rack::Test::Methods

  def app
    HelloWorld.new #let's have an instance now
  end

  it 'should return Hello, world at root URL' do
    mock_repo = mock('repository')
    mock_repo.stub(:get).with('foo').and_return('bar')
    app.stub(:repository).and_return(mock_repo)
    get '/'
    last_response.body.should == 'bar'
  end
end

Now, app is pointing to a HelloWorld instance, so we should be able to stub out its dependency, right? Wrong. It still calls the real repository method.

Surprise #2: HelloWorld.new doesn’t create an instance of HelloWorld.

Seriously, have a look yourself:

$ irb
>> require 'hello-world'
=> true
>> HelloWorld.new.class
=> Sinatra::ShowExceptions

WAT.

What is going on here? Well, Sinatra has overridden Object#new with its own new method, so that when you new up a Sinatra controller you get all sorts of Rack middleware for free. This middleware does nice things like catching and formatting exceptions. That’s all fine. But why is Sinatra’s method of attaching middleware the wholesale replacement of a core language feature?

What this means is that we weren’t stubbing repository on our app at all; we were stubbing it on Sinatra::ShowExceptions, the outermost middleware layer. Deep down the middleware stack, our app still had its real repository method intact.

Thankfully, Sinatra aliases the original Object#new as new! in your code, so if you want a naked controller, you can still get one. Here’s attempt #3:

require 'rspec'
require 'rack/test'
require 'hello-world'

describe 'Hello World application' do
  include Rack::Test::Methods

  def app
    HelloWorld.new! #let's have an unadulterated instance now
  end

  it 'should return Hello, world at root URL' do
    mock_repo = mock('repository')
    mock_repo.stub(:get).with('foo').and_return('bar')
    app.stub(:repository).and_return(mock_repo)
    get '/'
    last_response.body.should == 'bar'
  end
end

Unfortunately, this still doesn’t work, even though we have a real instance of HelloWorld and we really are stubbing repository on it. Why?

Surprise #3: Sinatra dups your controller before handling an HTTP request

From ‘sinatra/base.rb’:

# Rack call interface.
def call(env)
  dup.call!(env)
end

So when you call your Sinatra instance’s call method, it’s not your instance at all that handles it, it’s a dup of that instance. And the dup doesn’t have its methods stubbed like the original did.

At this point, I pause and realise that if I’m fighting something this hard, it’s probably because my mental model doesn’t match Sinatra’s, and that there’s probably a way to achieve what I want which doesn’t involve having to dig around in Sinatra’s source code to work out what deep magic it is wreaking with my controllers.

The problem I have is that I already have a mental model for web application development, which is based at the Rack level. I’ve used Ring and understood it; and that means I can then go and learn Rack in about 10 minutes. Rack is simple. I like Rack.

Sinatra, on the other hand, seems to be trying to hide Rack from me. It’s a shame, because Rack seems to be the only part of the system that I understand. As a result, I’m left flailing in Sinatra’s magical kingdom wondering why the ground keeps shifting under my feet.

Permalink | Leave a comment  »

Apologies for lack of November post; I’ve been somewhat snowed under recently. Normal service now resuming…

December’s Clojure dojo focused on difficult problems from 4clojure. I had dabbled with 4clojure before on some of the easier problems, but I honestly hadn’t anticipated just how difficult the hard problems can get!

Our team decided to go in gently, going for medium-difficulty problems rather than hard problems. This turned out to be one of our better ideas of the evening, since we only managed to complete one and a half medium problems in the time available!

Juxtaposition

The first problem we tackled was Juxtaposition, in which you have to reimplement the juxt function. Our team took an approach where we tried to develop solutions from first principles, rather than looking up (source juxt), so I think it might be enlightening to compare our solutions with the clojure.core model answers. First, our solution:

(defn juxt [& fns]
        (fn [& args]
          (vec (map #(apply % args) fns))))

And the output of (source juxt) produces something like this:

(defn juxt [& fs]
     (fn [& args]
         (reduce #(conj %1 (apply %2 args)) [] fs)))

…except that the original source has special cases for small numbers of arguments.

It’s interesting the difference of approaches here. We both use the form (apply % args) to apply the variable number of arguments to each function in turn; however, we use map to do this, producing a sequence, which we then must traverse in order to convert to a vector.

The clojure.core version, by contrast, starts with an empty vector [], and conjes each further result into the vector; in doing so, it avoids traversing the list twice.

Reductions

The second problem we attempted was Reductions. Here’s our attempt:

(defn my-red
  ([f coll]
     (if (seq coll)
       (cons (first coll) (map #(f % (first coll)) (my-red f (rest (seq coll)))))
       []))
  ([f init coll]
     (my-red f (cons init coll))))

It performs well enough for bounded-length sequences:

(my-red + [1 2 3 4 5])
;=> (1 3 6 10 15)

But it fails when it comes to infinite lazy sequences:

(take 5 (my-red + (range)))
;=> StackOverflowError

We were very confused on the night as to why this should fail. Isn’t map lazy by default? Why, then, do we get a stack overflow?

The problem, which I have only found out today, 4 days after the event, is that map is a function, and therefore its arguments are evaluated before map itself is. Therefore, every call to my-red necessarily makes a recursive call, and thus exhausts the stack. The solution is to add a lazy-seq to the recursive call:

(defn my-red
  ([f coll]
     (if (seq coll)
       (cons (first coll) (map #(f % (first coll)) (lazy-seq (my-red f (rest (seq coll))))))
       []))
  ([f init coll]
     (my-red f (cons init coll))))

And thus, the previous example now works fine:

(take 5 (my-red + (range)))
;=> (0 1 3 6 10)

It still doesn’t pass all of the 4clojure unit tests, though. Work for another time, perhaps.

The clojure.core/reductions source looks like this:

(defn reductions
  "Returns a lazy seq of the intermediate values of the reduction (as
  per reduce) of coll by f, starting with init."
  {:added "1.2"}
  ([f coll]
     (lazy-seq
      (if-let [s (seq coll)]
        (reductions f (first s) (rest s))
        (list (f)))))
  ([f init coll]
     (cons init
           (lazy-seq
            (when-let [s (seq coll)]
              (reductions f (f init (first s)) (rest s)))))))

It’s a very similar approach to the problem, with some important differences:

• It actually works (!)
• It uses if-let and when-let with the seq function, relying on the behaviour that for empty sequences, seq returns nil, but also binding the returned seq simultaneously.
• It treats the [f init coll] version as the primitive form and expresses [f coll] in terms of [f init coll]. We do it the other way, purely because we implemented [f coll] first. It’s ugly, though, particularly in cases where init is not the same type as members of coll.
• It has special case handling for the (reductions f []) case — where an empty sequence is provided, it returns (list (f)).
• I believe our use of map makes our solution quadratic rather than linear in the length of the input sequence, because the item in the nth position must be transformed by (n-1) fns and we don’t reuse the intermediate results like the clojure.core version does.

Summary

This has made me want to go back and give 4clojure a closer look. I had tried the first few problems, which seemed trivially easy, but now that I’ve seen that even the “Medium” problems present a significant challenge and raise all sorts of issues around laziness, algorithmic complexity, and efficiency, I can see I’ve a lot to learn from 4clojure.

The London clojure dojo happens on the last Tuesday of every month. During the dojo, we split into groups of four or five around a single computer, and each person takes a turn at the keyboard. This ensures that even if you have zero clojure experience, you will get the opportunity to write some code at the event.

Entry to the dojo is free, but advance booking is required. Listen for announcements on the London clojurians mailing list.

Permalink | Leave a comment  »

Today was the inaugural South East England Overtone Hack Day. We met up in Cambridge to hack on Overtone, a live music performance environment in Clojure. Here are some of the things we covered:

Audiocubes

Tom (didn’t catch his surname, unfortunately) brought some AudioCubes, a set of control interfaces. They have infrared detectors and can detect nearby surfaces and other AudioCubes.

We discussed interfacing them to Overtone. They come with midi and OSC interfaces, which would be easy to work with; but the real power apparently comes from the C API, which allows you to discover the network topology of the AudioCubes. We discussed the possibilities for working with the C API from Clojure.

Other controllers

Tom and I also discussed some other controllers:

• TouchOSC
  • very low barrier to entry
  • not nice to rely on wireless connections in a live environment
• Launchpad
  • cost effective introduction to real hardware

Environment setup

As is inevitable, some time was spent getting people set up with Eclipse, Counterclockwise, leiningen, overtone, and so on. I made use of my previous blogpost on lein eclipse, which i had totally forgotten about…

Overtone basics

I went through my skillsmatter talk with Stefan, Tak and Edmund, to show them the basics of creating instruments, oscillators, and filters; and scheduling beats and tunes in time.

Signal processing basics

We also had an impromptu introduction to signal processing – time domain vs frequency domain, linear filters — low pass, high pass, band pass, Fourier series, and suchlike.

Clojure basics

Finally, we discussed resources for learning clojure itself: labrepl and 4clojure.

Overtone documentation

I also made a start on writing a filters page for the overtone wiki.

Summary

We all had a great time, we all learned something and achieved something, and there was a lot of interest for another event next month in London. So I will see you all next time!

Permalink | Leave a comment  »

I was wanting to play around with counterclockwise, the eclipse plugin for clojure, recently, when I got stuck trying to open an existing leiningen project in eclipse. If I were coding Java, I’d have no trouble with the analagous problem of importing a maven project into IntelliJ, but I struggled a bit with this one enough that I thought I’d miniblog it so I’d remember in future.

This post assumes you have Leiningen, eclipse and counterclockwise installed.

Install the lein-eclipse plugin. (You may want to check the latest version on clojars. As of writing, there seems to be a rival 1.1.0 from robertrolandorg; I’m not sure of the difference.

$ lein plugin install lein-eclipse 1.0.0
Including lein-eclipse-1.0.0.jar
Created lein-eclipse-1.0.0.jar

Run lein eclipse to create the files eclipse needs:

$ lein eclipse
Copying 15 files to /Users/philippotter/src/mobile/jquery-mobile-experiment/lib
Created .classpath
Created .project

Then, in eclipse, do “File->Import->Existing Project into Workspace”. You should now have your project imported.

Permalink | Leave a comment  »

I’m currently attending the second annual Clojure Conj, the premiere Clojure-specific conference. One of the themes that has been emerging is evangelizing Clojure: getting more people to use it, and convincing people to use Clojure who otherwise wouldn’t choose to use it.

This was in fact the central theme of Neal Ford’s talk “Neal’s Master Plan for Clojure Enterprise Mindshare Domination”, in which he put forward his ideas for how to get large organizations full of institutional inertia to adopt Clojure. Phil Bagwell also made reference to this in the introduction to his talk, in which he asked everyone using Clojure in their day job to put their hand up, then asked everyone who’s never deployed Clojure to production to put their hand up, then asked group 2 to talk to the nearest person in group 1 and ask them how they got to work in Clojure.

Clojure evangelism has also been a common theme of Q&A sessions after talks: a talk on ClojureScript will often be followed with a question such as “How do you fit this into an existing JavaScript project?”

There are a few key themes emerging:

Know your enemy

There’s a lot of competition amongst the new language communities, particularly between Scala, Clojure, and Groovy. This is absolutely fine and as it should be. Furthermore, if Scala is successful, this is in no way directly detrimental to Clojure.

Scala, Clojure and Groovy are in competition, but they are not enemies of each other. The real enemy is the status quo. It is the nasty feeling that people have when they say things like:

• “Taking on Clojure is a big risk – I want to be certain”
• “I think Clojure might be a better choice than Java, but because Java is an industry standard, I am more likely to get blamed if I choose Clojure and the project fails.”
• “If I choose Clojure, I don’t know I’ll be able to hire developers who know it”

Ultimately, these statements reflect a sentiment that staying with the same old technology is safer than trying to improve productivity by choosing a newer, but less well-known, technology.

If a client has switched from Java to Groovy, Scala, or JRuby, they have already rejected the status quo. Encouraging an environment in which people feel able to explore new technologies will make more people who are interested in Clojure overcome their fears and try it out. In other words, a rising tide raises all ships.

Be positive about the new possibilities, not negative about the status quo

By and large the feeling of the conference has been upbeat and positive, rather than tribal. That is why, when a couple of off-hand jokes about Ruby were made, people immediately called it out as nonconstructive.

Clojure and Scala in particular are languages which make many things possible which simply aren’t possible in other languages. This can lead to a feeling of superiority. Fight that feeling! Clojure is not going to gain mindshare by denigrating Ruby and Java; it is going to gain mindshare by promoting itself and solving problems effectively.

Furthermore, there are many problem domains where the existing tools are entirely appropriate — Rails is a fine framework, and although it has limitations, those limitations don’t manifest in most use cases. Even some of the clojure.core team use Rails for most of their work, and Clojure only for the difficult problems. Denigrating Rails builds walls, when we want to be building bridges.

Build grassroots through the back door

Many recent successes in language proliferation have been achieved simply by providing great tools in those languages. Even if someone doesn’t particularly want Ruby, they might well want Cucumber. If they don’t want Groovy, they might still want Gradle.

On my current project, we’re using node.js for a test stub, even though none of us is a particular JavaScript advocate. Node was just the best tool for the job, so we used it. But that’s caused a lot of us to look at JavaScript in a new way, and I’d say we’re all more likely to use JavaScript again in future as a result.

This is a great way to build up mindshare. I think one tool I’ve learned about at the conj which could fill this role is pulse from Heroku, described in Mark McGranaghan’s “Logs as Data” talk. Pulse is a tool for processing logs not as a stream of bytes but as a stream of events and rich data objects. Another is Cascalog, from Nathan Marz at Twitter, which is a high-level abstraction over Hadoop MapReduce, which creates a very nice internal DSL for modelling MapReduce computations as queries and predicates.

In summary

It’s been a really exciting conference for me, because Clojure is both a great language, and at a key point in its history. It has reached maturity, it is being used by a few people in production to solve interesting and difficult problems; but the next step is to evangelize Clojure, to get it used in earnest by more and more people.

Happy hacking!

Permalink | Leave a comment  »

Sadly I wasn’t able to attend this month’s dojo; but Robert Chatley has done a great writeup.

Thanks also to Mark Needham for stepping in at the last minute as a replacement ThoughtWorks representative at the dojo!

Permalink | Leave a comment  »

I was inspired to learn about monads by Chris Ford recently; his description of encapsulating impurity safely within a pure language had me intrigued immediately. I decided that I wanted to learn about monads in Clojure, a language I am currently diving into.

However, I found learning about monads in Clojure full of fake difficulty (or accidental complexity, if you will). Here I document the issues I found. And the key issue I came across was this:

Learning monads requires reasoning about types

You probably know where I’m going with this. Clojure is dynamically typed. Haskell, the spiritual home of monads, is statically typed. For me, the key to understanding monads was reasoning about types — in particular, drawing a clear distinction between the ordinary type and the type of a monadic expression.

In drawing this distinction, it helped me reason about the behaviour of the monadic functions. By learning that m-bind must return a monadic expression and not a simple value, I learned a key fact about monads; but the number of times I tried to write m-bind expressions beforehand which did not return monadic expressions beforehand was too many.

It’s quite possible to reason about types in a dynamically typed language, but it’s made much harder. If your reasoning is faulty, the program will try to carry on regardless, and in Clojure’s case, give an incredibly cryptic error message. This is not an environment that makes learning easy. If I had been learning in Haskell, my failure to understand the distinction between monadic expression and ordinary value would have immediately been set right by the type checker.

But it’s worse than just making learning hard: Clojure’s dynamic typing has led to a pervasive failure of type reasoning.

A key example of this is that Clojure’s implementation of the maybe monad, maybe-m, breaks the monad laws! It does this because it does not properly distinguish between the monadic expression and the underlying type. The law in question is the first monad law, expressed here as a Midje test:

;;; given a monad which defines m-bind and m-result,
;;;       f, an arbitrary function, and
;;;       val, an arbitrary value
(fact "The first monad law"
    (m-bind (m-result val) f)
    => (f val))

The failure of maybe-m to adhere to this law is demonstrated thus:

;;; failing midje test
(fact "maybe-m should adhere to the first monad law"
    (with-monad maybe-m
        (m-bind (m-result nil) not))
    => (not nil))

The reason that this law is violated is that the maybe-m monadic expression type is no different from the underlying value type. It is therefore possible to find a value such that (m-result val) is nil, the maybe monad’s value for failure.

The Haskell Maybe monad is not so sloppy:

> let myNot x = Just (x == Nothing)
> (return Nothing :: Maybe (Maybe Char)) >>= myNot
Just True
> myNot (Nothing :: Maybe (Maybe Char))
Just True

This is because in Haskell, there is no value foo such that Nothing == return foo; in Clojure, there is such a value: (= nil (m-result nil)).

The repercussions of maybe-m’s violation of the first monad law are relatively minor: it means that when using maybe-m, the value nil has been appropriated and given a new meaning; which means that if you had any other meaning for it, you’re stuffed.

For example, suppose you wanted to implement a distributed hash table retrieval, where failure could be caused by a network outage. You want a function behaviour similar to (get {:a 1} :b), where if the value is not in the table you return nil. If you use maybe-m to perform this calculation, you cannot tell the difference between failing to communicate with the DHT, and successfully determining that the DHT does not contain anything under the key :b; both will result in the value nil. Worse, if you want to use this value later in the computation, the maybe-m will assume a value missing in the DHT to be a failure, and cut your computation short — even if that’s not what you wanted.

Summary

If you want to learn monads, do it in Haskell.

If you must do it in Clojure, the key is to understand and distinguish the various types in play. The monadic type is distinct from the underlying type. m-result takes an underlying value and gives you an equivalent value in the monadic type. m-bind takes a monadic value, and a function from an underlying value to a monadic value.

Permalink | Leave a comment  »

I recently found out that a vegetarian friend of mine had never heard of chestnut roast! Although I am a massive carnivore now, I was in fact a vegetarian for five years, and during this time I discovered that chestnut roast is possibly the best vegetarian dish there is. I mentioned it in passing to my fiancée, who immediately suggested that I could cook it for her. I should learn to keep my damn mouth shut.

ingredients

• 240g chestnuts, in one of those weird vacuum-sealed tins
• 150g cashews
• 4 closed cup mushrooms
• 120g goat’s cheese
• sage
• tarragon
• parsley
• 2 slices granary bread, turned into crumbs
• small amount of veg stock
• 2 onions
• knob of butter

Method

Heat the butter in a frying pan, then chop and fry the onions. After a while, chop and add the mushrooms.

Meanwhile, chop/bash/whizz the cashews. Add the chestnuts and whizz or mash.

When the onions and mushrooms are done, add to the nut mix and stir well. Add the goat’s cheese, veg stock, chopped parsley, sage and tarragon, and breadcrumbs.

Roast in a pre-heated oven at 200 °C for an hour.

Serve with potatoes and veg. We had mash and peas.

Results

Delicious.

I think next time I’d ditch the tarragon, as it tends to overpower the rest of the flavours. Also, I think I have too many things competing with the chestnuts — the mushrooms and breadcrumbs are probably diluting the flavour too much. This is a delicious meal, and one I had for christmas dinner when I was a vegetarian. It takes a fair amount of prep work and a long cooking time, so it’s not going to become a normal after-work evening meal for me, but I’ll keep it in my repertoire for special occasions.

Permalink | Leave a comment  »

The latest clojure dojo was on a subject suggested by Robert Rees: write a program to play 20 Questions, such that the program starts with a small dataset of known celebrities, but each time it fails to guess someone, you teach it a new question. So, for example, if it had guessed Barack Obama but I was thinking of Abraham Lincoln, I could teach it to ask “Are you alive?” to distinguish between these two people later.

I decided to take the idea and extend it by writing a shared server which everyone could use. They would download the initial dataset, play a game locally, then when they were finished, upload their dataset back to the server to share with everyone else.

The code for the server is on my github. It operates via simple HTTP — there is one URL at /20-questions/latest which accepts either a GET request to pull down the data, or a PUT to push it back up.

Once again, I learned a number of lessons — last time, they were about participating in a dojo, this time, they were about organizing a dojo.

Lesson 1: User input is hard.

It’s upsetting that I hadn’t thought about this beforehand, but just like in every other language, user input in Clojure is painful, and difficult to get anything done with in the time constraints of a dojo. Although one or two of the dojo teams managed to wrestle with read-line enough to get it working, the rest of the teams ended up writing internal DSLs (or, less grandiosely, functions) to play the game without having to solve some accidental complexity. In hindsight, I might have thought more deeply about how to deal with user input.

Lesson 2: Error handling feedback is important

A number of the teams had problems uploading to the server. Uploading is always going to be harder than downloading, because you have to generate the data in the correct format, and there’s more ways for that to go wrong than for parsing the downloaded data.

The server had actually been set up to issue useful responses to problems: if it thought a request was not well formatted, or that it didn’t resemble a tree of questions, it would return a 400 (Bad request) code. If, on the other hand, it received a valid question tree, but one which was based on out of date data, it would return a 409 (Conflict) response. An example of the latter situation is where you try to upload a tree, but somebody else adds a new celebrity and beats you to it. In order to prevent people from wiping out each other’s progress, the server would reject the request and issue a 409.

Two problems prevented this from being useful feedback. The first was that I just didn’t explain this at all; I felt that it might be too much detail to go in before people had gotten their teeth into it. But I could have at least mentioned that the error codes were meaningful and that they could have asked me for help once they had reached the point of trying to upload.

The second problem was that clj-http, the library we were using, did not show very much feedback when the error responses were received. The server had, in fact, filled in a full request body, explaining the situation in plain english; but clj-http just threw an exception which said “400” or “409” – not even a “Bad request” or “Conflict”! This made the fact that there was more information available very undiscoverable.

So all in all, as I could see these problems happening, people struggling with user input or seemingly random HTTP response codes, I was getting more and more nervous that nobody would be able to upload anything to the central server — whose data, by the way, was being projected onto a big screen for everyone to see. Thankfully, after a while, one of the teams successfully uploaded, and shortly thereafter, a number of the other teams did too. By the time we were ready for show and tell, we had built up quite a question tree:

{:no
 {:no
  {:no "Elizabeth I",
   :question "Did you find Nemo?",
   :yes "Nemo's Mum"},
  :question "Are you a king?",
  :yes
  {:no "Richard the 3rd",
   :question "Do you have blue suede shoes",
   :yes "Elvis"}},
 :question "Are you alive?",
 :yes
 {:no
  {:no
   {:no
    {:no
     {:no "Chuck Norris",
      :question "Are you in the building?",
      :yes "Elvis"},
     :question "Who's the Daddy?",
     :yes "The Daddy"},
    :question "Are you a panda?",
    :yes
    {:no "Kung fu Panda",
     :question "Are you ling ling",
     :yes "Ling Ling"}},
   :question "Are you a kung fu expert?",
   :yes
   {:no "Bruce Durling",
    :question "Are you (not Bruce Durling)?",
    :yes "Not Bruce Durling"}},
  :question "are you a newsreader?",
  :yes
  {:no "Trevor McDonald",
   :question "Are you the Irish newsreader (the only one)?",
   :yes "Sharon Ni Bheolan"}}}

I think I’ve never seen such a great example of the GIGO principle.

If you want to see some of the client code, team 4 and team 5 uploaded theirs. Also, after the dojo, Uday created a zipper-based client.

The London clojure dojo happens on the last Tuesday of every month. During the dojo, we split into groups of four or five around a single computer, and each person takes a turn at the keyboard. This ensures that even if you have zero clojure experience, you will get the opportunity to write some code at the event.

Entry to the dojo is free, but advance booking is required. Listen for announcements on the London clojurians mailing list.

Permalink | Leave a comment  »

Prologue: I’m going to start blogging every month’s London clojure dojo here. For more details, see epilogue.

August’s dojo, like most of them, was a bit of an experiment. The wonderful Dale Thatcher had prepared a tournament environment for all of our clojure programs to compete in.

The format was this: every ten minutes, we had to upload a working clojure program to a tournament server. The server would then play each team’s program against every other program and determine the winner each time. Points would be scored and added to a running leaderboard.

After settling in with a few rounds of rock, paper, scissors, we started the main feature: noughts and crosses! Your program is given a command line argument of the form

xx0-0----

representing a noughts and crosses grid. You have to output a number from the range 0-8 indicating your next move. One snag is that you are not told whether you are noughts or crosses, and have to determine this fact with the knowledge that noughts go first.

So for example, in the above board layout, you would instantly win if you output 6, because you are noughts, and you would have formed a line top-right-to-bottom-left.

I was on team 4 — you can grab the code and follow on from github.

Lesson 1 — automate common tasks

Since we were to be uploading a new contender every 10 minutes, the first thing we did was write a script called gen-zip which did just that. This script bundled our app into a zip file and uploaded our entry to the tournament server in one step. Hey presto, continuous delivery! Furthermore, this script also automatically committed to git, keeping a history of every single program we uploaded to the tournament server (from the second round onwards, at least).

Lesson 2 — start simple and evolve

Our first entry for noughts and crosses was simply (println 5). It was basically the fastest thing we could write which wasn’t simply guaranteed to lose. Since deployment is cheap, we uploaded it, although it wasn’t captured in the git history. Our second entry (aed257) simply searched for the first free space and went there:

(defn choice [board]
  (inc (.indexOf board "-")))

The simplicity of this is astonishing:

• no need to parse the board string into an actual board object
• no need to determine which player we are

Instead, we just use Java’s native String.indexOf() method to find the first free space and output the index corresponding to that space.

Our next contender (2db1fc1) was only slightly more complex:

(defn choice [board]
  (cond (= \- (nth board 4)) 5
    (= \- (nth board 0)) 0
    (= \- (nth board 6)) 6
    (= \- (nth board 2)) 2
    (= \- (nth board 8)) 8
    :else (inc (.indexOf board "-"))))

Here we use the fact that clojure strings are sequences and can have nth called on them just like anything else. Once more, we don’t need to parse the string into a board; we work only with the flattened string, and we add a bunch of special cases to make a brute-force strategy of getting the middle and then each corner in turn. If all are taken, we revert to our original strategy of the first available space.

Lesson 3a — understand the problem before rushing to a solution

Lesson 3b — get frequent, rapid feedback

You may have noticed some oddities in the above code. What is that inc doing there before .indexOf? Why do we say (cond (= \- (nth board 4)) 5 ...) not (cond (= \- (nth board 4)) 4 ...)? This is because we made the faulty assumption that the grid indexing was 1-based, not 0-based. As a result, we were often going one square to the right of where we actually had intended to go!

We only discovered this because one member of our team was checking each tournament result to see which other teams we lost to and why. He noticed that we were making invalid moves — something we thought we had avoided by always aiming for an empty space.

Our next version (d83b9a6c) fixed this problem. It also started work on a significant new effort — parsing the board and detecting if it was possible to win outright from the current position.

Lesson 4 — doing something simple correctly is better than doing something sophisticated but wrong

By this time our work on instant-win to determine if there was an instant winner was the most significant part of our efforts. The next step was a function to determine which player we were. This was deemed a challenging enough task that we wrote a test for the function. Next we wrote a test for our instant win function, and aimed towards putting together the pieces which could make that function pass. However, all of this was a waste, because we simply didn’t have the time or phenomenal brains required to solve the problem in the half hour we had remaining.

By contrast, the only significant refinements we made to our program in these rounds were based on feedback from tournament results:

• in 8e4e524, grabbing square 2 earlier, to stop us losing to those enemy programs who just filled the grid in order (as our example in aed257 had done).
• in 244c97c, changing from 4-2-0-6-8 to 4-2-6-0-8 to fill the 2-4-6 line one move faster.

Our final version had reams of abortive attempts at sophistication, all commented out using the #_ reader macro. But the real reward had come from those old tactics, rapid feedback and gradual evolution.

Epilogue : the London clojure dojo happens on the last Tuesday of every month. During the dojo, we split into groups of four or five around a single computer, and each person takes a turn at the keyboard. This ensures that even if you have zero clojure experience, you will get the opportunity to write some code at the event.

Entry to the dojo is free, but advance booking is required. Listen for announcements on the London clojurians mailing list.

Permalink | Leave a comment  »

In the spirit of define-measure-improve, here is a list of open source projects I have contributed to, and the largest individual contribution to each project.

• Test::WWW::Mechanize - a one-liner bugfix
• Perl 6 test suite - a one-liner to add a plan to a test (which was causing Smolder not to report test failures on my machine, because it incorrectly handles TAP files with no plan)
• Perl 6 examples - some example code for 99 problems
• Padre, the Perl IDE - some work on extract method refactoring.
• Build monitor for Go - the entire project (save for help from Juan Ibiapina); created during my training at ThoughtWorks

Hopefully maintaining this list will encourage me to improve my open source contributions.

"If you can not measure it, you can not improve it." -- Lord Kelvin

Permalink | Leave a comment  »

HTTP has four standard-usage verbs: POST, GET, PUT, and DELETE. They do not correspond to CRUD (Create, Read, Update, Delete). Forget about the distinction between create and update; it won't help you here. Both POST and PUT can be used for create and update operations in different situations. So what exactly is the difference between PUT and POST?

In a nutshell: use PUT if and only if you know both the URL where the resource will live, and the entirety of the contents of the resource. Otherwise, use POST.

POST is an incredibly general verb. Because it promises neither safety nor idempotence, and it has a relatively loosely-worded description in the RFC, you can use it for pretty much anything. In fact, you could make all of your requests POST requests because POST makes very few promises; it can behave like a GET, a PUT, or a DELETE if it wants to. It also can do some things that no other verb can do - it can create a new resource at a URL different from the URL in the HTTP request; and it can modify part of a resource without changing the whole thing (although the proposed but not widely-accepted PATCH method can do something similar).

PUT is a much more restrictive verb. It takes a complete resource and stores it at the given URL. If there was a resource there previously, it is replaced; if not, a new one is created. These properties support idempotence, which a naive create or update operation might not. I suspect this may be why PUT is defined the way it is; it's an idempotent operation which allows the client to send information to the server.

Very often, POST is used for creation because the server is responsible for assigning URLs to resources. As an example, a forum post is likely to be POSTed because the server must assign it a unique URL. If PUT were used, it would force clients to choose URLs for forum posts, and there would be no arbiter to prevent collisions when two clients chose the same URL.

Very often, PUT is used for update because the resource already has a URL which the client knows about. The client just has to supply a modified version of the resource.

Sometimes, PUT is used for creation. Generally this will be in a level 2 richardson maturity model situation, where the client knows about the structure of URLs and how to create them. For example, if I know a server has a URL scheme where users live at http://example.com/users/username, I could create myself a user account by doing a PUT to http://example.com/users/rhebus, because I already know what my desired username is, and therefore, which URL my user account will live at. [In theory, PUT could also be used for creation at level 3 richardson, where the server tells the client about URLs where resources may be created. If anyone has experienced this situation, I would love to hear about it.]

Sometimes, POST is used for update because only part of the resource is being updated. PUT requires a complete resource; but the client may not know the full contents of the resource or the client may not wish to send the full contents of the resource down the wire. (This is the use-case that PATCH would cover.) For example, a client may wish to append to a log file on the server, without caring about the existing contents of the file.

References:

• RFC 2616 - HTTP 1.1
• RFC 5789 - PATCH method for HTTP
• Martin Fowler, the Richardson Maturity Model

Permalink | Leave a comment  »

So I decided to take the plunge and see what Rakudo* has to offer and what Perl 6 is like as a language. I’ve always found the best way to learn programming is through examples and by doing tasks, so when I found the perl6-examples collection, I started to go through the 99-problems one by one.

Although I started out writing “baby” perl 6, I quickly found that other people’s solutions to the problems were shorter – and clearer – than my own. It’s quickly become clear that the menagerie of perl 6 operators is more expressive than I ever imagined possible. I’ve never complained that a language lacked a particular operator, or that a language should be extended by adding more operators. Perl 6 may be changing my mind.

Today all this was made clear by a silly example in #perl6: noughts and crosses! (or tic-tac-toe, if you prefer.) The problem: given a noughts-and-crosses grid, where the players are 1,-1 and the empty squares 0, determine who, if anyone, has won. Here’s the first example that masak made:

1
2
3
4
5
6
7
8

my @b = [-1, 0, 0], [0, -1, 0], [0, 0, -1];
my @lines = [0,1,2],[3,4,5],[6,7,8],[0,3,6],[1,4,7],[2,5,8],[0,4,8],[2,4,6];
for @lines {
    if ([==] (my @l = slicel(@b, $_))) && all @l {
        say "Someone won: @l[0]"
    }
}
sub slicel(@a, @s) { map { @a[$_ div 3][$_ % 3] }, @s }

He uses a slicel function to create cross-cutting slices of multidimensional arrays, and uses his list of lines to go through all possibilities, finding a winner. He also takes advantage of the [==] reduction metaoperator: [==] results in a == b && b == c. The reduction metaoperator can also create such useful functions as [+] sum and [*] product, which in other languages would require a separate function call. In Perl 5, for example, product is reduce { $a * $b } using reduce from List::Util.

Here’s my improvement:

1
2
3
4
5
6
7

my @lines = (0,3,6 X+ 0,1,2),(0,1,2 X+ 0,3,6),0,4,8,2,4,6;
my @b = -1,0,0,0,-1,0,0,0,-1;
for @lines -> $a,$b,$c {
    if @b[$a] && [==] @b[$a,$b,$c] {
        say "@b[$a] won"
    }
}

By flattening the board array, I avoided the need for a slicel function; I also used the X+ cross metaoperator to shorten creating the lines. Xop <1 2 3> gives a op 1, a op 2, a op 3, b op 1, b op 2, b op 3, c op 1, c op 2, c op 3. Then moritz_ pointed out that 0,1,2 can be written using the upto operator: ^3:

1
2
3
4
5
6
7

my @b = -1,0,0,0,-1,0,0,0,-1;
my @lines = (0,3,6 X+ ^3), (^3 X+ 0,3,6), 0,4,8,2,4,6;
for @lines -> $a, $b, $c {
    if @b[$a] && [==] @b[$a,$b,$c] {
         say "@b[$a] won"
    }
}

Further refinements are possible: 0,3,6 is ^3 »*» 3, using the hyper operator to multiply each of 0,1,2 by 3. But by this point, we’re just using different Perl 6 operators for the sake of it, not because it makes the code clearer. Like code golf, this is hackers at play; and while the product is not something necessarily useful, the learning I’ve gained from it is invaluable.

Permalink | Leave a comment  »

If there is one term I would like to erase from computing terminology forever, it is ‘lvalue’. It is a confusing and distracting term, and far from making a discussion more precise, it makes it less so.

The problem is that there are multiple competing definitions for lvalue.

1. A storage location: The C Standard definition

A C lvalue is an expression which refers to a location which can store a value.

This is the definition I am most familiar with. Given the C code:

int a[30];
int i=3,j=5,k=10;
a[i] = k;

the following are all lvalues:

a
i
j
k
a[i]
a[j]
a[k]

because every one of these expressions refers to a memory location which holds a value. The standard suggests in a footnote that it is helpful to think of “lvalue” as short for locator value.

2. Something which can be assigned to: The C Standard “modifiable lvalue”

In C, a modifiable lvalue is an expression which may appear on the left-hand side of an assignment.

Note that this is not contingent on there being such an assignment; only that assignment is possible. As a result, given the same C example code, the following are modifiable lvalues:

i
j
k
a[i]
a[j]
a[k]

Notably missing is a. a is an lvalue but not a modifiable lvalue, because it refers to a location in memory but can not appear as the left side of an assignment. In C, you cannot assign to a whole array, only to one cell at a time.

In contexts outside of C, this is often simply referred to as an lvalue, but when talking about C you should stick to Standard terminology to be as precise as possible. Especially if you’re asking questions on comp.lang.c.

3. Something on the left of an assignment: the “left-hand side” definition

This definition is of a totally different character to the C Standard definitions. In the perl code:

my $array = [];
my $i = 3;
my $j = 5;
$array->[$i] = $j++;

this definition states that the only lvalue is $array->[$i] because it’s on the left-hand side. $array->[$i] is an lvalue only in the context of this assignment; in another assignment (say, $foo = $bar) $array->[$i] is no longer an lvalue, $foo is instead.

There is an obvious problem in this example that $array->[$i] is not the only expression which gets written to: $j is also written to by the autoincrement operator.

I believe when people use this definition, they really mean definitions 4 or 2 but are using sloppy language. Nevertheless it is a definition which people use.

4. Something which is written to

This is similar to the above argument, but states more precisely that an lvalue is something which is written to in an expression. In the above example, there are two lvalues: $array->[$i] and $j, both of which are written to as a result of the code. I like it more than definition 3 for this reason, but you can start to see how thinking of ‘lvalue’ and ‘rvalue’ as analogous to ‘left’ and ‘right’ breaks down.

Mutual incompatibility

These four definitions are mutually incompatible. The biggest problem is that definitions #1 and #2 are absolute, while #3 and #4 are contingent on a particular assignment. Under #3 and #4, a variable can be an lvalue in one line of code, but on the next line it is not an lvalue because it is not written to. Under #1, a variable is always an lvalue because it always refers to a location in memory. Under #2, a variable is always an lvalue unless it is of some type unsuitable for assignment — a constant, for example.

In other words, definition #2 says an lvalue can appear on the left-hand side of an assignment, while definition #3 says an lvalue does appear on the left-hand side of an assignment.

Definitions #1 and #2 are incompatible in programming languages which have storage locations which cannot be assigned to. Such a location is a #1-lvalue but not a #2-lvalue. The chief problem requiring separate definitions in C is the existence of array and structure types — these cannot be directly assigned to. In modern programming languages, arrays and structures can typically be assigned, so this problem is moot.

The other main type of location which cannot be assigned to is a variable with constant type. There are many languages in which you can declare variables constant. A reasonable modification can be made to definition 2 to say “an lvalue is an expression which could be on the left-hand side of an assignment, ignoring constant qualifiers”; but then there exist lvalues which cannot be assigned to. Nevertheless outside of the context of C there are situations in which it is reasonable to conflate #1 and #2.

A survey of definitions

I did a search of various programming resources I know for definitions of lvalue. Here is what I came up with:

• Strachey’s seminal paper in which he coins the term L-value: “An L-value represents an area of the store of the computer.” pretty explicitly #1
• Wikipedia: “A value (computer science) that has an address” #1 as of time of writing
• Wiktionary: “A value that can be treated as an address or storage location.” #1 as of time of writing.
• Computer Programming and Precise Terminology, a Dr Dobbs article: #1
• C Standard (read a free draft here): #1 (and #2 for modifiable lvalue).
• Foldoc: “A reference to a location, an expression which can appear as the destination of an assignment operator indicating where a value should be stored.” #1 and #2 in the same sentence.
• about.com: #3 (top google hit for lvalue definition)
• c2.com: Direct quoting of Strachey; #1
• msdn: “An lvalue refers to an object that persists beyond a single expression. You can think of an lvalue as an object that has a name.” #1
• perldoc perlglossary: #2
• The Ruby Programming Language: page 92, #2
• The Camel Book, 3rd edition: page 52, #2

I must be honest here and say I fully expected more instances of definitions #3 and #4 than I found. I have come across people using #3 and #4 on mailing lists and in fora enough for the term to be a source of confusion — in fact, just today I saw somebody using definition 4. The mnemonic value of ‘lvalue’ and ‘rvalue’ versus ‘left’ and ‘right’ encourages people to think of lvalues as something to do with assignment, rather than as something to do with persistent storage locations.

In any case, in my experience there are enough people confusing the term ‘lvalue’ that I feel that it should not be used at all. I would instead recommend the term ‘location’. If you must use the term lvalue, please use definition #1.

On rvalues

I haven’t mentioned the term rvalue at all here. It doesn’t get used nearly as much, for which I am thankful; I believe the reason it isn’t used is because the term value does the same job much better, or where more precision is required, value of an expression.

Permalink | Leave a comment  »

I tried to find out about you online, I tried to read your leaflets to
understand who *you* are, I tried to find out what you would do with
my council. But none of you seem to be trying to tell me. The only
person who has any real Google visibility is Kirsty McNeill, who is
moving on to better things as Gordon Brown's speechwriter.

Your leaflets say nothing about local elections, only about the
Bermondsey and Old Southwark general election.

As a result, only two candidates stand out for me. Martin Seaton, who
helped chair a meeting with developers when our local park was
threatened with being developed into flats, came across as a very good
facilitator and though he was a neutral chair at that meeting, he came
across as genuine and concerned. And Ben Johnson, who is the only
candidate I have spoken to during this entire process, gave me a call
and put up with my complaints about dishonest Lib Dem leaflets. As a
young and enthusiastic candidate, I feel he's a much better bet than
old and apathetic alternatives.

The rest of you must try harder if you want to get my vote.

Permalink | Leave a comment  »

Some very common phrases you hear in political debate is "so-and-so is
very right wing" or "he's a socialist" or "she's all about big
government". You also get refinements to the left/right divide, from
"far left" (communist party), "left" (old labour), "left of centre"
(lib dems), "right of centre" (new labour), "right" (conservatives),
"far right" (bnp). But what does it all mean? Are they useful labels,
and do they form a coherent continuous line from far left to far
right?

Here are some policies I associate with "left": big government, higher
taxes, equality, government provided services (education, health,
social security, pensions), regulated markets, redistribution of
wealth, an "equal society", relaxed attitude to immigration, relaxed
attitude to the EU,

And some I associate with "right": small government, lower taxes,
individual freedom, privately chosen and paid-for services (education,
health, social security, pensions), free markets, wealth as an
incentive to betterment, the "American Dream", restrictions on
immigration and asylum claims, euroskepticism,

When we use terms such as "left" or "right" we assume several things:
that somehow, these baskets of policies are associated with one
another; that left and right policies are diametrically opposed; and
that these policies are the only ones which matter. I don't believe
any of these to be the case.

First, are these policies associated with each other? Well, I would
say tight immigration controls are diametrically opposed to free
markets. If Polish plumbers are prepared to work for less money than
their British counterparts, in a free market I should be able to
choose for myself which one I want to fix my bathroom. But immigration
controls remove that choice and mandate that I pay a British plumber,
by preventing the Polish plumber from entering the country. "British
jobs for British workers" also inevitably means "less choice for
British consumers".

Are left and right policies diametrically opposed? If we look at free
markets versus regulated markets, while it initially sounds like they
are totally opposite, it is not as simple as that. A market cannot
exist without some rules: for example, without rules, why would I
trade with you when I can just steal from you? We need property laws
and regulations to punish those who break the rules. I want to employ
someone, but I want to know they are competent at mathematics -- we
need a system of regulations around qualifications to give me the
freedom to choose the candidate I want. Some types of market freedom
require regulation.

And are these the only policies that matter? I haven't mentioned the
environment, Scottish independence, ID cards, policing, housing or
defense (including nuclear weapons), because I have no idea what the
generic "left" or "right" think about them. But they are important
policies.

In the end, it's far better to choose the policies you care about the most and find the politicians who support those policies. TheyWorkForYou doesn't try to work out whether each MP is "left" or "right", but instead tells you how each MP voted on a range of issues. See for example Malcolm Bruce (LD, Gordon), Ed Miliband (Lab, Doncaster North), Eric Pickles (Con, Brentwood & Ongar). Once you've seen that, you might not be able to divide these three into simple positions on the left-right spectrum, but you cannot come to the oft-heard conclusion that "they're all the same". No, they're all different. With 650 MPs, you have 650 different opinions on the way the country should be run, and it would be optimistic to hope they all fit onto one line from "left" to "right".

Permalink | Leave a comment  »

I don't like banker's bonuses. They have two problems: 1) Someone is
getting paid lots of money to do something which can't be that hard,
and 2) Someone is incentivised to take more risks than is good for the
bank, since greater risk gives greater potential reward and greater
potential bonus. The solution to 1) is to get over it; it's the way
the world works, and bankers are hardly unique in this position
(footballers, opera singers, chief executives also spring to mind).
The solution to 2) is to have a system where the banks aren't
incentivising excessive risk-taking.

Consider that I am a banker, with a bonus scheme where I am rewarded
more for earning more for the bank. Consider that in a given year, I
have a 1/4 chance of losing £1m and a 3/4 chance of making £1m for the
bank. If I am lucky and make the £1m, I get a bonus of £X; if I make a
loss for the bank, I get nothing.

But suppose I also have the option to gear up my trading strategy by a
factor of 10. Then I have a 1/4 chance of a £10m loss and a 3/4 chance
of a £10m gain. If I have made £10m for the bank, I should be getting
quite a bit more than £X bonus, while if I make a £10m loss I can't
get a smaller bonus than zero, so I'm quids in either way. But the
bank is now running the very real risk of losing £10m on my trades. It
might have considered the £1m risk acceptable, but the £10m risk as
unacceptable, but the bonus structure does not reflect this fact at
all.

This is what is meant when it is said that bonuses encourage excessive
risk-taking. The problem is exacerbated by the fact that there is so
much randomness involved -- if I might make a huge loss due to a
market crash beyond my control, I'm not going to be terribly motivated
to work hard on a marginally better strategy unless the potential
bonus is pretty huge [2] -- and in many cases, bankers' bonuses are as
big as or bigger than their salaries.

What is the solution? Well, if you curtail banker's bonuses, the banks
will maintain remuneration packages by pushing salaries up instead.
And without bonuses, bankers have no incentive to take excessive risks
-- but they also have no great incentive to make money for the bank
either. If you made £10m for your employer, wouldn't you want a slice
of that pie?

The lib dems' proposals just seem to be a grand "up yours" to the
bankers without actually trying to understand and solve the problems
represented here.

[1]http://www.bbc.co.uk/blogs/thereporters/robertpeston/2010/04/lib_dems_smaller...
[2]http://www.forbes.com/2006/05/20/executive-compensation-tournament_cx_th_06wo...

Permalink | Leave a comment  »