This weekend I had a project to write.  I have a huge archive of MP3s left over from the mid-90s through mid-00′s, and they’re a bit of a mess and moving them onto my iPod is always a pain in the neck because they might be inconsistently labeled or have proto-Unicode issues or whatever.  So I decided to write a little command line utility that would help me sanify my files.  Since I’m old-school and organize my songs by album and genre, my tool  would pick the most frequent album and genre names from the ID3 files, while taking in alternative sources for those if they weren’t complete, such as the name of the directory (which also would give artist, if not otherwise available).  It would take the song title from the filename if it weren’t available in ID3.

And I decided to write it in Hy.  Hy is a Lisp parser that produces Python AST, and since the Python VM accepts and executes raw AST, it’s possible to write performant Python using a completely Lisp-like syntax.  Having not written Lisp in many years, this sounded like fun.

You can find the entire source code at my elfsternberg/mp_suggest repository on Github.

I like the feel of this code.  It’s a series of let statements that gets the MP3s and then creates a new list of details (genre, album, artist, title, position), then searches through the command line options and details for genre and album names.  Each let statement creates a new object; it looks mostly like immutable code, although that fantasy is blown by my frequent use of ap-reduce.  Some of the objects created are anonymous functions containing closures, so they’re new functions with pre-created knowledge of things they care about, such as the legal commands and whether or not to override the ID3 title with a modified copy of the filename.

Overall, it doesn’t feel much like Python.  It doesn’t feel like Javascript, either.  It feels most like RistrettoReginald Braithwaite’s version of Coffeescript, chock full of functional declarations rather than procedures.  The threading macros and anaphoric macros are huge time-savers, and the organizational principles of Lisp’s parenthetical syntax gave me plenty of vertical room to work with, a luxury I don’t usually have with Python’s whitespace-delimited syntax.

I’m not sure I’ll ever write in Hy again.  I’m not sure I have a need to; I know I’d never be able to talk my employer into using Hy; they don’t even want to discuss preprocessors for web apps like Haml, Less, or Coffeescript.  I’d love to have something like this that I really enjoyed using for Javascript, but I haven’t found anything beyond Coffeescript that really has lispy expressiveness and pythonic power.  Still, if I were doing more Python at home, Hy is really the language I’d be doing it in.  Hy for Python, Clojure for Java, and Coffeescript for Javascript.  Because Clojurescript is still not good enough.


Facebook as a psychic shopping mall

Posted by Elf Sternberg as Uncategorized

I was hanging out with a friend last night and she said, “I think Facebook is designed to distract you. You go in looking for one thing, but oh, hey, there’s this other thing, and what’s Aunt Marsha up to? Eventually you find yourself spending hours on it.”

What she’s describing is the Gruen Transfer, “the moment someone experiences after entering a shopping mall when, surrounded by an intentionally confusing layout, lose track of their original intentions.” You enter saying “I came to buy a coat” and leave “Ooh, shoes! Cameras! Necklaces! Smoothies!” that much poorer.  Modern mall architects deliberately go for this effect; their objective is to keep you inside the mall as long as possible, in order to increase retailer’s opportunities to sell to you.

Facebook appears to have hit a Gruen Transfer state completely by A/B testing, which if you think about it, is exactly what A/B testing of ad-driven sites is intended to do: keep your eyeballs in front of the advertiser’s windows for as long as possible. Facebook has just had more time, more money, and more talent to throw at the issue than anyone else.  The objective remains the same: to distract you from your objectives, to encourage an increase in sales.  Your pleasures and interests don’t enter into it.

This may be why Twitter is “experimenting” with their newsfeed.  A/B testing shows them that they could keep your eyeballs on their interests for much longer; the only question remains how corrupt they want to deviate from their original mission.

Programmer Zef Hemel recently made a confession, aptly titled I Hate Puzzles. In it, he confesses to a deep loathing for the recruitment come-ons that read “Do you love puzzles?” and even more for recruitment ads that are puzzles. He says, eventually, that he and his wife agreed they don’t understand why people do puzzles. “Sure, if you don’t have anything else to do. We had plenty more useful stuff to do.”

And while I enjoy video games, and especially enjoyed the puzzles within the Portal series, in general I avoid the kind of puzzles found on the average bookshelf or in the average game store window display. Not just jigsaw puzzles, but sudoku, crosswords, the whole gamut of paper-and-pencil puzzling that seems to take up some people’s entire commutes.

Zef then says that he doesn’t care much about algorithms. Algorithms aren’t at the heart of programming: design is. Coming up with a syntax, a domain specific language, an API, that reduces the friction between what another developer wants to acheive and the computer’s capability to achieve, are at the heart of almost all that he does.

And he’s absolutely right.

If I want a word puzzle that’ll increase my vocabulary, I’ll write a fucking novel with my thesaurus and my mind open. I’ll delight in solving the puzzle (yes!) of making sure the characters are three-dimensional, the plot is consistent, the foreshadowing is appropriate, the guns on the mantlepieces have all been fired or elided or revealed to be red herrings. I will hope to make someone else’s day sexier, happier, thoughtful, and delightful.

If I want a brain-teaser, I’ll spend my days trying to understand monads, write tools that let users write tests to a database more efficiently, or (my current unreleased project) write my own goddamn programming language. I will solve not just a puzzle, but a problem. I will hope to make someone else’s day easier, more interesting, and more effective.

I won’t write experimental novels, messing with the structure to produce quasi-lyrical novel-like works, although I admire writers who try that. I don’t have to create new algorithms for my programming language; transducers, scope, homoiconicity, Hindley-Milner, pattern-based dispatch and automatic currying are solved problems; I just want to put those puzzling pieces together in my own idiosyncratic way and hope to learn something about how programming languages work as I do, to make myself a better programmer and to give myself tools for the project after that.

I am a novelist who doesn’t do crosswords. I am a programmer who doesn’t enjoy sudoku. And I don’t do those things because I have much more interesting things to do with my grey matter.

New things.

Last night, I went to a coding meet-up where a few of the others were playing a game.  There’s a website called Exercism.IO, where you can download code exercises and submit your answers to a global community of like-minded players.  The others had been at it for a few minutes; I had to take time to install the exercises and begin.  The basic problem, coded in JavaScript, was “Here are seventeen sentences, each marked as being one of four kinds of conversation: statement, question, argument, or silence.  Write a function that successfully matches the expectations of the test case.”

While the others struggled with both regular expressions and multiple if / then / else if ... trees, I went straight for a different solution:

    this.hey = function(input) {
        var tests = [[function(i) { return /^\s*$/.test(i); },                                      "Fine. Be that way!"], 
                     [function(i) { return /[A-Z][A-Z]+.*!$/.test(i) || /^[A-Z\s]+\??$/.test(i); }, "Woah, chill out!"],
                     [function(i) { return /\?$/.test(i);},                                         "Sure."]];
        for(var i = 0, l = tests.length; i < l; i++) {
            if (tests[i][0](input)) return tests[i][1]; };
        return "Whatever."; };

Seven lines. (Yes, my Javascript coding conventions look like Python and Coffee. Whitespace is one of my secret weapons.) Where everyone else had 20-plus lines of comparisons, I went with the brute-force solution. It took four iterations to shake out all the bugs, all 17 tests passed, and it was done.

There are plenty of places where this is silly code; it's basically arbitrary string reduction, and is completely dependent upon the order of the tests, but as the last result says, "Whatever." To me, however, this reduces a lot of if / then statements down to a single if expression in a hand-written map/reduce: map all possible regexes with a single string, returning [String | Nothing], then reduce down to a single memo of the String. But really, this comes down to a basic rule: wherever you have a list of things to do, see if your language has a proper List with which to contain them.


Airplane Mode from the Ubuntu Terminal

Posted by Elf Sternberg as Uncategorized

I ride a bus to and from the office every day. While I love Ubuntu, getting a constant barrage of “WiFi Available” popups whenever we pass by a coffee shop is annoying as hell. Digging into the network manager every day is a chore. Solution: A simple command line handler with a memorable name:

if [[ -n "$1" ]]; then
nmcli nm wifi $MODE

Typing airplane will turn off your wi-fi stack; typing airplane off will turn it back on. (Actually, typing airplane four score and seven years ago will turn it back on. Any argument is accepted as turning airplane mode off. Even airplane on.)


git wordcount

Posted by Elf Sternberg as Uncategorized

This is an early draft, but I figured you might enjoy taking a look at my latest little toy:



if [ "${1:0:2}" == '-h' ]; then

REM=`git diff -U $ARG | grep '^-' | sed 's/^-//' | wc -w`
ADD=`git diff -U $ARG | grep '^+' | sed 's/^+//' | wc -w`
DIF=`expr $ADD - $REM`
echo "Word count: $DIF"

As I’m sure I’ve mentioned before, I write award-winning science fiction as well as hack code, and I use emacs for both activities. I’ve always written primarily in what I think of as Usenet mode, with asterisks and underscores for emphasis and bolding, respectively, a mode that has become codified as Markdown. I used to have my own homegrown toolkit for that, but these days I just use the Python version of Markdown instead. I’ve always used a VCS for that, living entirely in GIT almost since the beginning.

I wanted a simple script to tell me my daily progress. This is it. It compares what’s on the filesystem right now to the latest check-in, or, if you pass it the -h option, it will compare the last check-in to the previous check-in. Each additional -h will push the oldest comparison back one commit, so -hhh will compare the last check-in to the one three check-ins ago.

After writing this, I learned about git-sh-setup and --word-diff=porcelain, both of which will find their way into a future version; hopefully soon you’ll be able to say ‘git wc HEAD^^^..HEAD^^’ or similar, and see how productive you were any two days in the past.


Now I get mapping and currying…

Posted by Elf Sternberg as Coffeescript, javascript

Okay, so I’m slow to the party.  Forgive me.  I was reading Reginald Braithewaite’s Javascript Allonge and I understood where he was heading long before he got there, and the lightbulbs were going off in my head, and although his technique for getting there was far more verbose than my own, I finally grokked the big point. It’s a classic programmer’s rule of productivity: If you find yourself doing something more than twice, automate it!

We do this all the time with makefiles and functions, but Braithwaite’s book brought the question down to the micro level: If you have to apply the same transformation to three different arrays, or if you have to apply three transformations to the same array, why are you doing it like this:

somenumbers = [....]
squared = (x) -> x * x
cubed = (x) -> x * x * X
halved = (x) -> x / 2

When you could do this?

toMap = (a) -> (f) ->
[squared, cubed, halved] = [
    ((x) -> x * x), 
    ((x) -> x * x * x), 
    ((x) -> x / 2)].map(toMap(somenumbers))

By creating a function that already has the array to process, and then processing it multiple times, you prevent the emergence of typos in your code caused by repeatedly typing the variable name. And you accurately put the point of your code in the left-hand column once, which is very attractive to me.

This is good stuff, man. And Coffeescript is so much more expressive than Javascript in the same realm. Imagine writing toMap in JS:

function toMap(a) {
    return function toMap(f) {

A lot of clutter to express what Coffeescript expresses so cleanly.

I added the following two items to my .emacs file, and they have changed my life:

(defun night-mode ()
  (menu-bar-mode -1)
  (scroll-bar-mode -1))

(defun daylight-mode ()
  (menu-bar-mode 1)
  (scroll-bar-mode 1))

Both of these rely on the emacs “color themes” handler, but really, when working late at night when my eyes are tired, green on black seems to have a nice bit of contrast, whereas when working with a lot of light, the more traditional black-on-light-grey works well for me.  By banishing the menu bars and scroll bar, I eliminate further sources of distracting contrast.

Which makes working with Emacs a heck of a lot more pleasant.  I used to issue these commands by hand, but macro-izing them makes switching modes a much happier and more productive experience.



Be smart. Don’t be clever.

Posted by Elf Sternberg as Uncategorized

At the office, I led a major initiative to create a lovely drag-and-drop manager for a toolkit of pre-packaged views of.  The views had their own complexities; Splunk search views come with the own controls, and the controls are populated by underlying searches, which may in turn be controlled by other controls.  I fumbled around with various design decisions before realizing the obvious.  What I just described was a tree.  I threw out all my fancy analyzers and weird Backbonification stuff, and wrote a tree handler.

It worked perfectly.

Later, I realized that each view dropped onto the screen could be described using the RDFa trio of typeof/about/property, three rarely used HTML attributes that allow any HTML object to have a rough correspondence a database notion of table/row/column, or the object-oriented notion of class/object/field. Instead of doing all my previous, ridiculously hairy code managing the visual layout, I used the HTML5 drag/drop for positioning, some very small Javascript for grid management, and classic event handling to figure out when new views had been dropped in, moved, or deleted, and I let the existing libraries do all the heavy lifting.  Using data attributes to describe the visual state and CURIE for de-obfuscation, I had a nice event-driven system where everything on the page represents every and only those things I cared about.

The lessons learned were fairly straightforward, and classical.  The data structures you learned in university are usually Good Enough; don’t borrow trouble trying to create a hack for performance reasons.  Premature optimization is always the root of evil.  Second, always survey the tools available to you before starting to code.  You’ll be surprise just how little work you actually have to do.  Clever is writing a lot of code to do a lot of stuff; smart is figuring out that you don’t need a lot of code to do a lot of stuff.

Another lesson learned is that purity matters.  In web application design, view every event as a miniature program– it does two things, it transforms your existing data to create a new structure, and then it illustrates that new structure on the page.  Keep those two things as separate as possible, even if you have to go around your framework to do it, and you’ll have a much more testable system when you’re done.


The Functional Programming Crock

Posted by Elf Sternberg as Uncategorized

Quick, what’s the difference between these two expressions?

// Javascript

var add21a = function(a) { return a + 21; };

var add21b = (function(b) {
    return function(a) { return b + a };

// Coffeescript

add21a = (a) -> a + 21
add21b = ((b) -> (a) -> b + a)(21)

When I first encountered functional programming, I internalized the definition of a pure function: A pure function is one that always returns the same result given the same arguments.  You should always write your code in a transformative way: Given a set of inputs, write the toolchain to unambiguously create a set of outputs and, only then, perform the side-effect of updating your presentation to reflect your result.

It wasn’t until I was knee-deep into a program with a lot of functions-writing-functions that I stopped to ask myself the question posed above.  I was peeling off dozens of functions that, themselves, were created with closures that retained a lot of state, were themselves mapped to various inputs into the system, and were themselves responsible for producing the output.

The magic is that each individual function was pure: it always gave the same result given the same arguments.  But each function itself was created by a pure function that, given the same arguments, produced the same pure function.  So, was the entire result pure?

The answer, apparently, is yes.  The answer to my question above is, there is no difference.

On the one paw, I understand the point being made here: I can reason about my code much more rationally.  The products of each step are smaller, more reasonable, and generally easier to comprehend, so both ensuring that the code is correct, and debugging it, are facilitated by the functional approach.  On the other paw, this approach allows one to create a vast array of functions rather than a large object with a single function and a vast array of data, with the possibility of code sprawl now simply existing in a different form.

I like functional programming.  It fits my brain, as the Python folks say.  But I don’t think we should at all kid ourselves about the complexity of writing software, or the possibility of anti-patterns emerging out of functional programming.  I’d be happy to say that there are fewer anti-patterns in languages like Haskell or Coffeescript, but the ones left behind are much harder to manage, avoid, and debug.


Recent Comments