Software development needs copy editors

Posted by Elf Sternberg as programming

Magazine, newspaper, and book publishing, in both fiction and non-fiction, all have two specialized roles that the work goes through before it reaches it’s final form:

The Project Editor and the Copy Editor.

Software teams have a project editor, although usually he’s called the project manager. In literature, the project editor doesn’t usually edit; his responsibility, like that of the project manager, is to see the work go from manuscript to bound volume, and is responsible for schedule, budget and the interaction of other departments such sales and marketing, in order to assure the success of the work.

There is no job in software anything like the copy editor. And that’s a problem.

Linters and style checkers can make sure that you have the right number of indentations, ensure that your line length fits into the house editor, check you have comments where comments are required. What they can’t do is ensure that your variable names have meaning, that your classes are SOLID, that each function does one thing and does it well, and that a future maintainer can look at any given piece of code and not say “WTF?”

Every creative profession that works on a commercial production line, that is treated as a craft, has an editorial pass. If you’ve ever worked in advertising, there’s one guy whose job is to make sure the studio musician is creating background music that matches the taste of the ad; there’s also someone who makes sure that every background image in an animation matches the overall feel of the production. These aren’t “I know it when I see it” jobs, either; these are people who are expert studio musicians and illustrators who have earned the trust of their peers and graduated to the role of guiding others to excellence.

Every creative profession except software. Software developers are a weird bunch: the good ones are absolutely ready to take criticism from their peers, as they want to get better. The bad ones believe that “it passes all the tests, so you shouldn’t care about its style” and you can’t tell them anything at all. Given just how much money is sloshing around this industry, any change to impose more discipline is opposed with ferocity. We all have someplace else to go.

Software copy editing would be a line job; it wouldn’t be management, and yet it would have the final, human-eye check on whether or not something goes into the repository. It wouldn’t be about if the code is testable; it would be about if the code is maintainable. Can a future code monkey at a glance see what you meant and how you acheived it? As a line job, it would be dependent upon the productivity of others to succeed, yet in agile the editor would have no velocity himself.

There are tons of anti-patterns that are opposed to this idea:

  • “I don’t want to pay some guy just to sit around and read code.”
  • “Testing will find all the bugs.”
  • “The next revision will be a complete re-write, so code readability isn’t an issue.”

I’m sure you could name more.

But if you really give a damn about code quality, and your company has more than twenty developers, consider hiring a software copy editor, a hands-on, polyglot programmer with an opinion about how software should be written, who has the chops necessary to gatekeep your code quality and mentor your junior developers.  You’ll save more than you spend: a good copy editor will reduce your need for testing cycles; the whole point of the job is to ensure no code escapes your office that isn’t clear, sane, readable, testable, and maintainable, and to teach those fresh-out-of-college faces how code in the real world looks.


SMART goals and Petered principles

Posted by Elf Sternberg as Uncategorized

List three to five goals for the next year. These goals must be SMART (Specific, Measurable, Attainable, Relevant, Time-Oriented). Include both those goals that would help you in your current role as well as those that prepare you for future roles in the organization.

Few work-related things fill me with greater dread than this annual question. And I’ve gotten it at every job I’ve ever worked at where there were more than a hundred people.

I’m a programmer. In every position I’ve ever worked, I was hired on the basis of a skillset I had at the time. Spry hired me because I knew Perl when Perl was barely a year old; F5 because I knew Perl and Curses; Isilon because I knew both kernel development and web application development at a time when both were truly esoteric; Canvas because I knew both Django and assembly language; IndieFlix because I knew Django; Spiral because I knew Single Page Development when that was brand new; CK12 because I knew web development and the EPUB 2.0 standard.

While I was there, other problems became apparent to me: Spry needed me to learn both C++ and Python; F5 needed me to learn Lex & Yacc; Isilon needed me to learn server development; IndieFlix needed me to take my hobbyist-level transcoding skills and go professional; CK12 needed me to learn WebGL. None of those needs were apparent until they were immanent. It’s not as if I could scan the horizon of issues and see, “Months from now we’re going to discover that this rendering technique won’t work on an iPad without WebGL” or “Months from now, we will discover that the application server needs an ‘always on’ userspace component for consistent kernel communication; I’m going to need to know how to write one of those.”

Questions like the one above imply a kind of restlessness that can only be cured one way: with money. The idea is not that you want to be great at your job; the idea is that you want to be great at the job that leads toward ascension, and in most corporations the only ladders available lead away from a lifetime of training and experience toward “leadership.” I have deep respect for good leadership, and I hear the siren song that says, “You have all these skills; your duty is to help others achieve the skill level you have.”

Given the level of interpersonal skills I have, I ultimately end up wondering: “If they achieve the same skill level I have, does that mean you’ll Peter them too?”

As a software developer, I love learning new things. I’ve been studying a couple of things at random: Category theory, type systems, interpreter cores; basically, a lot of meta around the lambda calculus. Along the way I learned a little Haskell, and learning even a little Haskell made me a much better programmer. The odd thing is that few of those interests map to those of my employer; usually, they don’t want esoterica, they want better in the realm in which they work. It has been my fortune to have “significant advanced skills” in customer-facing engineering; it’s my misfortune to realize that I’ll be working in the same four damn languages for the rest of my life if I want to keep making the same salary.

I have only one goal: Get better at what I do. Everything else is commentary.


A-hah! learning and Grind learning

Posted by Elf Sternberg as Uncategorized

I have two modes of learning, and one I enjoy, and one I resent mildly.


“A-hah!” learning happens from time to time while I’m working through a larger problem in my head. One might call it the holistic learning moment; I’m reading a textbook or paper and suddenly I get it. I not only understand what the author is trying to say, but I get the ramifications of it. This happens a lot when I’m reading about type systems and it clicks for me that a type system is an additional language, with it’s own strengths and weaknesses, that helps the developer say what the program means, and more importantly, lets the programmer talk about the genericity and applicability of his code. These come in a flash, as if several different bundles of neurons that had been slowly growing toward each other suddenly connected and something suddenly makes a lot of sense.

The Grind

“Grind” learning happens when I have to learn something that should have been an “a-hah” event but wasn’t, and I have to repeat the lesson over and over until I internalize and understand it. A recent example is in my Language in 20 Minutes exercise. the hardest thing for me to get was the difference between dynamic and lexical scoping from within the interpreter. I had to write, rewrite, and rewrite again the way a functions’s free variables are informed by its lexical scope (the closure), and the way its bound variables are populated by values found in the current execution scope.

In some ways, the second problem was informed by the idea that I didn’t even have the notion of “bound” and “free” variables in my mind until I started reading up on the lambda calculus and trying to grok what it means and how it works. But even though I do have those terms now, it’s been difficult for me to internalize the ways of lexical and dynamic scoping.

It doesn’t seem to be a matter of problem size. The LangIn20 project is very elegant and doesn’t have many moving parts. SparQL is a large project that has a lot of moving parts; both were grinds to master.

I wonder what, in particular, contributes to a problem being an “A-hah!” versus a grind, and if I can turn more of the latter into the former.

This may be an idiosyncrasy, but I doubt it, because I see too many examples of it in open-source code all the time: I have this strong suspicion that software developers think too holistically. There’s even a rather trenchant and famous cartoon illustrating how a developer loses context with a simple interruption and has to rebuild the scaffolding of his thoughts before he can fix the issue.

I had this problem recently. I’ve been noodling with a small lisp interpreter that I was building as an exercise, and one of my steps was to write a lisp-like “cons” list library, recodify each node from a record-in-a-vector to lists-in-lists, and then build upward from there. My target language was Coffeescript. There are three major components to the interpreter: an eval() function, the environment, and the syntax.

Special forms are injected into the environment at initialization time, and obviously both the “define function” and “if” statements need to be able to call eval() to evaluate their arguments, since this is an interpreter that works its way to the bottom of the AST and then evaluates upward to a final result.

As I was making the changes, I got terribly lost making the special forms respond to cons lists rather than vectors or dictionaries. The structure of a node from the reader became (type value blame), where a value would be a cons list of nodes when type became “list”. The distinction between the two types became muddled in my head, and I kept worrying about what it would mean to switch to the other phase of processing.

And then I realized: I should stop worrying. Everything I needed really was right before my eyes. Knowing the code in front of me, I could easily reason about everything I was doing right then, right there. This data structure would get handed off eventually, but I shouldn’t care. There shouldn’t be context above and beyond what’s on the screen.

I think this is why I’m really starting to like F# and Haskell and that family of languages so much. (And Lisp, really); I shouldn’t have to think that hard, I shouldn’t be spending every day at the utter limit of my human understanding just to get my job done. I know, that’s the holy grail of development, yet I’m convinced that it’s not only a reachable goal, it’s one we perversely avoid. We’re proud to plumb code through multiple layers when we ought to be ashamed of it.

I think we ought to consider the baseball bat as a unit of code quality: if the code is so hard to understand the next guy will want to come after you with a baseball bat, maybe you should consider re-writing it.

My first impression was , “What a fucking bunch of greybeards.”

I went to Beerly Functional, a get-together of down-in-the-trenches programmers who were either using or interested in using Functional Programming, and my first impression upon walking into the room was exactly that: What a bunch of fucking greybeards, myself included.

And yet, as I paid attention to what was being said, the reason why we were all there became more and more apparent. We were tired of dealing with lousy development cycles. The biggest promises of Functional Programming are that it narrows the gap, both temporally and spatially, between where you create the bug and where it manifests. Functional programming emerges from the needs of lifelong programmers to stop fucking around with code-run-debug cycles; they want to produce excellent code the first time; they’re willing to adopt strong constraints on shoddy thinking and poor code in order to avoid that. They want to make software that gives a damn. They want to make software that they are comfortable saying, “This cannot fail.” They value quality and correctness, whereas most business people… don’t know how to assess that question, and they see the rapidity and widespread availability of developers in the shoddy languages like PHP and Javascript as signs of those language’s legitimacy.

So we raised a glass together, and we took on our missions: to teach each other what we knew, to get better at our craft, and to sell it to businesspeople who need to know there’s a better, faster way to get quality code in front of consumers.  We were greybeards; lifelong programmers who, whether we’d made our millions or not, wanted to keep making great code, not graduate into management and executive by 30.   We wanted to be the best.  And we knew the tools were within our grasp.


Working around the feeling of cheating…

Posted by Elf Sternberg as Uncategorized

This is just going to be random.  Don’t expect it to make sense.

The obvious next steps in my Scheme are: Strings. A Common Lisp 22.1.1  compliant hand-written parser, as a prologue to re-writing it in Scheme, where the record objects used in Coglan’s version are replaced with cons list of the various fields being collected.

Macros.  And that’s where my brain comes a screeching halt.  Because “macros” seem to imply a certain purity of lispyness.  How do I make a Lisp macro engine that successfully handles, and yet successfully ignores, all of the extra stuff after the content?

Maybe I don’t.  Maybe I just write it so that it accepts all that, and more, because ultimately I want to add a deterministic more to the list to carry typing information.

Grief, I have no idea at this point.  I just know what I want it to be able to do next.


They lied to me in university…

Posted by Elf Sternberg as Uncategorized

As revealed last time, my university training was focused on a business degree with a programming minor, the point of which was to prepare me for being a dark programmer in some basement of a bank or other financial institution. There was an actual track for this at my university.  As such, my instructors told me not to worry too much about data structures; I didn’t need to know about trees, linked lists, and deques.

They lied.

I thought, since I’d done the exercise, that I was done with A Language In 20 Minutes, but this morning I had an inspiration and so, on the commute into work, I hacked out a new version that’s available on Github. It’s more “functional,” as I said, for some definition of functional.

But what I did add was a Lisp-like list class that behaves more or less like a Lisp cons list: A linked list of two cells, the second of which points to the next cell. Like so:

[(data:C), (next)] -> [(data:B), (next)] ->
[(data:A), null]

These lists start out with just the ‘A’ entry; when you cons an entry, it looks like: cons(D, list), and it would add a ‘D’ entry to the “head” of the list. The amazing thing about this structure is that you only have access to the head, but through it you have access to the whole list.



When you’re describing the scope of an operation for an interpreter, you still need a traversible, ordered collection of scopes that go all the way from your deepest scope to your root in order to ensure along the way that every variable is defined, and add new variables to your current scope as needed. When you create a new scope in your interpreter, you do it in a given function call that will interpret expressions within that scope. When you leave that function, you need to expunge the memory allocated for your scope. So let’s say that the D object mentioned above is a dictionary for resolving variables in the current scope; by consing it onto your list, you can now traverse the scope, checking all the dictionaries down to the root automatically.

And when you leave the scope, the D object and its list reference get reaped. You get memory management for free.

Admittedly, that’s only true because Javascript has its own memory management going on, but since lots of practice projects involve writing an interpreter on an interpreter, that’s often going to be the case. But it’s a good starting point.  And understanding the principle means that you get to apply it later if and when you decide to write this thing in C++.

The dirty secret inside Coglan’s example (see the first commit to my github) is that his Scope object is a cons list; it’s just a very informally defined one, using the parent field to point to the next cell, all the way down to the “TopScope.”

And, if you look at my javascript, I don’t call things the way most people do. You might write car(list_object) to retrieve the data from the current list object, but all throughout my code I use, in Coffeescript, (car list_object), just because Coffeescript lets me. I’ve created a Frankensteinian example of Greenspun’s Tenth Rule of Programming.


Paying my dues, Studying the masters…

Posted by Elf Sternberg as programming

I sometimes give the impression that I’m an excellent programmer, and to the extent that I’m empowered to code within my narrow little specialty, web-based interfaces for industrial and enterprise users, I’m comfortable saying I’m one of the best.  It’s pretty much all I’ve done for the past 15 years: Isilon, Spiral Genetics, and now Splunk all used my jQuery/Backbone/Python/NodeJS/HTML5 expertise to create and maintain lively and vibrant cybernetic applications (in the classic sense of “cybernetic,” i.e. control systems) for their specific deployments: A cloud-based storage solution, a cloud-based gene sequencing engine, a database and query engine for semi-structured information.

But outside of that specialty, I’m somewhat lost.  I know a lot of things, like SQL, video processing, natural language processing, and so on, that I’ve only ever played with or that I’ve only ever had to use once, professionally; these things live in the back of my brain and just kinda lie there, useless.

In the Serendipity stories I told earlier this year, I highlighted specific hiring instances where skills I had acquired for unprofessional reasons (and let’s face it, “I watch a lot of anime” is pretty unprofessional) had serious professional uses.

Today, I posted to my github a Language In 20, a quick programming language based on James Coglan’s lecture “A Language in 20 Minutes“, in which he snowflakes a programming language from nothing to a Turing-complete integer calculator capable of recursion.  I still haven’t had the necessary insight to understand the scoping issue, despite getting it right… but then I was just following Coglan’s lead.  My implementation is different; since PegJS doesn’t auto-create interfaces to parser classes, I had to write my own switch statement about types, and I tried to avoid using classes as much as possible.  I didn’t succeed; I may go back and revise it to be more “functional,” using closures rather than classes.

It’s frustrating seeing how far away I am from what I really want to know and do; it’s equally frustrating knowing that precisely zero of my corporate masters have ever had any interest in my learning this stuff.  It doesn’t make me a better programmer; in fact, it makes me aware of just how bad most programming, mine and others, really is.


The Best Firing I’ve Ever Experienced

Posted by Elf Sternberg as Uncategorized

Paul Petrone writes What it’s like to watch the worst firing ever, and in it he describes a process by which a boss, in the process of announcing forthcoming layoffs, does something pretty horrible. You ought to read it.

I can’t do one better, but I can describe the most dramatic firing I’ve ever seen. It was at CompuServe.

In 1998, after I’d been there about five years, CompuServe was purchased by AOL, then immediately sold to WorldCom in a complex stock transaction that allowed AOL to keep the users, but WorldCom to keep the wires and infrastructure. The group I was with was part of the wires and infrastructure group, but we were a redundancy. WorldCom wanted CompuServe’s Omaha offices but could care less about the Seattle space, which to them was an internal competitor with their own internet services.

The trouble was they actually couldn’t, according to Washington law, just lay us all off. A mass firing in Washington involves some rather arcane proceedings, much like a bankruptcy, and any layoff involving more than 20% of a large workforce requires the company to pay out substantially large severance. The idea is that you’re about to dump a large group of highly skilled people into the workforce; it may take a while for all of them to find work and they may have to move to do it.

So what did WorldCom do? It made us miserable. At least it tried. It took away all our work and enforced work hours. Basically, it took twenty bright developers and all the support staff of a dot-com and locked us in an office and said, “You have nothing to do for nine hours.” They did this for three months, hoping that we would see the light of our desperate straits and quit. If enough of us quit we’d drop below the minimum worker threshold and they could fire the rest without triggering the penalties.

Except we were software developers. And our in-house agreements with our in-house management allowed us to contribute to open-source projects. We made significant contributions to Perl, Apache, Python, and Linux in that time. (We also, I confess, played an awful lot of Quake: Painkeep.) Still it wasn’t meaningful work in the same way that having a real job would have allowed.

So there came a day when we were all called to the big conference center on the top floor. I’d never been there before, and it looked like the sort of place one spent too much money on: wood paneling, deep carpet, huge polished wooden table, expensive chairs. There were many more than 20 people there; all the management, support staff, sales and phone folks were there. There was a fellow there who we’d never seen before in an expensive dark grey suit. He explained, bluntly but not unkindly, that he was the guy with the hatchet. He handed out envelopes on them and told us not to open them. They had dots on them: green, blue, and white. There were a lot more green dots than anything else. “Now,” he said, “I’m afraid the green dots are being let go today.”

A roar went up from the crowd. The hatchet man stepped back, his hands in front of his face, looking quite terrified. Then he realized we were cheering. “I have to say,” he said, “This is first time I’ve done this when people cheered!”

“You have no idea how bored we’ve been!” someone shouted. “They took away all our work!”

“I’m a little sick of playing Quake!” That got a chuckle.

Our division’s senior manager, bless his heart, had managed a real hat trick of severance. Since our group was the unwanted stepchild and “in transition” involving multiple dot-com wannabes, he had managed to score concession from every single company involved in the transaction. Five months severance from one; ongoing health insurance for that same period from another; twenty hours each of professional headhunter services, including really slick resume preparation from a third; educational vouchers at the county community college and five sessions of third-party career coaching from another.

I’ve left some jobs for better jobs; I’ve been laid off four times now, three times for economic reasons rather than performance (“The Dot Com Bust” and “The Great Recession” were both brutal to web developers, even ones with obscure industrial skillsets like mine), and once because the company was “consolidating” its offices to Palo Alto and I didn’t feel like moving. The Bust and the Recession layoffs were the worst; I felt cold, clammy, and dazed upon leaving those discussions. But the CompuServe shutdown had to be the best firing I’ve ever experienced.

Open office plans has been a subject of significant controversy recently, with everyone and his brother pointing out that the quiet craft of software development clashes horribly with the loud, communal, extroverted environment of open office plans. So the question then becomes: who benefits from this arrangement?


Venkatesh Roy has this thesis– he calls it the Gervais Theory– that all modern office hierarchies have three kinds of people: the losers, the clueless, and the psychopaths. Extroverted and manipulative, pychopaths form companies and temporary alliances as they claw their way up the corporate and financial hierarchies; their lives are wrapped up in playing this game. Losers don’t play the corporate game at all; they knowingly enter into arrangements with psychopaths to do the work and go home to their “real lives.” The clueless are mostly middle management: they believe in the company, but play by the rules, believing they’ll be rewarded for their loyalty with greater rewards. Eventually, the company is filled by successive layers of middle management until it becomes a hollow shell, at which point the psychopaths cash out; the losers, who knew this was coming, transfer their transferrable skills elsewhere; the clueless are left wondering what the hell just happened. The psychopaths go on to form a new company; lather, rinse, repeat.

Software development is so hot right now that skilled developers can just up and leave. They can find work elsewhere. If they’re bored and unrewarded, they will find work elsewhere. Psychopaths need to keep a closer eye on their software developers than on other productive roles, like in manufacturing, refinery, delivery, and so on.

The open bay permits that. It gives the psychopaths in the room a horizon they can scan for trouble. It gives them an intelligence edge they wouldn’t have if developers were all in their own little rooms.

Open office bays are a mechanism for controlling restless developers.

Recent Comments