Industrial Design and Bespoke Software

I appreciate well executed industrial design, but the aesthetic and feel of a product is only a small part of that. For nearly every physical item I own, I’d much rather have an item made generic with an aesthetic, usability and accessibility marketed to millions and affordable by almost as many. Industrial design helps me not be too attached to physical items in my life: these items can and will be replaced and aren’t what I should feel nostalgic about in 10 years. To me, this is primarily how the art in industrial design is represented: an understanding of the needs of a diverse yet large population, and establishing a process from which these products can be actualized. I believe for a moment that this concept of industrial product design carries over to writing software.

Compromise is a concept represented in both the discipline of creating software and creating products, but represented in dramatically different ways. In a product, compromise is finally represented by the decisions that product will make for the user. This tradeoff in flexibility via decisions a product makes for the user is part of what makes it accessible to millions.

If we want to make the discipline of creating software accessible to millions, then software libraries need to not make these decisions. If these decisions come up, then chances are the library is trying to do too much. Amateurization of software development is not amateurization of product development. For a non-professional software developer to be able to create something that satisfies a purpose in their life, the decisions need to be available and represented by individual libraries available in a public registry. This registry serves as a toolbox, not a product catalog, for both the amateur and professional. When approaching a software problem, the developer’s role becomes choosing which decisions to make, not fighting against the inflexibility of a product.

When an amateur software developer can create bespoke items which individually satisfy needs in their life, the role of professional software developers will change. If creating software that automates your life is as easy as cooking yourself eggs in the morning, where does that leave a professional developer? There will probably always be some level of need for the professional software chef at a Michelin starred restaurant. But I think we’ll see a greater need for people of orange software developer aprons at the future version of the Home Depot.

Lexicographical Key Sorting in LevelDB

As an end user, one of my favorite features in LevelDB is the lexicographical manner in which it sorts keys. Taking advantage of this special sorting when doing a range query in LevelDB allows you to really narrow in on keys without having to do a complex and/or resource intensive map-reduce. Actually, forget about map-reduce altogether. If you don’t know about it already, I’m envious.

Lexicographical key sorting in LevelDB results in keys being ordered by the position of their characters according to their position in the ASCII character set. (‘A’ being 65th, ‘!’ being 33rd, ‘8’ being 56th, and so on)

Similarly to how you applied alphabetical order to lists of words in elementary school or community college, LevelDB does so based on the first unique character in the key, and uses its position in the ASCII character set. Alphabetical sorting and lexicographical sorting have the same result in an example like this:

technology
technovangelist

Both words are similar up until the 7th character, which is then used to determine the sorting.

Let’s introduce non-letter characters, say we have keys like the following:

fear!uncertainty!doubt
fear-uncertainty-doubt

Since the first unique characters are ‘!’ and ‘-‘, the key with ‘!’ is sorted first (‘!’ is number 33, ‘-’ is 45).

These are pretty contrived use cases, so let’s try a more real world example with keys for user profiles and media the user has uploaded. It might look like this:

user-media!luke!2013-06-13T00:32Z
user-media!ramrod!2013-06-12T00:32Z
user-media!vonfudson!2013-06-13T00:32Z
user-profile!luke
user-profile!ramrod
user-profile!vonfudson

The keys here are first sorted based on the first unique character (following ‘user-‘). Within this sorting namespace we’ve created, keys are sorted again based on ASCII character set position. You may have noticed an ISO-8601 formatted timestamp as well in the user-media keys! Timestamps like this are extremely useful when doing a range query sorting due to their uniform length.

So, assuming you have a basic understanding of how a lexicographical sorting system works, let’s discuss how it makes LevelDB’s range queries so exciting.

Assume our keys are formatted as the above example, but with a million entries! What if we want to get a list of users whose name starts with the letter ‘L’?

We’ll do a range query with the following parameters:

start range: 'user-profile!l'
end range: 'user-profile!l\xff'

The result of this query in LevelDB will give us all documents with a key starting with ‘user-profile!l’. The start range should be straightforward, but what’s that \xff on the end range? You might recognize this as an ASCII character expressed by an escape code. When using it like this in the end range, it represents the last key that matches the end range argument. Since our start range represents the first key in the set we want and our end range represents the last, we’ll get everything that is sorted between the prefix specified!

What about the ISO-8601 timestamp I was over-exuberant about earlier? Was that an excuse to use an exclamation point in a sentence to betray my true feelings or is that actually useful?

Consider this example:

start range: 'user-media!vonfudson!2013-06'
end range: 'user-media!vonfudson!2013-06\xff'

You might see what we’re doing here already — retrieving keys that were timestamped in June, uploaded by our main marketing guy, Von Fudson. Since ISO-8601 timestamps have a consistent character length for each time representation and use a 24 hour clock, we can reliably retrieve our namespaced range chronologically. This guy sure loves cat photos!

These are all pretty simple examples, but hopefully you now have an idea of how this type of sorting allows a LevelDB user to query ranges of keys without resorting to a map-reduce algorithm from the beginning. Namespacing can get a lot more involved than this, and you may even be interested in using some of the node modules LevelDB community members have written to help automate the process.

LevelDB is a fast & reliable way to store your data on the server, and it’s exciting to see an open source community grow around software that’s usually fostered by large software companies with significant financial interests in mind. If you run into trouble or have questions about how to implement stuff, join us in the node.js freenode LevelDB room: ##leveldb (yes, two hashes).

Unix Is Not Fragmented

darwin: lsof -i | grep LISTEN

solaris: lsof -p

linux: netstat -lnp

openbsd: netstat -nat | grep LISTEN

Killing Buttons Don’t Kill People

     It is the future and it’s now illegal to manufacture, sell, buy, trade, or own any kind of firearm. First, manufacturing ceased, and governments in both hemispheres compensated industries as necessary (at the expense of citizens). As public incidents of violence persisted, buying, selling, and trading of firearms was outlawed multinationally.

     Many years after the outlawing and subsequent confiscation of firearms, the killing button was unreluctantly invented. The idea behind the killing button was simple: create a device that can only operate as instructed on a specific target. The goal was to require a certain amount of skill and finesse to operate the button — it was of course designed by a committee. The committee was not politically aligned or representative of a single region. Scholars, engineers, poets, consultants, senior managers and filmmakers from all around the world provided thoughtful input into creating the killing button.

The eventual technical requirements for the killing button were as follows:

  • the killing button must be red, to indicate that it involves deadly force (this rule was quickly revised to include options for black, dark green, various camouflages, neon pink, and gold plated)
  • the killing button must have a minimum diameter of 24mm and a maximum of 48mm
  • the killing button must make a loud noise to alert, but not to disturb. This was eventually decided to be no louder than 99 dB.
  • the noise must sound terrible, but not too terrible.
  • the user must mentally assure himself of the target. the killing button will not operate if the operator is not reasonably sure he is using the button on the target he intends to kill.

     Manufacturing was handled by state owned industry in most nations. Law enforcement personnel were the first to be trained and equipped with the killing button for field use. Through dedicated practice, pressing the button became an action programmed into muscle memory. Shortly after law enforcement around the globe realized the effectiveness of the button, nations allowed private citizens to purchase and own them as well. Many citizens were delighted upon realizing that it did not actually require skill or practice to operate a killing button.

     Like any product on the market, a killing button could be easily abused. Appropriate warnings were made public and printed on the package of every killing button in nations that required full disclosure:

  • A killing button will not decide to kill on its own.
  • In most cases, it can not select a target, it can not press itself and it generally can not operate without the interaction of a human.
  • It can not be purchased by a criminal and legally cannot be operated without proper licensing.
  • Using a killing button without proper authorization may be a violation of law.
  • Please use killing buttons responsibly.

     Unfortunately, misuse of killing buttons became rampant after only a few years on the market, and citizens of nations around the world were protesting the sale and private ownership of killing buttons, pleading with governments to intervene and outlaw the weapons. Various chapters of the Responsible Killing Button Users Group publicly rallied, proudly displaying responsible use and carry of killing buttons. They claimed that Killing Buttons don’t kill people: it was the users, or rather, criminals.

Wai Khru Ram Muay

Prior to fighting, Thai Boxers perform a traditional dance in the ring with the intent of paying respect to their teacher, or khru, and to apologize to the king for the upcoming violence. This elaborate dance goes on for a few minutes as traditional Thai instruments play outside of the boxing ring. No Wai Khru is the same, as many of the actions have a regional and cultural significance to the fighter and his camp. A fighter’s Wai Khru tells his or her individual history as a fighter.

While the Wai Khru ritual is important for paying respect in the ring, its varied movements are also meant to energize the fighter and intimidate the opponent before a brutal battle of elbows, fists, knees and shins. As much as the dance is about displaying respect, it’s also an important warm-up activity.

A foreign Muay Thai practitioner is known in Thailand as a Nak Muay Farang. Many Nak Muay Farangs choose to skip this ritual before fighting. It’s not mandatory, even in Thailand, but you’ll rarely see a native Thai boxer not perform a Wai Khru prior to fighting. A native Thai Boxer trains to fight from childhood and this includes training the Wai Khru. Not performing the Wai Khru prior to fighting would feel foreign and possibly even insulting to a native fighter.

Involvement and contribution to open source on an individual level is driven by various values. For some, it’s hoping that creating open source projects and contributing to others will help with finding employment in the future. Many employers want to see a candidate’s involvement in open source projects to not only demonstrate ability to code, but also cooperation with other programmers working towards solving a similar problem. Savvy computer science students are building active Github accounts in addition to securing internships at well-known organizations. These students are learning and performing their Wai Khru, but it’s not the only way to do so.

There are many smart and capable programmers who don’t use Github or contribute to open source through other avenues. These programmers are not necessarily any less capable of getting the job done and it is impossible and wasteful to make any assumptions about their values and competency. One of the most endearing things about open source is the complete lack of any obligation whatsoever. This lack of obligation exists even despite consumption entitlement and absurd community backlash we sometimes see when a project is of a size to displease enough consumers.

In Thailand, when the combat officially officially begins, the musicians play the same music that was played during the Wai Khru. As the fight’s rounds pass, the music’s tempo increases and the tempo of the fight often follows suit. The tempo increases until a fighter is knocked unconscious or until the rounds are exhausted.

Many professional Thai Boxers fight multiple times a month in more populated areas of Thailand, hoping to support a family by sending participation payouts or a winning purse back home. The Wai Khru is not something performed once or occasionally during the fighter’s career. It’s performed at every fight. The Wai Khru is a part of the match as much as the combat. The fighter is not paid to show up and perform his or her Wai Khru, but at no point does the fighter consider it superfluous. For a lot of programmers, involvement in open source has a lot of similarities. The companies we work for don’t pay us to contribute and write software that others can use freely. Sometimes it may help prepare us for the work that earns us a paycheck, but it’s not something to be done once. It’s part of the act of building software as a whole, and it contains your individual history as a programmer.

It’s Morning

At NodeConf Summercamp, conference attendees/campers can either bring their own tent and camp out, or stay in one of the provided cabins. As a convenience oriented person with a minor in laziness, procuring and transporting a tent all the way from NYC to SF seemed like a lot of work, so I chose the sleeping indoors option.

Cabins are of various sizes, and Mikeal was kind enough to assign me to the “Teacher” cabin, which was pretty luxorious by camping standards. It had its own showers, bathrooms, a fridge, and was only two beds per room. Those things are a big deal when you’re in the woods.

There were a few of us staying in Teacher, and I ended up rooming with Raynos. He was a great roommate (probably much better than myself), but every morning before the 8am breakfast, he would open up all of the blinds in the room. Bright daylight would attack me and force my eyes open to see Raynos standing in the room — ready for the day and looking back at me. Both mornings of the unconference, he would do this prior to departing for breakfast and say to me, “It’s morning.”

Once he left I hurredly closed all of the blinds and attempted to sleep for another hour, figuring I would catch whatever was leftover from breakfast before heading to the sessions. I’m the type of person that upon waking up, I can not mentally process the idea of anything more important than returning to sleep, even if only for a few extra minutes. My multiple attempts to change this have all been met with failure, but this is the type of failure I’m happy to accept.

I’ve thought about this series of wakeups every couple of weeks over the past few months. There’s a good chance Raynos was just intending to let me know it’s morning and time to start the unconference, but I took a little bit more out of the saying. At any given moment we can reflect on the things we’ve done over the past year or two and be happy with ourselves. I’ve been asked multiple times by a few, “What’s your end game?.” That’s not an easy question to answer. Other than assuming that there’s a beginning, middle, and end (or even some semblance), it assumes that we are in or have experienced the beginning, and are in or awaiting the middle. For many of us, there’s always a lingering urge to step further. This endgame finality does not exist, and this endgame question is paradoxical.

The beginning, middle, and endgame business metaphors are lost on me, so I’d much rather rely on correlation to real life events. The beginning is when I was born and the endgame is when I will no longer be alive. The middle is all of the time until the endgame. The middle is when things happen. The middle is when you respond to the lingering urge to step further.

The middle is where morning exists.

It’s morning.

Hassle-free CouchDB Installation on Debian

I like to tell myself that someday the official Debian repositories will have a more up-to-date version of CouchDB than 0.11.0-2.3, and users will rejoice while they easily and quickly install CouchDB like they can currently do with more popular database servers. Until that day (or until a good-natured person maintains a public repo of their own with current CouchDB packages for Debian), building yourself is the way to go.

I’m a big fan of Iriscouch's build-couchdb repo. It’s easy to use and reliable, but start to finish takes kind of a while due to retrieving and building additional sources via a git submodule prior to the actual Couch build process. Then you run rake, that ruby build tool that you don’t use in all of your other ruby projects that you don’t make, because you (like myself) are not a decorated rubyist. Regardless, this is a dependable way to build CouchDB.

…It’s just a lot of steps for a lazy person.

If you’re lazy like me and want to run a single script to install and setup CouchDB, check out couch_install_scripts. There’s only an installation script for Debian 6.0 at the moment, but this will grow in the future to gather scripts for more platforms. The goal here is to have a single script you can run instead of a series of commands, without requiring anything to start other than your system’s native package manager. Not only does it install dependencies, build and install CouchDB, it also takes care of setting up a dedicated user in the system and linking Couch’s init script. Since it’s just a simple bash script, it should be no problem for users of various skill levels download, run, and hopefully grok.

CouchDB is fun and easy to use, and installation should be just as fun and easy.

The TypeScript Community

On Monday afternoon at about 1:30PM Eastern I’m getting off a plane, and my Twitter feed is buzzing with various comments about TypeScript, Microsoft’s new compile-to-Javascript language.

Wait a second — Microsoft, of all companies, released a new compile to javascript language? The Microsoft we love to hate, despite their promotion of node.js, the open source tools they’ve released, their involvement in the javascript community, and sums of the money they’ve given as sponsorhip of community JS conferences around the world, has released a project that elicits a very predictable emotional response from most of the community.

Let’s try to be logical.

Facts

  • TypeScript is technically a superset of javascript and supports regular javascript.
  • TypeScript can be written and compiled to javascript on any platform that supports node, using the typescript module on NPM.
  • TypeScript compiles to javascript, like CoffeeScript and Dart. Unlike Dart, it can not run in a standalone TypeScript VM as such a thing does not currently exist (to my knowledge).
  • TypeScript is given special attention on Windows, evident by the Visual Studio plugin offered by Microsoft.
  • Microsoft is fostering TypeScript development only on CodePlex, which is sort of their version of Google Code.

Reflection

This might be a bit of a stretch, but TypeScript seems to be pretty targeted towards programmers that stick to Microsoft’s programming languages, platforms, operating systems and IDE. Vim user on Unix/Linux? You’re probably not the target developer to whom Microsoft is attempting to appeal. The point of the language structure and paradigms offered by TypeScript seeems to be more about engaging programmers familiar with [classical inheritance](http://en.wikipedia.org/wiki/Inheritance_(object-oriented_programming) and less about offering a replacement to javascript. Maybe the goal is encourage popular design patterns [1] without the user actually having to know how to do such a thing in real javascript. I’m not totally sure about this part, but it wouldn’t be too far of a stretch.

There’s a New Restaurant in Town

On Taco Conf day 2, Malte Ubl gave a great talk regarding compile to Javascript languages and community reaction to them. The talk video isn’t online yet, but the gist of it is a genius analogy regarding a new restaurant opening up in your city. Just because you’re not a fan of the spot doesn’t mean you would rally against others that might honestly enjoy it. The same goes for programming languages. There are plenty of people that enjoy writing pure javascript, and a language that compiles to javascript offers little threat to the language or the people that use it, so why bother spending your time advocating against it? If TypeScript adoption gets to a mentionable level, I bet it ends up being predominately in enterprise. My only concern from this would be that enterprise might not need TypeScript over pure javascript, but the idea that TypeScript is the reason pure javascript adoption might not happen is silly.

Javascript clearly isn’t going anywhere. It has the most diverse and most exciting programming community on the planet, and it’s understandable to feel threatened when a big company debuts a language that at first glance makes javascript feel like a second-class citizen. In actuality, it’s incredibly unlikely that we’ll see TypeScript make a dent in what most of us know and love about javascript programming.


  1. I think most of the patterns on this page are a really bad idea and should not be used. 

The Internet & Open Source

Every now and then I get an unsolicited reminder of why the Internet and open source are awesome. It could be a friend I haven’t spoken to in a while casually mentioning he’s using a library I’ve written or contributed to in his own project, or even discovering an awesome tool or library that saves me the time of having to write it myself.

Today, it was an unsolicited email from the author of a command line password management utility:

Hey Luke & Roman,

I saw your twitter conversation about pass — namely that it doesn’t support multiple users. Interestingly enough, just yesterday someone emailed the pass mailing list about this (both copied), and I replied that although it doesn’t support it, it’d be trivial to add. Then, thirty seconds later, I realized that due to a super nice GPG feature, pass does in fact already support this. Check it out:

Fundamentally, pass comes down to this gpg command:

gpg -r “Some user provided id” -e -o “output file name”

gpg can support multiple recipients by adding more -r commands — this is the trivial change I was talking about. But it can also do something much better and more robust. GPG has a notion of “groups”, where you can assign one recipient ID to multiple recipients. So, this command:

gpg -r “team” -e -o “output file name”

winds up expanding to all of the individuals on the team, encrypting each password efficiently and securely with each of their public keys, allowing any one of their private keys to decipher it.

But nevermind all these technical details. How does it work in practice? Super simple:

You set up a GPG group using “gpg —group”. This simply adds a line to ~/.gnupg/gpg.conf that looks like “group team = paige 0x12345678 joe patti”.

Then, you init pass using “team” instead of your key:

pass init team

Simple as that. From now on, passwords will be encrypted using multiple keys.

Hope this helps. Let me know if you have any questions, and please don’t hesitate to join the pass mailing list: http://lists.zx2c4.com/listinfo.cgi/password-store-zx2c4.com

Jason


The internet is awesome.