Why Install When I Can Dockerize?

March 2nd, 2015 PersonalOps, Technical

I haven't posted in forever. This post is mostly so I can break that streak. As such, there may be a lack of super interesting things in it.

Software Rant

I'll avoid the tired "containers are just encapsulation, man" rant. I might normally make the whole "nothing new under the sun" point if it weren't for the fact that the I'm currently mired in how wrong I think the "microservices are just SOA" crowd is. Just be glad that people are interested in the new thing and hope that they make it shinier.

I dislike having to install things on my laptop. Not everything, of course, but I don't want to install something for every little thing I want to play around with. Everything starts with install Ruby, Python, Go, D, Node.js then let them install who knows what (because everybody has to reinvent the package manager), who knows where, and create whatever inter-version incompatibilities they feel like. Then I wind up on StackOverflow trying to figure out the installation and debugging tricks those communities take for granted.

Sadly I still wind up with most of this shit on my box thanks to a handful of awesome tools like Ansible and Vagrant. I know it's ironic but I still have to draw the line somewhere.

WiFi Rant

So, I'm working at Google now (if you say congratulations I will punch you in the taint) and taking the bus to work. The bus has very shitty WiFi so to pass the time I either read e-books or watch downloaded content. XFinity has the best DVR option for this since their Android app will let you download anything you've recorded for offline viewing–fucking bad ass. Netflix and TiVo (and YouTube I believe) think that super awesome, streamy, creamy WiFi is everywhere. It's not and companies that don't support offline viewing hate America, freedom, and teenage Jesus. You really don't hear enough about teen Jesus…

Then I thought it'd be great to download some of the blog post backlog I have in Feedly (my RSS reader currently because fucking Google shut down Reader). That'd let me catch up on some of the stuff that falls under the "sharpening the tools" category of time wasting. Unfortunately 95% of the stuff in there comes from InfoQ which, to their credit, have video AND slides synced in their presentations. This means simply downloading video alone isn't good enough. What's a boy to do?

"Someone Beat Me to it" Rant

Someone else already wrote something to download InfoQ presentations for offline viewing. Since it's written in Python I took it for a test drive in a VM to avoid unnecessary dependency baggage from invading my system. The sumbitch works like a charm. It pulls everything into a folder and creates an HTML page you can use for offline viewing.

Fine. So I don't get to do that as a fun mini-project. I could still do something cool with Docker. For this post I don't care if Docker (or Rocket) will save the world. I care that I can encapsulate a bunch of shit into a container that is lighter weight than a virtual machine and treat it like a command. The inspiration came in the form of someone else's post that takes it even further with X11 applications. Baby steps.

Well some motherfucker beat me to that, too. Luckily though he used a bad naming convention and I couldn't get his container to work. So I built my own any way.

What Did I Accomplish?

I now have a bash script that wraps the slightly onerous Docker command to launch my container and download an InfoQ presentation for offline viewing. Yay! I also got to increase my public GitHub footprint (never a bad thing).

I learned that Docker handles host to container file permissions badly, Docker security has a long way to go, Docker Hub has some pretty nice continuous integration options for GitHub projects that have Dockefiles, and finally that I think it's better to waste 458mb of disk space to avoid putting more Python stuff on my laptop. Not a bad Saturday afternoon.


Everyday Usability (Cruise Edition)

March 4th, 2013 Random Thoughts

To take advantage of my self-imposed unemployment the wife and I have been vacationing it up lately. In February 2013 we took a cruise on the Disney Magic to Grand Cayman and Cozumel out of Galveston. In your state room you are given a door hanger that has two very different messages on it. This is to let your room host (the person responsible for servicing your room) that you either don't want to be disturbed or that you'd like for them to attend to your state room. Here's what they look like:

IMG_9271 IMG_9272

A couple of cruises ago we spent almost half the cruise not realizing the two sides were different. Around the third or fourth time the host tried to service the room while we were taking a mid day nap we realized what idiots we are. That got me to thinking about how you could improve these door hangers both for the guests and for the room host. For the guest you want something that makes it even more obvious that the two sides have different meanings. For the host you would like a method of differentiating the message being communicated by the door hanger that can be discerned from a greater distance or from a more extreme angle that doesn't allow reading it. Of course you have the different images on the hanger but that's not terribly visible when looking down a hallway of a couple of hundred doors.

I think using a different color for each side would work even better. Since we're not communicating the intent of the different sides solely through color I think were still good in terms of accessibility and color blindness (I should also note the lack of Braille on the different sides of the door hanger). I would naturally pick red and green if not for the horrible memory of a former manager and a firmware engineer arguing for hours about the inherent ambiguity of the two colors when multiple nationalities and cultures were involved. Anyway, here's a stab at what my "improved" version would look like.

door_privacy_fixed door_service_fixed


Splitting Files By Column Value Using Awk

August 9th, 2012 Programming, Technical

At the day job a data fairy gives me a giant pipe delimited text file that contains data for a bunch of our customers. The customer ID is contained in one of the columns. Ideally I'd like to have one file per customer but it's usually very difficult to get data fairies to do the things you want.

For reference here's a reasonable facsimile of what the file looks like. Let's pretend this is some sort of interesting survey. Bonus points if you can figure out a question that would make sense for these answers.

"Once in college but it wasn't my idea."|3|"CUST1"|"blah blah"
"Like your mom."|14|"CUST2"|""
"Blame it on the dog."|15|"CUST1"|"Frankenberry"
"That wasn't chicken."|9|"CUST2"|"Definitely the mouth."
"Never professionally"|26|"CUST3"|"And then she stepped on the ball!"

What we want is three files: one for each customer. We drop the split file in a different directory for each customer to keep things a little neater and we name the file with the customer code prepended to the original file name. All nice and orderly.

As with many things involving text files this winds up being stupid easy using Awk. I'm showing it here mostly so I can find it again and because this type of command line file processing always makes me giddy. The comments should do a good enough job of explaining things.

#! /usr/bin/awk -f
  if($CUSTOMER < 1) {
    print "Usage: split -vCUSTOMER=[split column] [files]";

  # Set the input and output field delimiters
  "mkdir -p split" | getline;

  # If this is the first line of a file...
  if (FNR==1) {
    # Grab the entire first row as the header

    # Close open files from the previous file (if any)
    for(customer in customers) {
    # Clear the array of customers / output files   
    delete customers;

  if (FNR!=1) {
    # Grab the customer code and strip out the quotes
    gsub(/"/, "", customer);

    # Store the output file name.  This is the customer code followed 
    # by the original file name.
    outputFile="split/" customer "/" customer "_" FILENAME;

    # If this is the first time this file we've seen this customer code...
    if(customers[customer]=="") {
      ("mkdir -p split/" customer) | getline;

      # Overwrite any previous output file and print the header
      print header > outputFile; 
      # Track the fact that we've seen this customer code and store the output file

    # Append the current line to the output file
    print >> outputFile;

I'm sure someone could do this more succinctly and without some of the odd things I've done in there (maybe parameterize the delimiters or the output directory structure), but I kind of like it. It's already proved useful for a number of other cases for me. Also the fact that it's relatively tiny and super fast is all the answer I need if one of the co-workers asks why I didn't write it in Java.


Door Handles and Usability

May 25th, 2012 Business, General, Random Thoughts

I just finished watching "J. Crew & the Man Who Dressed America" on CNBC. I recommend it but then I love those sorts of pseudo documentaries. It's about Millard "Mickey" Drexler and mostly his time spent at the Gap and J. Crew. One interesting thing about him is that he is mostly responsible for the success of the Gap and quite possibly deserves a large amount of credit for the success of the Apple Store. He's a retail / fashion specialist.

So I'm watching this documentary about an incredibly smart micro-manager (in a good way) that turned the Gap around and is slaying it at J. Crew. He and the interviewer are in front of one of the J. Crew retail stores and Mickey Drexler spends 5 or 6 seconds trying to open the door for the store:

Mickey Drexler trying to open a door

Now I can't fault him because even though the tiny text above the handle says "push" the handle screams "pull." Donald Norman goes over this in The Design of Everyday Things (another in a long line of books I recommend highly). You can see some of his thoughts on door handle design here:

Door Handle Design

You may not be equally amused as I was at the idea of a design / fashion obsessed CEO of a multi-billion dollar corporation not being able to open a door to his own store in under 6 seconds but for some reason the whole thing made me absolutely giddy, especially since it pointed to an example of the practical application of the lessons I had learned from Norman's book. This all has practical application potential in my day job as well: buttons need to simply look "pushable", there needs to be a visual cue for click to edit, the manual won't save you from poor design (see the small text above the door handle), and your poor user has no idea there's a context menu lurking behind that rather innocuous looking area of the UI. But, I digress.

Here's another instance that always makes me smile:

Intentionally Bad Usability

This is an instance where intentionally difficult usability is appropriate. The exit door for a school for handicapped children is made hard to operate. Making it extremely easy to use could have dire consequences (with no "undo" operation available to save the day).

I'll leave the "what we can learn from all of this" wrap up text as an exercise for the reader. For me it's just read more books and watch more non-fiction television. It helps me get my point across with what I consider to be cool phrases like: there are some doors that CEOs can't open.


He's Got This Ultimate Set of Tools

May 6th, 2012 Business, Programming, Project Management, Random Thoughts, Technical

"Relax, all right? My old man is a television repairman, he's got this ultimate set of tools. I can fix it." If you don't remember your Fast Times at Ridgemont High quotes you're probably not alone. The scene is worth remembering because the context is ridiculous. So it is sometimes with software development. The cost and effort of fixing the existing implementation is sometimes just too great. The changes cut too deep. You're better off throwing out the current stuff and starting from scratch.

In software development you rarely understand your problem domain perfectly, if ever. You learn what your customers want through trial and error. Sometimes your organization has made such poor attempts at delivering the product people want that you can't help but throw away what you've currently got and try again with what you learned from your previous attempt.

Managers usually hate to hear such talk from developers. Developers always want to rewrite things. But in some rare cases they're absolutely right. Refactoring is great if you're even remotely close to what you want to do. But what if your product is built on bad assumptions of epic proportions?

Could CVS have been refactored incrementally to arrive at git? Could Windows have been refactored to create Linux? Could MacOS have been refactored to create OSX? Could Internet Explorer be refactored to create Chrome? When do you come to the realization that what you want, what you need, is so far away from what you have that you can't get there from here? When is the cost of making changes to your current product artificially inflated by the technical debt and faulty abstractions to the extent that it's better to throw it all away?

That's the advantage your competition has. You've shown them your near miss at a great product. If the people in your organization advocating a rewrite were magically transported into a competing startup that was creating a competing product from scratch would you be at all worried? If the answer is "yes" then you should use the advantages you have (those very same people plus a more intimate knowledge of the problem domain and where you went wrong) and do something about it. Plus if something in your product actually proves useful you can copy and refactor it into the new product.

There are certainly risks but the rewards are incredible.


Autowiring Jackson Deserializers in Spring

May 2nd, 2012 Java, Programming, Spring, Technical

Recently I was working in a Spring 3.1 controller for a page with a multi-select of some other entity in the system. Let's say an edit user page that has a User object for which you're selecting Role objects (with Role being a persistent entity with an ID). And let's further say that I'm doing some fancy in place editing of a user within a user list so I want to use AJAX and JSON to submit the user to the server, for whatever reason (probably because it's rad \oo/).

Okay now that we have our contrived scenario I want to serialize the collection of roles on a user so that they're a JSON array of IDs of said roles. That part is pretty easy. Let's just make all of our persistent entities either extend some BaseDomainObject or implement some interface with getId and then write a generic JSON serializer for Jackson:

package com.runningasroot.webapp.spring.jackson;

import java.io.IOException;
import org.codehaus.jackson.JsonGenerator;
import org.codehaus.jackson.JsonProcessingException;
import org.codehaus.jackson.map.JsonSerializer;
import org.codehaus.jackson.map.SerializerProvider;
import org.springframework.stereotype.Component;
import com.runningasroot.persistence.BaseDomainObject;

public class RunningAsRootDomainObjectSerializer extends JsonSerializer<BaseDomainObject> {

    public void serialize(BaseDomainObject value, JsonGenerator jgen, SerializerProvider provider) 
            throws IOException, JsonProcessingException {

Awesome if that's what I want. We'll assume it is. Now if I submit this JSON back to the server I want to convert those IDs into real live boys, er, domain objects. To do this I need a deserializer that has access to some service that can find a domain object by ID. I'll leave figuring out ways to genericize this for multiple domain objects as an exercise for the reader because frankly that's not the part I'm interested in.

So how do I control how Jackson instantiates deserializers and make sure that I can inject Spring beans into them? You would think it would be very easy and it is. Figuring it out turned out to be unnecessarily hard. The latest version of Jackson has a class for this and even says that's what it's for. So let's make us an implementation of a HandlerInstantiator that is aware of Spring's ApplicationContext. Note that you could do this entirely differently with an interface from Spring but who cares? Here's what I did:

package com.runningasroot.webapp.spring;

import org.codehaus.jackson.map.DeserializationConfig;
import org.codehaus.jackson.map.HandlerInstantiator;
import org.codehaus.jackson.map.JsonDeserializer;
import org.codehaus.jackson.map.JsonSerializer;
import org.codehaus.jackson.map.KeyDeserializer;
import org.codehaus.jackson.map.MapperConfig;
import org.codehaus.jackson.map.SerializationConfig;
import org.codehaus.jackson.map.introspect.Annotated;
import org.codehaus.jackson.map.jsontype.TypeIdResolver;
import org.codehaus.jackson.map.jsontype.TypeResolverBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.stereotype.Component;

public class SpringBeanHandlerInstantiator extends HandlerInstantiator {

    private ApplicationContext applicationContext;

    public SpringBeanHandlerInstantiator(ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;

    public JsonDeserializer<?> deserializerInstance(DeserializationConfig config,
            Annotated annotated,
            Class<? extends JsonDeserializer<?>> deserClass) {
        try {
            return (JsonDeserializer<?>) applicationContext.getBean(deserClass);
        } catch (Exception e) {
            // Return null and let the default behavior happen
        return null;

    public KeyDeserializer keyDeserializerInstance(DeserializationConfig config,
            Annotated annotated,
            Class<? extends KeyDeserializer> keyDeserClass) {
        try {
            return (KeyDeserializer) applicationContext.getBean(keyDeserClass);
        } catch (Exception e) {
            // Return null and let the default behavior happen
        return null;

    // Two other methods omitted because if you don't get the idea yet then you don't 
    // deserve to see them.  phbbbbt.

Great now we just need to hook up a custom ObjectMapper to use this thing and we're home free (extra shit that would probably trip you up as well included at no extra charge):

package com.runningasroot.webapp.spring;

import org.codehaus.jackson.map.DeserializationConfig;
import org.codehaus.jackson.map.HandlerInstantiator;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.SerializationConfig.Feature;
import org.codehaus.jackson.map.annotate.JsonSerialize;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.stereotype.Component;
import com.fasterxml.jackson.module.hibernate.HibernateModule;

public class RunningAsRootObjectMapper extends ObjectMapper {

    ApplicationContext applicationContext;

    public RunningAsRootObjectMapper() {
        // Problems serializing Hibernate lazily initialized collections?  Fix here.
        HibernateModule hm = new HibernateModule();
        hm.configure(com.fasterxml.jackson.module.hibernate.HibernateModule.Feature.FORCE_LAZY_LOADING, true);

        // Jackson confused by what to set or by extra properties?  Fix it.
        this.configure(DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        this.configure(Feature.FAIL_ON_EMPTY_BEANS, false);

    public void setHandlerInstantiator(HandlerInstantiator hi) {

Now you just have to tell everything to use your custom object mapper. This can be found elsewhere on the web but I'll include it here in case of link rot:

package com.runningasroot.webapp.spring;

import javax.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJacksonHttpMessageConverter;
import org.springframework.stereotype.Component;
import org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter;

public class JacksonConfigurer {
    private AnnotationMethodHandlerAdapter annotationMethodHandlerAdapter;
    private RunningAsRootObjectMapper objectMapper;

    public void init() {
        HttpMessageConverter<?>[] messageConverters = annotationMethodHandlerAdapter.getMessageConverters();
        for (HttpMessageConverter<?> messageConverter : messageConverters) {
            if (messageConverter instanceof MappingJacksonHttpMessageConverter) {
                MappingJacksonHttpMessageConverter m = (MappingJacksonHttpMessageConverter) messageConverter;

    public void setAnnotationMethodHandlerAdapter(AnnotationMethodHandlerAdapter annotationMethodHandlerAdapter) {
        this.annotationMethodHandlerAdapter  = annotationMethodHandlerAdapter;

    public void setObjectMapper(RunningAsRootObjectMapper objectMapper) {
        this.objectMapper = objectMapper;

I think you can also perform this bit of trickery inside of an application-context.xml. But whatever works for you works. I think Yogi Berra said that.

Of course you still need to annotate your getters and setters with special Jackson annotations:

public Collection<Role> getRoles() {

// Some deserializer with some hot Spring injection going on in the back end (if you know what I mean)
public void setRoles(Collection<Role> roles) {

So there you have it: an example of a Spring Jackson JSON serializer that serializes the contents of collections of domain objects as an array of IDs and then deserializes JSON arrays of IDs into domain objects to be put into a collection. Say that three times fast.


I Think We're Going to Need a Bigger Box

April 10th, 2012 Business, General, Programming, Project Management, Random Thoughts, Technical

I was reading this post on the Instagram buyout by Facebook today and it got me to thinking about the benefits of the cloud, DevOps, horizontal scalability (one of my favorites), and well thought out architectures and monitoring.

One of the more interesting things about the $1 billion purchase price is that Instagram has 13 employees and 35 million users. That's just so crazy to me. It also ends up being yet another argument against the "bigger box" method of solving scalability issues. Eventually you cannot simply add more RAM to fix things. Trying to solve your problems that way is like trying to solve world hunger by breeding a single, giant cow.


Let's Just Burn It All Down and Start Again

April 7th, 2012 Business, Programming, Project Management, Technical

All software sucks to some extent including everything you are working on right now. If you reexamine your code six months from now and don't think it sucks then it probably means you didn't learn anything in those six months. That's the downside of being a software developer. You feel like the code you're working around is some degree of horrible. For the most part you just accept it and try to make incremental improvements to things. If you're lucky you'll work on something that you think is magnificent (and then think it's shit in six months).

But what happens when the code is truly horrific? For example: you wrote your own FTP client, your own templating engine, you have mutating getters, there's database access in your pages and data objects, you cut and paste DDL statements into SQL clients and call it "upgrading the schema", etc. We can argue about whether some of those things are truly bad but from my perspective they're pretty rotten. Throw that into a 100k+ line code base with many active customers and too few developers and then you've got some real fun.

In these situations I can envision a more ideal code base pretty easily. Update the libraries and start using them, fix the schema that no longer matches the problem domain (if it ever did), start pushing things into neat little tiers, get rid of that shitty build, run a continuous integration build server, use Chef or Puppet to manage configuration, scale your shit horizontally and get all elasticy with the cloud, etc. Pretty soon I've built a shining city on the hill in my mind. The only problem is I'm still calf deep in shit and I need to go back to standing on my head just as soon as my lunch break is over.

My solution has always been to burn everything to the ground and start over. It's not a popular position even among software developers. "Let's just slowly fix everything that is wrong," they say. It sounds good but progress on paying down your massive technical debt always seems to take a backseat to a shiny new feature (with its own share of technical debt). Pretty soon you're not even paying the interest on that debt. Nope. Burn it all down. Or at least build a new bridge next to the old bridge and then blow the old bridge up. Maybe you can even be nice enough to divert traffic first.

The "fix in place" crowd always sounds like this to me: "I bought a new motorcycle. It's a Honda. I kind of want a Harley instead. Can you turn it into a Harley while I ride it around? Thanks. xxxooo"

At least I'll always have these rants before the void. Thanks for listening.


Geek TGI Friday's Flair

September 19th, 2011 General, Programming, Random Thoughts, Technical

TGI Friday's walls are littered with "vintage" wall decor. Red Lobster has old lobster traps and fish photos all over their walls. Then it hit me: geek hangouts need their own brand of wall flair. Why not outdated tech books?

I've got a ton of books on technologies that aren't in widespread use any more. I'd donate them but even Goodwill doesn't want stuff like that. When you think about it it makes sense. So where do they go? The landfill? I like to pretend I'm much more environmentally friendly than that.

Some hangout for geeks needs to step up and offer a free appetizer or something for anyone that brings in a tech book that was published before, say, 2000? That seems like a reasonable cutoff. Then all the geeky people can laugh at the titles lining the shelves above their tables. "PowerBuilder? Oh, shit! I wrote something in that once!" (Apologies to Sybase, but you really need to give up on that shit.)


Thoughts on Call of Duty Elite

July 20th, 2011 Business, Community Management, Entertainment, Random Thoughts

For those of you that may not know Activision recently announced an online service called "Call of Duty Elite". This is directly related to their Call of Duty game series and works on kind of a freemium model. Ultimately I think it'll be a cool idea. However, the way they announced it to their customers lead to a pretty big negative reaction. Recently I recorded my thoughts (over a Call of Duty: Black Ops game play) on how they could have handled the announcement better, not abused their community managers in the process, and even whether or not they should build some of this stuff themselves (in particular in the social networking area of things).

If you're interested you can see the video on my YouTube channel or via the embedded player below: