There Must Be a Better Way

The Current State of Things

I'm in my third week of work at the new place. As with every place I've ever worked, they (we from here on out) have a lot of the same old problems. We have a shipping product, so we've got legacy code. Most legacy code I encounter anywhere is brittle, lacks unit tests, and has a severe lack of encapsulation / cohesion. The code hasn't rotted, peoples' view of the right way of doing things have evolved. When you look back at whatever you did 6 months to a year ago you're horrified. In addition the build, project organization, and installer are all screwed up. That too is typical. These things require more planning than they get on a normal project. They're the areas most likely to grow organically and as such they get messy and out of control very easily. Luckily with software everything if fixable.

The other process level problem we have is that we're dealing with a geographically separated development team. Although this goes against a major Agile rule, it seems to happen everywhere in the software world. We use IRC, revision control commit emails, daily status meetings, and announcement emails for major code changes to try and keep everyone in sync. Given that we're trying to fix all of the code level problems you can imagine that a lot of the system is changing very rapidly. The need for communication is even more heightened than normal.

Problems with IRC

At lunch on Friday with my co-workers we got into a discussion about the concerns about the effectiveness of our communication. A couple of people on the team (me among them) have made the statement that things aren't nearly as bad as other members seem to think. That is to say I have seen worse situations. We have a very good team of highly competent people that have an interest in improving things. The communication is certainly not the worst I have seen. That being said, I certainly think we can fix a few things.

I jumped into the lunch conversation saying that I thought both our email and IRC communication needed addressing. First, IRC is being misused. Someone said it is like hallway conversations for people that are off site. I like that analogy, but we're still missing slightly. We have too many inane, off topic conversations. I'm not interested in reading peoples' small talk. Second, we have too many one on one conversations taking place in the main channel. I think you can usually determine when you need to have a one on one conversation and take it to a private chat or phone call. I don't buy the argument that everyone needs to overhear everything in case they might have input. If these are hallway conversations, it's like every jackass has decided to have their meetings right outside my cubicle.

Another problem is that some people don't understand what is appropriate to communicate via IRC. Last week someone posted a link to a document they wanted the developers to review. Only one developer saw it. That shit needs to be sent in an email. And finally, any decision that gets made in IRC needs to be announced to a wider audience in a more visible way (most likely an email).

Problems with Email

One of the developers seemed to be getting a little irritated with me pointing out problems with our usage of IRC. Since I didn't pick up on it at the time I brought up why I thought our email communication sucked, too.

My problems with email are that it's a little to broadcasty, there's no archiving, no searching, it's non-threaded, and not good for soliciting responses. All of these problems have solutions within email, it's just that I feel that these solutions are tacked on after the fact and may still not be as effective as other solutions that were built with the problems in mind in the first place.

By broadcast, I mean both that I'm on email distribution lists that often have emails I don't particularly care about. Yes, we could make finer grained email distribution lists for a larger array of general subjects, but that gets messy especially since I can't currently control my membership on lists. This has to be done by someone with the correct permissions. Email rules to auto-delete are pretty imperfect as well.

As for archiving and searching, most people say I can archive my email locally and search on it whenever I feel like. This doesn't work unless I'm on my computer and doesn't work for any email sent before I started working here. Yes, you could tack on another product to automatically archive and index the email centrally and expose a web interface. This can even add some threading capabilities. This is one of the better solutions to a problem but I still find it lacking. Threading is typically done by parsing the subject line. Screw that up and you may not be in the right thread. Responding to old threads would require you paste the subject line from the web archive site and if you're really nice paste the body of one of the later posts. That would prevent people from having to run to the archive site to see what the hell you're talking about.

The problem of soliciting response from email is that you send out an email and end it with, "Is everyone good with that?" You almost never get a response. Maybe people don't respond if there's no problem. Maybe they didn't read it. Maybe they disagree but don't want to be the only one. Who knows? Sure you can put email receipt requests on your email, but the client on the other end has to support it and generally the receiver has to explicitly allow it. And so what if they read it? Do you want everyone to respond and say agree or disagree? That seems a little intrusive for most topics.

Bitcher!

At that point the slightly annoyed developer asked how I intended to solve these problems. I said I didn't think I had a solution for every one of these but that I thought there was a better way of doing things. For some of the problems I just don't know. She made the statement that she was tired of me bitching about the problem if I didn't have a solution.

I complain often, so I'm used to that response. People tend to feel that where there's smoke there some asshole standing next to a fire saying, "Someone should really do something about all this goddamn smoke." I feel that premature labelling of "bitcher" (since I've been there 3 whole weeks) is like me seeing dials that say something is going to blow up and being told, "Don't go bitching about the dials until you have a solution. Keep it to yourself." Of course, as always, the truth is likely somewhere in the middle.

My problem is that the first step to improving a situation is identifying the current shortcomings in your situation. Yes, something has to happen after that, but that doesn't invalidate the position of the person saying our status quo is inadequate. I'm sure as the other developer and I get more used to each other's quirks we'll be fine. I'm a non-complaining complainer while she appears to be very passionate and frustrated by institutional inertia that has prevented solutions from being implemented in the past.

Solutions

Since there seemed to be a real desire for me to put up or shut up, I detailed what I thought were some of the solutions. They're not perfect, but they were the best I could do at the time.

I like blogs as a developer diary. This is a place developers can go into more detail about their daily activities. The daily stand up should remain short and very high level. If you want more detail on what I am working on, check my internal blog. Blogs are also a good place to share quick tips on doing things (like how to use find, sed, tr, etc to perform a quick search and replace while copying the original file to another location). Blogs are also good for initial conversation starters and what-if scenarios. If an idea is worth discussing it moves off the blog and into threaded discussion forums.

Blogs aren't great for discussing things, but forums are pretty good at it. RSS feeds are a must so you get better notification of activity in the forums. You pay attention / subscribe to whatever you want. You can control the influx of information, unlike with email distribution lists. It's centralized, archived, searchable, and threaded. What's not to love? If you reach an actual decision move it to the wiki.

The wiki is not good for making announcements, having discussions, or keeping a developer diary. It is good for documenting things that have been decided. Yes, you can continue to tweak an idea, but major change discussions should happen in the forums with the results being captured in the wiki. I have my own problems with wikis that I'll detail in another post (they're very fixable). The wiki itself should have RSS feeds so you can easily see changes to the topics you care about.

If you need to solicit opinions, maybe you need to post a poll on the forums. I'm least happy with this solution, but I think it is a step in the right direction. Big decisions still should have a design review meeting I believe. Again, the designs should be captured somewhere, most likely the wiki.

Email is still used for quick announcements like "I modified the POM, so be sure to regenerate your Ecllipse projects" that don't really fit anywhere else. Email and shared calendar solutions are also great for scheduling meetings. Email is also used to communicate one on one changes like "Hey, I just checked in those changes you needed to complete your task for this iteration." I'm probably missing a lot of valid use cases for email so I'll just assume that email is the traditional catch all for things until we find / agree upon a better place.

There are other tools that I haven't mentioned but that are still needed. For example, you need a bug tracking system and an iteration planning tool.

My, Aren't We Tool Heavy?

I don't believe the tools are the complete solution. If your process is completely screwed up, the tools aren't going to fix it. I do think that the tools are a part of the solution though. For example, I don't like leaving the decision to document a change up to the developer making the change. They're too close to things. I like peer code reviews on check in with one of the items on the checklist being that an announcement needs to be made if the change is "big." A second set of eyes that didn't make the change helps greatly in making this decision. But, you still need a tool of some sort to communicate the change. Peoples' fear of having too many tools seems to make them force the one universally accepted tool (email) into roles it wasn't meant to fulfill. You wouldn't try to track bugs in email, would you? Maybe you shouldn't be using it for a lot of the other things for which you're currently using it…

As always, keep in mind that my ideas are a work in progress. Feel free to chime in if you have advice for better ways of doing things.

Share

5 Responses to “There Must Be a Better Way”

  1. Cote' Says:

    That's a good list. There are, also, our friends search, IM, and teleconferences. Screen-sharing (Windows Meeting, WebEx, whatever) is another good one.

    Whenever I think of process improvement, I just think about what people do on the public Internet, which matches well with what you have above.

    But, you know me: I'd foist everything on the Internet on the Intranet if I could…or just eliminate the firewall all together. Then you could use flickr, del.icio.us, and, best of all, meritocracy.

    On complaining, I was having an interesting discussion with a development executive the other day, and he remarked that developers are usually the most set in their ways of all software people. Reflecting on my own experience, this seems true: as Larry Wall pointed out long ago, developers are lazy. Once they figure out something that "works," there's reluctance to do otherwise.

    Of course, that's a generalization that probably applies to less than 40% of developers, but that's enough of a % to slow things down.

  2. Robert Says:

    Excellent points. And of course eliminating that firewall would allow my favorite: bloglines. It’s all well and good that most of the stuff we normally think of is finally getting an RSS feed, but it’s a shame not to be able to use bloglines (or whatever your favorite online aggregator is). Sticking more stuff like an aggregator in Outlook never felt ideal to me…

    I agree that good developers are usually pretty lazy (in a good way), but they also don’t like to be bothered. Broadcasting status via blogs and documenting procedure updates via wiki may be a good way to keep other people from getting up in your grill and killing your productivity. Forums also allow you to deal with the conversation when it’s convenient for you. Meetings have a habit of wandering and never producing anything very useful. Luckily, I think the people on my team are thinking the same way. We just need to get the process and the tools to solidify I think.

  3. Chris Says:

    We use the Wiki as a main documentation reference. We use an email list software, think listsoft (not what we use but an example), for email communication. Those messages are stored centrally, can be read and searched by anyone and they have persistance beyond a single developers lifetime.

  4. People Over Process Says:

    Perhaps Face-to-face Isn't So Valuable, or, Time-shifted Collaboration in Agile Software Development…

    One of Robert's recent posts on process and tools in software development got me to thinking about one of my eternally unanswered questions: if you're using even a fraction of the collaboration tools available "properly," do you really need to……

  5. Running as Root » Blog Archive » Improving Wiki Says:

    […] I've already gone over my desire to add more tools and procedures to our distributed team development process. My next issue is with one of the tools that I like quite a bit. I think that the way we're using Wiki is leaving something to be desired. […]

Leave a Reply