Falling Apart

I have been working on my current project for a long time. We are talking over 10 years now. So I have seen the project change a lot. I focus mainly on the development side of things. That is where I have the most insight.

We used to do a lot of good things in development. There were daily builds to make sure nobody broke anything. There were efforts to document the system in a formal way. We kept track of metrics on bugs to see how we were doing overall. Towards the middle of the project, we started writing white papers. We also attempted to make our documentation more accessible with a Wiki. We used our bug tracking tool to manage work.

Now most of that has been thrown out the window. Ad hoc processes, if you could call them that, are being used in place. Sometimes software gets reviewed. Sometimes it does not. There is some upfront documentation on new features. But that is about it. We no longer use a tool to manage work. We use email. Oh joy.

Of course we have a hit in quality. And what is more depressing, it takes a lot more effort just to move things along now. Where did we go wrong? Well different companies have taken over the project now and then. The management style is starkly different between these companies. And the results flow down hill to the rank and file.

Continuous

Oh boy. It has been a long time since I posted an entry to this blog. Well let's get right to it. I want to talk about continuous delivery. In the old days, our team would produce one major software release a year. Then it grew to 2. This year we did 3. I think you see the pattern.

Incremental builds were a way to integrate changes in the system faster. Less changes. So there is less to search through when something goes wrong. When taken to the extreme, this is continuous delivery. You do a release each time you do a build. Damn.

This level of delivery almost requires you to automate the heck out of everything. Problems get exposed right away. You need to be ready for this level of release. Let's get one thing straight. There is going to be pain. We are not ready for this on my project. There is pain all right. However continuous builds will make all the pain come to fruition at once. We can't handle that. Not yet at least.

Lessons from 37 Signals

37 Signals is a successful start up company. They have a number of rapidly developed small products which they sell. I have read a white paper from them on the lessons they have learned from their experience with software development.

They recommend you keep projects very short. We are talking about the order of 2 weeks or so. They advise against project plans which are set in stone. 37 Signals does not believe in design documentation or functional specifications. That seems like a radical idea. They also tend to focus on what is real as opposed to abstract theories.

Regarding content, 37 Signals likes to focus on things that are constant. They want to solve problems which will be encountered for the next 10 years. They also believe in sharing ideas with the public so that others may learn. Here is an idea which is not new. 37 Signals believes that you should “question everything”. I could use that advice myself. Having programmed for many years, I carry a lot of baggage in the form of untested assumptions.

37 Signals believes that there is a time when you should give up on a problem which is hard. They also back the idea of working 32 hours a week instead of 40. When you think about it, that is not so radical when compared to the Tim Ferris 4 hour work week. In terms of timing, 37 Signals thinks you should release the best half of your application first.

According to this company, client work is depressing. I second that motion. You can work hard to solve a specific problem for a client. However in the end, all you get to do is charge for those hours. You are not building a cash cow. I would like to end on a funny perspective from 37 Signals. They have no formal training program. Their method is to just drop people in the pipeline and hope they figure things out.

SCM

There is a book online by Eric Sink about source control. You would think this guy knew what he was talking about. He founded a company that produces source control software. I thought I would share some of the things I learned or recalled while reading his book.

The first order of business is defining source control. It is also called version control or SCM. This let’s developers work in parallel on the same code base. You must learn how to do this. You don’t get taught this at college. Some example of source control systems are SourceSafe, Subversion, CVS, and Perforce. The product created by the author Eric Sink is Vault.

Source code gets stored in a repository. You can get a copy of the code to view. You can also check out a copy of the code. The repository will automatically merge your changes in with changes from other developers. How is this repository maintained? It can store the whole copy of every version. However that is not very space efficient. Instead I believe most repositories only store the deltas between each version.

You normally have the ability to name a certain version of a file or your whole code base. This name is called the label. Another idea in source control is branching. This is where you have two or more copies of your code base at the same time. The good source control systems are integrated right into your IDE. I must confess that, even though my own source code tools integrate with my IDE, I do not use this functionality. I prefer to do things manually myself.

Complex Requirements

I recently checked out an article by Scott Ambler in Doctor Dobbs magazine. And I paid attention because his name sounded familiar from the past. I think he may have written about C++ in the past. From his bio, he is into process and agile methods. His article affirmed that requirements are complex, but applying Agile to them can help.

We do not want a heavy level of documentation for requirements. Take an example from Scrum. They have something called a backlog which prioritizes requirements. This is a good methodology to manage simple requirements.

You would think that complex requirements could be broken down. The problem is that if you do break down the requirements, the smaller pieces might not make sense by themselves without the context. You need to try to group the smaller sets of requirements in a smart fashion for this to work.

Just like traditional object oriented analysis and design, you should break the requirements into use cases and scenarios. The use cases should be simple. There should also be very little documentation.

Agile Evangelism

I read an editorial lately about the relentless evangelism by the Agile tribe. Their attack on traditional waterfall methods is vicious. Sure many software project fails. But don't Agile projects fail as well? I doubt anybody is keeping track of this. I put a picture of Wendy Friedlander up for your amusement. She is one of the agile evangelists out there.

Software projects worked before Agile came around. and they will continue to work after Agile has come and gone. So what is there to truly tie any success to the Agile methodology? As you can tell, I have a lot more questions than answers here.

I am big on unit testing myself. That does not mean I advocate Agile methods. Unit testing itself is not a new technique created by the Agile movement. I will say that I like some ideas behind Test Driven Development (TDD).

One of the blogs I faithfully read is Joel on Software (Jos). Most of the good stuff comes from comments posted by JoS readers. It is funny that Joel has blasted Agile for the main part. He says that you cannot accomplish much when you are writing endless unit tests. And if you don't know Joel, he is big on getting things done.

Life at Google

One of my favorite bloggers is Steve Yegge. He initially worked at Amazon. Now he is with Google. A few years back he wrote a post bashing Agile. That was not the real story. His advice for developers is to work for a company like Google instead of using Agile methods. This was a firsthand look into the life of Google developers. Yes I have heard about the 20 percent side project work at Google. But Steve really gets down to the details of what it is like to work there. With the falling stock price, Google may not be the optimal location for the best developers now. But it is still interesting to take a look at a day in the life of the Google developer.

There are managers at Google. However they all write code. Furthermore they do not tell developers what to work on. Developers are self organizing. There are incentives to be part of a successful project. You have the potential to make giant bonuses and get stock options galore. This often translates into developers who feel like they owe Google. However there is also a drive to gain the respect of your peers at Google. Your peers also happen to be very smart if you work at Google.

Google has some environmental differences than your average development shop. It is very quiet there. You get free meals. Meetings are scheduled for the middle of the day so you don’t have to come in too early. It is kind of like a start up mentality. Google offers a lot of training. Some of it is formal. Most of it is not. There are apparently technical presentations going on all the time at Google.

On the development side, Google takes software engineering very seriously. There is a lot of emphasis placed on unit testing, design documentation, and code reviews. Some people just build developer tools at Google.

As usual, Steve generated a lot of comments from readers. Many people state that Google is unique in that it does not have paying customers who need software developed and shipped. Also Google has a lot of smart people. Google projects are financial failures. They just get a lot of money from ads in search. Google has deep pockets. If the Google model is so good, why does not everybody use it?