Code complete != ship ready


I used to work for a team where whenever an engineer said he was done, the next question would invariably be are you ‘done done‘?

Do you find yourself always under-estimating the amount of time needed to complete some software project? Does it take more time than what you planned? Or do you say you are done but need to change just one small feature.

Accurate estimation is difficult but again an unclear definition of what done means also contributes to this. So what does done mean?

Checklists and Processes

Boeing introduced preflight checklists in 1935 after a prototype crash killed two pilots. Investigations revealed that they had forgotten to carry out a critical step before takeoff. Thus, the fix was to introduce a preflight checklist to avoid this (they learnt from the mistake and became better for it).

A catastrophic software failure can have disastrous outcomes – it is expensive, can harm (imagine if embedded car software were to fail), and is bad PR for the company (how many security breaches have you heard of in recent times?).

So, do we need such heavy formal checklists for software development? Some companies require standardized processes while others don’t. However, it is always good to have a mental model of the various stages of the software shipping pipeline.

Here is a checklist to guide you through the various phases of done; you can now know which stage you are at any given point in time.

1. Code complete

You have written the code (and hopefully added unit tests) but have not done any deep testing. This is what most developers erroneously refer to as done. No, code complete does not mean done. It is the first milestone in a long journey.

Always have tests for code completion

2. Test complete

Thorough exploratory testing of the feature has been carried out. This covers extreme conditions, edge cases and load/performance scenarios.

It’s an exercise in futility to attempt to find all bugs and exercise all possible code paths. Rather, the goal of testing is to reduce the chances of a user finding a glaring simple-to-discover error that should have been found earlier. It also surfaces other issues (e.g usability, security holes etc.) that might have been latent.

Every discovered bug should ideally be fixed and a unit test added to prevent future regressions. However, not all bugs are fixable (and those turn into features…). Early knowledge of these issues enable developers to proactively prepare mitigation plans, workarounds and also notify users.

If you don’t have dedicated testers (they are worth their weight in gold and very good); you  can use James Whittaker’s testing tours method to get some fun while doing this and also ensure a good coverage.

3. Integration complete

You have completed the various isolated features, written unit tests and also done exploratory testing. You have a high confidence in each of the individual features however your components work exactly like the similitude below:

So get to it and do integration of all the disparate components and ensure that things still work as they should. More testing, more discussions with collaborating software engineers and maybe some more glue code.

Why do I keep emphasizing on testing? Have you ever had a bad experience with some software? It’s nearly always a quality issue.

4. Ship ready

This is the cross your t’s and dot your i’s moment; verify that the code solves the problem, issues discovered during testing have been fixed and components have been integrated. Code complete, test complete and integration complete? Great, time to ship.

Some organizations enforce strict limits on some parameters like performance, security, reliability and compliance. If so, you should do another pass to ensure that your software meets these criteria.

The product owner / customer likes the software and viola! Your software piece rolls out like a new car off the production line.

5. Shipped (maintenance ready)

Software is never done. When it is shipped, customers would start using it in ways you never imagined and will find issues. Some can be petty – for example, can you change move this button by 0.2 pixels? While others would be pretty serious stuff.

In maintenance mode, the software already exists but there are new bugs to fix, new feature extensions to make and behavioural modifications to make.

And when you think it’s all done, then maybe it’s time to release a new version. Start at number 1 again.

Enjoyed this post? Follow me on Twitter or subscribe for weekly posts. No spam for sure.

Related Posts

  1. The Myth of Perfect Software
  2. The Effective Programmer – 3 tips to maximize impact
Advertisements

How function spies work in JavaScript


If you write unit tests, then you likely use a testing framework and might have come across spies. If you don’t write unit tests, please take a quick pause and promise yourself to always write tests.

Testing framework suggestions? Try Sinon or Jasmine.

Spies allow you to monitor a function; they expose options to track invocation counts, arguments and return values. This enables you to write tests to verify function behaviour.

They can even help you mock out unneeded functions. For example, dummy spies can be used to swap out AJAX calls with preset promise values.

The code below shows how to spy on the bar method of object foo.

spyOn(foo, 'bar');
foo.bar(1);

expect(foo.bar).toHaveBeenCalled();
expect(foo.bar).toHaveBeenCalledWith(1);

Jump into the documentation for more examples.

That was pretty cool right. So how difficult can it be to write a spy and what happens under the hood?  It turns out implementing a spy is very easy in JavaScript. So let’s write ours!

The goal of the spy is to intercept calls to a specified function. A possible approach is to replace the original function with another function that stores necessary information and then invokes the original method. Partial application makes this quite easy…

The Code

function Spy(obj, method) {
    let spy = {
        args: []
    };

    let original = obj[method];
    obj[method] = function() {
        let args = [].slice.apply(arguments);
        spy.count++;
        spy.args.push(args);
        return original.call(obj, args);
    };

    return Object.freeze(spy);
}

let sample = {
    fn: function(args){
        console.log(args);
    }
};

let spy = Spy(sample, 'fn');
sample.fn(1,2,3);
console.log(spy.args.length); //1
console.log(spy.args); //[[1,2,3]]

sample.fn('The second call');
console.log(spy.args.length); //2
console.log(spy.args); //[[1,2,3], 'The second call']

//try modifying the spy
spy.args = [];
console.log(spy.args); //[[1,2,3], 'The second call']

Taking the code apart

The spy method takes an object and a method to be spied upon. Next, it creates an object containing the call count and an array tracking invocation arguments.

It swaps out the original call with a new function that always updates the information object whenever the original method is invoked.

The Object.freeze call ‘freezes’ the spy object and prevents any modifications of values. This is necessary to prevent arbitrary changes of the spied values.

Limitations

The toy sample is brittle (yes I know it). Can you spot the issues? Here are some:

  • What happens if the method doesn’t exist on the object?
  • What happens if the object is null?
  • Can it work for non-object methods? Would pure functions work? Would using window as the parent object work?
  • What happens if method is a primitive and not a function?

These can (and should) be fixed but again, that would make this post very complicated. The goal was to show a simple toy implementation.

Challenges

How do you ‘unregister’  spies without losing the original method? Hint: store it in some closure and replace once you expose an unregister call.

How would you implement a spy in Java?

Related

Spying Constructors in JavaScript