Creating Great User Experiences


Have you ever wondered why some applications always look and feel similar? Why for example does Apple have a unified experience across devices? Why are Google products starting to adopt the material experience?

This post explains some of the underlying themes influencing design choices. Developers can use these concepts to craft better user interfaces and experiences.

1. Consistency

A well designed user experience is consistent through out its constituent parts. Imagine how easy it would be to use an app with varying colour shades, item sizes and page styles? Wouldn’t that be confusing and difficult to understand?

A couple of examples of design consistency include:

  1. All computer applications usually have the set of three buttons on the title bar (close, minimize and maximize).
  2. Right-click nearly always opens up a context menu
  3. The menu bar for applications starts with drop downs titled File, Edit and so on

Such consistency lowers the learning barrier as they are implicit assumptions in the minds of users. When was the last time you tried to figure out the ‘close’ button? You don’t need to think, you subconsciously understand and use them.

UI frameworks like semantic-ui, bootstrap or Fabric provide a good foundation for quickly ramping up web applications. A colour palette helps ensure visual colour consistency. Do be careful of contrast though; having too much contrast is not visually appealing while having too little makes it difficult to see components.

And this leads us to the next point : ‘learnability’.

2. Learnability

One of the enduring messages I picked up from Steve Krug’s great book: ‘Don’t make me think’ is to avoid making users think. Why? When people have to ‘think’ deeply to use your application, that interrupts their concentration and flow; such interrupts might be fatal as some users might just leave your site.

A low learning curve helps users quickly become familiar and deeply understand how to use your product. Think of any software you are proficient in, now think about the first time you used that product, was it difficult? Did you grow to learn?

A couple of tips here:

  1. Component Behaviour – One common issue I run into with components is variable behaviour for the same action in different scenarios. For example, a set of actions might trigger a dialog to pop up while in other scenarios the dialog is not triggered. It’s much better to have the dialog pop up consistently otherwise you’ll have your users build many mental models of expected interaction responses.
  2. Grouping Related collections can be grouped and arranged in hierarchies that can be drilled into. I prefer using cards for such collections and allowing users drill up/down. Another thing to note is to ensure that the drill ladders are consistent; no point having users drill to a level they can’t drill out of.

3. Clickability

My principle of ‘clickability’

The number of clicks required to carry out a critical operation is inversely proportional to the usability scores for the web application.

The larger the number of clicks needed to achieve a major action, the more annoying the app gets over time. Minimize clicks as much as possible!

But why is this important? Let’s take a hypothetical example of two applications – app 1 and app 2 – which require 2 and 4 clicks to register a new user.

Assuming you need to add 5 new users, then app 1 requires 20 clicks while app 2 requires 40 clicks, a difference of 20 clicks! It’s safe to assume then that users who have to use the application for hours on end will prefer app1 to app2.

The heuristic I use is to minimize the number of clicks in core user paths and then allow short cuts for cases that are not possible.

Yes, ‘clickability’ does matter.

4. Navigability

Building upon the principle of clickability, this implies designing your flow so that the major content sections are well interlinked. It also includes thinking of the various links between independent modules so that navigation is seamless.

For example, you have a navigation hierarchy that is 3 layers deep and want users to do some higher-level action while at the bottom of the navigation tree. You can add links to the leaf nodes and even subsequent tree levels to ensure users can quick jump out rather than exposing a top-level option only. Why? See the principle of clickability above.

A good exercise is to build out the interaction tree which exposes the links between sections of your app. If you have a single leaf node somewhere, then something is really wrong. Once done, find ways to link leaf nodes if such interactions are vital and enhance the user experience.

5. Accessible + Responsive + nit-proof

One of my favorite tests is to see how a web application works on smaller devices.That quickly exposes a lot of responsive issues.

Another one is to add way too many text characters to see if text overflow is properly handled. The fix for that is very simple: just add the two CSS rules

.noOverflow {
   text-overflow: ellipsis;
   overflow: hidden;
}

Another important concept is adding titles to elements and following accessibility guidelines.

All these are important and show that some thought was put into the process.

Conclusion

Do a quick test of your application on some random user.The way users interact with your application and navigate around might surprise you!

While running such tests, ask if users feel they have an intuitive understanding of how the app works. If yes, bravo! Pat your self on the back. Otherwise, your assumptions might be way off and you need to return to the drawing board.

The best experiences always look deceptively simple but making the complex simple is a very difficult challenge. What are your experiences creating great user experiences?

Related

  1. Things to check before releasing your web application
  2. Tips for printing from web applications
  3. How to detect page visibility in web applications
  4. How to track errors in JavaScript Web applications

Things to check before releasing your web application


This post originally started out as a list of tips on how to break web applications but quickly morphed into a pre-release checklist.

So here are a couple of things to validate before you press the ‘go-live’ button on that wonderful web application of yours.

General

  1. Does the application handle extremely large input? Try copying a Wikipedia page into an input field. Strings can be too long and overflow database models.
  2. Does it handle boundary values properly? Try extremely large or small values; Infinity is a good one.
  3. Do you have validation? Try submitting forms with no entry.
  4. Do you validate mismatched value types? Try submitting strings where numbers are expected.
  5. Has all web copy been proofread and spell-checked? Typos are bad for reputation.

Localization (L10n) and Internationalization (I18n)

  1. Do you support Unicode? The Turkish i and German ß are two quick tests.
  2. Do you support right-to-left languages? CssJanus is a great tool for flipping pages.
  3. Time zones and daylight saving time changes.
  4. Time formats: 12 and 24 hour clocks
  5. Date formats: mm/dd/yyy vs dd/mm/yyyy
  6. Currencies in different locales.

Connections

  1. Does your web app work well on slow connections? You can use Chrome or Fiddler to simulate this.
  2. What happens when abrupt network disconnections occur while using your web application?
  3. Do you cut off expensive operations when the user navigates away or page is idle?

Usability + UX

  1. Does the application work well across the major browsers you support (including mobile)?
  2. Does the application look good at various resolution levels? Try resizing the window and see what happens.
  3. Is your application learnable? Are actions and flows consistent through the application? For example, modal dialogs should have the same layout regardless of the action triggering them.
  4. Do you have your own custom 404 page?
  5. Do you support print?
  6. Do error messages provide enough guidance to users?
  7. Does your application degrade gracefully when JavaScript is disabled?
  8. Are all links valid?

Security

  1. Do you validate all input?
  2. Are all assets secured and locked down?
  3. Do you grant least permissions for actions?
  4. Ensure error messages do not reveal sensitive server information.
  5. Have you stripped response headers of infrastructure-revealing information? E.g. server type, version etc.
  6. Do you have the latest patches installed on your servers and have a plan for regular updates?
  7. Do you have a Business Continuity / Disaster Response (BCDR) plan in place?
  8. Are you protected against the Owasp Top Ten?
  9. Do you have throttling and rate limiting mechanisms?
  10. Do you have a way to quickly rotate secrets?
  11. Have you scanned your code to ensure no valuable information is being released?

Code

  1. Did you lint your CSS and JS (see JSLint, JSHint, TSLint)?
  2. Have all assets (JavaScript, CSS etc) been minified, obfuscated and bundled?
  3. Do you have unit, integration and functional tests?

Performance

  1. Have you run Google’s Page Speed and Yahoo’s YSlow to identify issues?
  2. Are images optimized? Are you using sprites?
  3. Do you use a CDN for your static assets?
  4. Do you have a favicon? Helps to prevent unwanted 404s since browsers auto-request for them.
  5. Are you gzipping content?
  6. Do you have stylesheets at the top and JavaScript at the bottom?
  7. Have you considered moving to HTTP2?

Release Pipeline

  1. Do you have test and staging environments?
  2. Do you have automated release pipelines?
  3. Can you roll back changes?

Others

  1. Do you have a way to track errors and monitor this with logging?
  2. Do you have a plan to handle customer reported issues?
  3. Have you met all legal and compliance requirements for your domain?
  4. Have you handled SEO requirements?

Conclusion

These are just a few off of my head – feel free to suggest things I missed out. I should probably consider transferring these to a Github repo or something for easier usage.

Book Review:Build your own AngularJS


As part of my continuous learning; I started reading Tero Parviainen‘s ‘Build your own AngularJS‘ about 6 months ago. After 6 months and 127 commits, I am grateful I completed the book.

While I didn’t take notes while reading, some ideas stood out. Thus, this post describes some of the concepts I have picked up from the book.

The Good

1. Get the foundational concepts right

This appears to be a recurring theme as I learn more about software engineering. Just as I discovered while reading the SICP classic, nailing the right abstractions for the building bricks makes software easy to build and extend.

Angular has support for transclusion which allows directives to do whatever they want with some piece of DOM structure. A tricky concept but very powerful since it allows you to clone and manage the scope in transcluded content.

There is also support for element transclusion. Unlike the regular transclude which will include some DOM structure in some new location; element transclusion provides control over the element itself.

So why is this important? Imagine you can add this to some element to only show up under certain conditions? Then you can use element transclusion to ensure that the DOM structure is only created and linked when you need it. Need some DOM content to be repeated times? Just use element transclusion, clone and append it the times. These two examples are over-simplifications of ng-if and ng-repeat respectively.

Such great fundamentals allow engineers to build complex things from simple pieces – the whole is greater than the sum of parts.

2. Test Driven Development (TDD) works great

This was my first project built from the scratch using  TDD and it was a pleasant experience.

The array of about 863 tests helped identify critical regressions very early. It gave me the freedom to rewrite sections whenever I disagreed with the style. And since the tests were always running (and very fast too, thanks Karma!); the feedback was immediate. Broken tests meant my ‘refactoring’ was actually a bug injection. I don’t even want to imagine what would have happened if those tests didn’t exist.

Guided by the book – a testament to Tero’s excellent work and commitment to detail – it was possible to build up the various components independently. The full integration only happened in the last chapter (for me, about 6 months later). And it ran beautifully on the first attempt! Well, all the tests were passing…

3. Easy to configure, easy to extend

This is a big lesson for me and something I’d like to replicate in more of my projects: software should be easy to configure and extend.

The Angular team put a lot of thought into making the framework easy to configure and extend. There are reasonable defaults for people who just want to use it out of the box but as expected, there would be people who want a bit more power and they can get desires met too.

  • The default digest cycle’s repeat count of 10 can be changed
  • The interpolation service allows you to change the expression symbols from their default {{ and }}
  • Interceptors and transform hooks exist in the http module
  • Lots of hooks for directives and components

4. Simplified tooling

I have used grunt and gulp extensively in the past however the book used npm in conjunction with browserify. The delivery pipeline was ultimately simpler and easier to manage.

If tools are complex, then when things go wrong (bound to happen on any reasonably large project), you’d have to spend a lot of time debugging or trying to figure out what went wrong.

And yes, npm is powerful enough.

5. Engineering tricks, styles and a deeper knowledge of Angular

Recursion

The compile file which would allow two functions to pass references to each other – an elegant way to handle state handovers while also allowing for recursive loops.

Functions to the extreme

  1. As reference values: The other insightful trick was using function objects to ensure reference value integrity. Create a function to use as the reference.
  2. As dictionaries: functions are objects after all and while it is unusual to use them as objects, there is nothing saying you can’t.

function a() {};

a.extraInfo = "extra"

Angular

Most of the component hooks will work for directives as well – in reality, components are just a special class of directives. So you can use the $onInit, $onDestroy and so on hooks. And that might even lead to better performance.

Issues

Tero did an awesome job writing the book – it is over a 1000 pages long! He really is a pro and knows Angular deeply; by the way, you should check out his blog for awesome deep dives.

My only issues had to do with issue resolution; there were a few issues with outdated dependencies but nothing too difficult. If he writes an Angular 2 book, I’d like to take a peek too.

Conclusion

I took a peek at the official AngularJS repository and was quite surprised by how familiar the structure was and how it was easy to follow along based on the concepts explained in the book.

I’ll rate the book about 3.9 / 5.0. A good read if you have the time, patience and curiosity to dive deep into the  Angular 1 framework. Alas Angular has moved on to 2 but Angular 1 is still around. Moreover, learning how software is built is a great exercise always.

How to detect page visibility in web applications


You are building a web application and need the application to pause whenever the user stops interacting with the page; for example, the user opens up another browser tab or minimizes the browser itself. Example scenarios include games where you want to automatically pause the action or video/chat applications where you’d like to raise a notification.

The main advantage of such an API is to prevent resource wastage (battery life on mobile, internet bandwidth or unnecessary computing tasks). Definitely, something to have in mind especially for developers targeting mobile devices. So how would you this?

Can I use event listeners?

Technically, you could use a global event listener on the window object to listen for focus/blur events however, this can not detect browser minification. Also, the blur/focus event would be fired whenever the page loses focus; however, it is possible that a webpage is still visible despite losing focus – think about users having multiple monitors.

The good news is that this is possible with the PageVisibilityAPI which comes with the browsers and this post shows how to use this.

Deep dive into details

The Document interface has been extended with two more attributes – visibilityState and hidden.

Hidden

This is true whenever the page is not visible. What counts as being not visible includes lock screens, minimization, being in a background tab etc.

VisibilityState

This can be one of 4 possible enums explaining the visibility state of the page.

  • hidden: page is hidden, hidden is true
  • visible: page is visible, hidden is false
  • prerender: page is being pre-rendered and not visible. Support for this is optional across browsers and not enforced
  • unloaded: page is being unloaded; hidden would also be false too. Support for this is also optional across browsers

Show me some code!

document.addEventListener('visibilitychange',function(){
    if(document.hidden) {
        console.log('hidden');
    } else {
        console.log('visible');
    }
}, false);

Browser support

You can have it in nearly all modern browsers except Opera mini. Also, you might need to specify vendor prefixes for some of the other browsers. See this.

Conclusion

There it is; you now know a way to effectively manage resource consumption – be it battery, internet data or computing power.

You can use this to determine how long users spend on your page, automatically pause streaming video/audio (with some nice fadeout effects for audio especially) or even raise notifications.

Did you enjoy this post? Here are a few more related posts:

How to track errors in JavaScript Web applications


Your wonderful one-of-a-kind web application just had a successful launch and your user base is rapidly growing. To keep your customers satisfied, you have to know what issues they face and address those as fast as possible.

One way to do that could be being reactive and waiting for them to call in – however, most customers won’t do this; they might just stop using your app. On the flip side, you could be proactive and log errors as soon as they occur in the browser to help roll out fixes.

But first, what error kinds exist in the browser?

Errors

There are two kinds of errors in JavaScript: runtime errors which have the window object as their target and then resource errors which have the source element as the target.

Since errors are events, you can catch them by using the addEventListener methods with the appropriate target (window or sourceElement). The WHATWG standard also provides onerror methods for both cases that you can use to grab errors.

Detecting Errors

One of JavaScript’s strengths (and also a source of much trouble too) is its flexibility. In this case, it’s possible to write wrappers around the default onerror handlers or even override them to instrument error logging automation.

Thus, these can serve as entry points for logging to external monitors or even sending messages to other application handlers.

//logger is error logger
var original = window.onerror; //if you still need a handle to this
window.onerror = function(message,source,lineNo,columnNo,errObject){
    logger.log('error', {
        message: message,
        stack: errObject && errObject.stack
    });
    original() //if you want to log the original
    return;
}

var elemOriginal = element.onerror;
element.onerror = function(event) {
    logger.log('error', {
        message: event.message,
        stack: event.error.stack
    });
    elemOriginal();
    return;
}

The Error Object

The interface for this contains the error message and optional values: fileName and lineNumber. However, the most important part of this is the stack which provides information about the stack.

Note: Stack traces vary from browser to browser as there exists no formatting standard.

Browser compatibility woes

Nope, you ain’t getting away from this one.

Not all browsers pass in the errorObject (the 5th parameter) to the window.onerror function. Arguably, this is the most important parameter since it provides the most information.

Currently the only big 5 browser that doesn’t pass in this parameter is the Edge browser – cue the only ‘edge’ case. Safari finally added support in June.

The good news though is there is a workaround! Hurray! Let’s go get our stack again.

window.addEventListener('error', function(errorEvent) {
    logger.log('error', {
        message: event.message,
        stack: event.error.stack
    });
});

And that wraps it up! You now have a way to track errors when they happen in production. Happier customers, happier you.

Note: The eventListener and window.onError approaches might need to be used in tandem to ensure enough coverage across browsers. Also the error events caught in the listener will still propagate to the onError handler so you might want to filter out duplicated events or cancel the default handlers.

Related

Tips for printing from web applications

Liked this article? Please share, subscribe or drop a comment.

Maturing as a software engineer


Looking back on my time as a developer, there are a lot of things I would have avoided doing if I had as much knowledge and maturity as I did now.

While I am grateful for the experiences and don’t regret them; I felt it would be a good idea to share these. These might motivate others or at least speed up their careers.

Here goes!

1. Patterns, patterns, patterns

When I take part in code reviews, I tend to look for recurring style patterns. Why? This helps to reduce the cognitive load on readers of the code (after all, code is written to be read).

I am  not advocating for bad software patterns rather having a plethora of ways for doing the same thing in a codebase creates confusion and productivity losses. How do you determine the ‘right’ pattern?

For example in JavaScript, there are several ways for creating an array.


var a = [];

var a = new Array();

var a = new Array(3);

Having a haphazard mixture only takes away brain processing cycles. Rather, have your team decide on a style and stick to it.

By the way, the first style is the ‘expected’ and preferred approach although there might be use cases for the latter two.

Ever wonder why the Google codebase is rated to be easy to work with? Well, think about consistency and established patterns.

2. Break the big picture down and make incremental progress

Building and distributing the smallest software piece you can imagine requires more effort than you would think. It is much more efficient to break down the big picture into small chunks of work that can be completed in an hour or less. Such breakdowns make you more effective and help in understanding progress and forecasting completion times (which is a tricky problem to solve).

I used to break down only the code pieces before (which itself was an improvement over my earlier dive-into-code-and-figure-it-out-as-you-go approach). Nowadays, I try to take some time and reflect on the end product itself: its behaviour, look and feel and how users would interact with it.

For a typical software project, such road maps covers:

  • Testing – unit tests, continuous integration,
  • Documentation – extensibility guides, tooling
  • Implementation
  • Discoverability and Distribution – release targets, getting started articles
  • Maintenance – handling bugs, user feedback etc

Sounds like too much work? Well, just focus on one small bit at a time and keep making progress.

3. Be lazy – start first on tasks with the largest impact/effort ratios

Two things matter: results and impact. There is no point in slaving for 20 hours to choose between blue and light blue if it has no impact on the users. Ditto for spending endless hours ‘arguing’ over what language should be used. Just choose the best usable one and deliver results.

My heuristic for tasks is thus:

  • Does completing the task move me closer towards the big picture?
  • Is this the easiest-to-achieve task with the biggest impact?

If so, I pick up that task and just do it – the goal is to maximize the impact/effort ratio.

Before I’d just stick to a task and spend endless hours on it even if it was something as trivial (and probably low-impact) as beautifying test scaffolding test output and elaborately designing test functions. Now? Common, my time is more valuable than that – I get the test functions right and try to get the coverage I want but won’t spend too much time once that is achieved and is readable for others.

Excessive polishing time can be spent on other more impactful pursuits like having fun with family or delivering high impact features.

4. Technical skills plateau

Sooner or later, you’ll get to the technical plateau. By that time, you’ll have so many successes under your belt and can detect potential pitfalls easily. Then, what next?

There are tons of ways to extend your impact and that is the way people become even better engineers. For example, I doubt if Anders Helsberg is still writing a lot of code, yet his ideas continue to empower and influence millions around the world.

Think about that, how do you scale your influence and make it possible to touch the lives of thousands of people? Are there engineering problems crippling your organization? Process pitfalls to improve with huge impact? Education ramp ups? There are always challenges to solve and problems to fix.

5. Choose career investments carefully

How would you set up an investment portfolio? Would you just go about investing in everything? Nope, you would evaluate the risks and benefits, consult experts and then invest in a select few areas while ignoring other areas.

You could spread out your risk by investing in a wide area but doing this excessively dilutes your returns. Conversely, investing in only company could be very risky too. Thus, it’s generally advised to spread  out your investment portfolio

Careers are investment portfolios. A typical career spans a long period ( upwards of 30 to 40 years) and shares some similarities with investments:

  • technologies, frameworks etc -> investment options
  • time -> funds

Just as you wouldn’t jump on every new fund, why would you do the same with your career? There is no harm in taking measured risks in careers but you should be strategic and know what your end goal is.

Every now and then a new framework pops up in the news. Before, I’d hop on the bandwagon and try figure it out. Nowadays? Well, if it really piques my interest, then I might spend some time learning about its core design principles and problem-solving approach.

If it neither solves any of my problems nor brings anything new to the table, then no thank you; I’d rather continue nurturing my current investment portfolio and hedging my bets.

Think about your bets and stick to them.

Conclusion

I am still learning and pray I continue. One thing that has struck me as being really critical is the will to try. We don’t know if something would work out or not however we can always try and then learn from the outcome (success or failure).

Don’t give up – continue learning and growing.

Understanding Bit masks


Bit masks enable the simultaneous storage and retrieval of multiple values using one variable. This is done by using flags with special properties (numbers that are the powers of 2). It becomes trivial to symbolize membership by checking if the bit at a position is 1 or 0.

How it works

Masking employs the bitwise OR and AND operations to encode and decode values respectively.

New composite values are created by a bitwise OR of the original composite and the predefined flag. Similarly, the bitwise AND of a composite and a particular flag validates the presence / absence of the flag.

Assuming we start off with decimal values 1,2,4 and 8. The table below shows the corresponding binary values.

Decimal Binary
0 0000
1 0001
2 0010
4 0100
8 1000

The nice thing about this table is that the four bits allow you to represent all numbers in the range 0 – 15 via combinations. For example, 5 which is binary 101, can be derived in two ways.

5     -> 1 + 4

or

101 -> 0001 | 0100

       -> 0101

7 which is 111 can also be derived in the same form.

Since any number in a range can be specified using these few ‘base’ numbers, we can use such a set to model things in the real world. For example, let’s say we want to model user permissions for an application.

Assuming the base permissions are read, write and execute, we can map these values to the base numbers to derive the table below:

Permissions Decimal Binary
None 0 000
Read 1 001
Write 2 010
Execute 4 100

Users of the application will have various permissions assigned to them (think ACL). Thus a potential model for visitor, reader, writer and admin roles with our base permissions is:

Role Permissions Decimal Binary
Visitors None 0 000
Readers Read 1 001
Writers Read + Write 3 011
Admins Read + Write + Execute 7 111

Noticed a pattern yet?

All the binary values can be obtained by ‘OR-ing’ the base binary values. For example, admins who have read, write and execute permissions have the value obtained when you do a bitwise OR of 1, 2 and 4.

The UNIX model uses the same numbering system. E.g. 777 translates into 111 111 111 which grants owners, groups and others read, write and execute permissions.

Checking Access

Now, the next question is how do you check if a composite value contains a particular flag? Going back to the binary data basics, this means checking if a bit at some position is a 1 or 0.

The bitwise AND operator comes in handy here – it guarantees a 1 when both values sharing the same index are 1 and 0 in all other cases. Thus, ‘AND-ing’ a composite value and the desired flag would reveal the desired outcome. If we get a value greater than zero as the result, then the user has the permission, otherwise a zero means there is no permission.

The admin role has a bitmask value of 111. To check if he really ‘has’ the execute permission we do a bitwise AND of 111 and the execute flag 100. The result is 100 which proves the permission.

More tables! The two tables show how to check for 2 users; one with the read + write + execute (111) permissions and another with the read and execute (101) permissions.

Read + Write + Execute (111)

Permissions 111 bitmask Binary Has permission?
Read 111 001 Yes: 111 & 001 → 1
Write 111 010 Yes: 111 & 010 → 1
Execute 111 100 Yes: 111 & 100 → 1

Read + Execute (101)

Permissions 101 bitmask Binary Has permission?
Read 101 001 Yes: 101 & 001 → 1
Write 101 010 No: 101 & 010 → 0
Execute 101 100 Yes: 101 & 100 → 1

See? Such a simple way to check without having to make unnecessary calls to the server.

Usage

This post shows the permission model however bit masks can be applied a wide variety of scenarios. Possible examples include checking membership, verifying characteristics and representing hierarchies.

A possible use in games could be to verify the various power-ups an actor has and to add new ones rapidly. Eliminates the need to iterate, use the bitmask.

Hierarchical models, where higher members encompass lower ones (e.g. the set of numbers) can also be adequately modeled using bitmaps.

Language support

Explicit language support is not needed to use bitmasks, the rules to know are:

  • Use increasing powers of 2 – ensures there is only one flag at the right spot
  • Create basic building blocks that are easy to extend and combine
  • Remember to watch out for overflows and use values that are large enough to hold all possible bit values. For example, C’s uint_8 / unsigned char can only hold 8 different flags, if you need more, you’d have to use a bigger value.

Some languages provide extra support for bit mask operations. For, example, C# provides the enum FlagsAtribute which implies that the enum would be used for bit masking operations.

Teaser

Q: Why not base 10? After all, we could use 10, 100, 1000 etc.

A: The decimal system falls short because you can have ten different numbers in one position. This makes it difficult to represent the binary ON/OFF state (which maps well to 0/1).

 

Safely handling destructive loops


Collections abound everywhere in the real world and loops are very common in programming. A simple loop is shown below:

var arr = [1,2,3,4];

for(var i = 0, len = arr.length; i < len; i++) {
    console.log(arr[i]);
}

//Output
1,2,3,4

Destructive loops

Simple loops can be boring, after all, why not have the collection change during a looping operation for more fun? Won’t happen? Believe me it does, have run into such at least twice.

An example scenario involves collections of listeners (say for example in a pub-sub implementation). Ideally, you want to go through all the listeners and invoke them. However it is possible a listener can either directly or indirectly remove itself while the loop is processing. For example, it might call a destroy function on some object which in turn leads to a unsubscribe call.

When this happens, it’s all chaos. Funny things happen or even worse – bugs.

Suddenly, the loop invariants are all wrong and you get funny results and/or errors (most likely errors). Let’s take an example:

function createListenerArray(arr) {
   for(var i = 0; i < 4; i++) {
       arr.push(function(i) {
           console.log(i);
           if(i === 2) {
               arr.splice(i, 1);
           }
       });
   }

   return arr;
}

The createListener function pushes 4 functions onto the array reference it gets. One of these functions (the third one) will remove itself from the array once invoked. To make tracking things easier, each function will log its index to the console.

var arr = [];
createListenerArray(arr);
for(var i = 0, len = arr.length; i < len; i++) {
    arr[i](i);
}

0
1
2
//TypeError: arr[i] is not a function

Oops! Blows up…

The loop was proceeding smoothly until it got to the function at index 2. Invoking that function changed the array by removing the corresponding entry from the array. Thus the valid length of the array decreased from 4 to 3.

The next step of the loop tried to access the element at index 3; however that index is now an invalid position since the element at index 3 got moved to index 2 when the array shrunk. arr[3] is undefined and invoking undefined as a function leads to the TypeError.

Imagine the ripple effect this can cause in larger arrays

Let’s try forEach.

var arr = [];
createListenerArray(arr);

arr.forEach(function(val, i){
    val(i);
});

0
1
2
undefined

This is not the desired result but on the bright side, at least there were no Exceptions. So what other approaches exist?

1. Clone

One way is to create a clone of the array to provide a validation checker before invocations as shown below.

var arr = [];
createListenerArray(arr);
var clone = arr.slice(0);

for(var i = 0, len = clone.length; i < len; i++) {
    if(arr.indexOf(clone[i]) > -1) {
        clone[i](i);
    }
}
//
0
1
2
3

This guarantees that the functions would always exist; to verify, we can re-run the snippet again to prove that the value at index 2 is now gone.

for(var i = 0, len = clone.length; i < len; i++) {
if(arr.indexOf(clone[i]) > -1) {
        clone[i](i);
    }
}

//Output
0
1
3

2. Count down

Cloning can lead to new issues with memory allocation and garbage collection. There is another approach that is cleaner, doesn’t require cloning and should not leave danglers around.

The fix relies on understanding the true cause of the looping issue. The problem occurs because the removal of a loop element affects unprocessed elements in the loop (by changing the array length).

We can circumvent this by counting down from right to left – if an element removes itself, the unprocessed elements to the left are not affected since their indices are still valid (compare to the left-to-right count-up approach).

Now, even if a function should remove itself, things are still alright, since its index would be removed and not be valid anymore.

let arr = [];
createListenerArray(arr);

for(var i = arr.length - 1; i >= 0; i--) {
    arr[i](i);
}

//Output
3
2
1
0

console.log(arr.length);
//3

Neat! Now, this is another reason to use utils like _.forEachRight.

Liked this trick? Let me know your thoughts.

Code complete != ship ready


I used to work for a team where whenever an engineer said he was done, the next question would invariably be are you ‘done done‘?

Do you find yourself always under-estimating the amount of time needed to complete some software project? Does it take more time than what you planned? Or do you say you are done but need to change just one small feature.

Accurate estimation is difficult but again an unclear definition of what done means also contributes to this. So what does done mean?

Checklists and Processes

Boeing introduced preflight checklists in 1935 after a prototype crash killed two pilots. Investigations revealed that they had forgotten to carry out a critical step before takeoff. Thus, the fix was to introduce a preflight checklist to avoid this (they learnt from the mistake and became better for it).

A catastrophic software failure can have disastrous outcomes – it is expensive, can harm (imagine if embedded car software were to fail), and is bad PR for the company (how many security breaches have you heard of in recent times?).

So, do we need such heavy formal checklists for software development? Some companies require standardized processes while others don’t. However, it is always good to have a mental model of the various stages of the software shipping pipeline.

Here is a checklist to guide you through the various phases of done; you can now know which stage you are at any given point in time.

1. Code complete

You have written the code (and hopefully added unit tests) but have not done any deep testing. This is what most developers erroneously refer to as done. No, code complete does not mean done. It is the first milestone in a long journey.

Always have tests for code completion

2. Test complete

Thorough exploratory testing of the feature has been carried out. This covers extreme conditions, edge cases and load/performance scenarios.

It’s an exercise in futility to attempt to find all bugs and exercise all possible code paths. Rather, the goal of testing is to reduce the chances of a user finding a glaring simple-to-discover error that should have been found earlier. It also surfaces other issues (e.g usability, security holes etc.) that might have been latent.

Every discovered bug should ideally be fixed and a unit test added to prevent future regressions. However, not all bugs are fixable (and those turn into features…). Early knowledge of these issues enable developers to proactively prepare mitigation plans, workarounds and also notify users.

If you don’t have dedicated testers (they are worth their weight in gold and very good); you  can use James Whittaker’s testing tours method to get some fun while doing this and also ensure a good coverage.

3. Integration complete

You have completed the various isolated features, written unit tests and also done exploratory testing. You have a high confidence in each of the individual features however your components work exactly like the similitude below:

So get to it and do integration of all the disparate components and ensure that things still work as they should. More testing, more discussions with collaborating software engineers and maybe some more glue code.

Why do I keep emphasizing on testing? Have you ever had a bad experience with some software? It’s nearly always a quality issue.

4. Ship ready

This is the cross your t’s and dot your i’s moment; verify that the code solves the problem, issues discovered during testing have been fixed and components have been integrated. Code complete, test complete and integration complete? Great, time to ship.

Some organizations enforce strict limits on some parameters like performance, security, reliability and compliance. If so, you should do another pass to ensure that your software meets these criteria.

The product owner / customer likes the software and viola! Your software piece rolls out like a new car off the production line.

5. Shipped (maintenance ready)

Software is never done. When it is shipped, customers would start using it in ways you never imagined and will find issues. Some can be petty – for example, can you change move this button by 0.2 pixels? While others would be pretty serious stuff.

In maintenance mode, the software already exists but there are new bugs to fix, new feature extensions to make and behavioural modifications to make.

And when you think it’s all done, then maybe it’s time to release a new version. Start at number 1 again.

Enjoyed this post? Follow me on Twitter or subscribe for weekly posts. No spam for sure.

Related Posts

  1. The Myth of Perfect Software
  2. The Effective Programmer – 3 tips to maximize impact

10 years of programming: Lessons Learnt


I wrote my first program around 2006 or thereabout – it was some toy program for the FORTRAN course at school. Ever since then, I have been given the opportunity to take on more challenging programming tasks. It’s been a heck of a journey for me and I am very grateful for the learning, the mentoring from rockstars and the chance to do exciting work.

Looking back, I have learnt a couple of lessons the hard way and wanted to share some of these so that other engineers know what to avoid. These are my rules of thumb currently (in no particular order). Enjoy and let me know your thoughts in the comments!

1. Write plain simple code

Code is meant to be read by computers. Computers and compilers could care less. So if you are feeling super smart then write in 0s and 1s; otherwise, please write extremely simple code.

When exceptions arise necessitating unorthodox or clever approaches; please do add comments and also create documentation. This would help the poor fellow who’s going to maintain it in the future (most likely you).

But why should we avoid clever code? After all, how else does a programmer show off his cool tricks?

if all you have is a hammer, everything looks like a nail – Abraham Maslow, 1966.

The problems associated with smart code are plenty –

  1. They require a higher level of concentration to understand
  2. They are prone to bugs since the smart fluff can hide bugs
  3. They can nearly always be simplified leading to smaller code and bug footprints

Suppose you have a pounding headache and your doctor orders you to buy any non-steroidal anti-inflammatory drug (NSAID). Imagine trying to decipher what NSAID stands for while coping with the raging headache. Not fun eh… What if the doctor told you to buy Panadol (substitute with Tylenol or Ibuprofen etc.)? That’s the similitude of simple vs smart code.

One way to simplify code is by avoiding surprises and establishing consistent patterns. For example, consistent naming patterns, predictable branching styles or standardized design patterns.

Why does this matter? Well it allows someone else to pick up your code, see the structural pattern and then extend it. There is a common language that everyone understands which makes deviations and buggy approaches easy to spot.

But writing dumb plain code is not so easy; so let’s go on.

2. Make it work then clean it up

While working on Odo, the JavaScript Stream library I wrote last year, I  agonized a lot over getting the perfect style. At times too, I catch myself trying to create a masterpiece at the first attempt.  The truth however is that this doesn’t happen often; maybe if you are Donald Knuth.

Most authors do free-flow writing while trying to get their thoughts together; afterwards, the editing and clean up occurs. Why do you think we have book editors? Coding is similar too; rather than rewrite that 3-line function 20 times, why not get the app working and then start refactoring? It’s much faster and code reviews would help catch coding issues you might have missed (just like editors with books).

Typically I just write and get a feature out, then I take a fine sieve to it. All the rules now come into play: DRY, YAGNI, code simplication etc. I do try my best to make it read like poetry (yes I am serious).

3. Never make the same mistake twice

Yes, once is good enough. Twice is bad.

You shouldn’t be shy of making mistakes, no human is infallible but what should scare you is making the same mistake more than once. Do a retrospective after completing each feature to learn what could have been done better and learn from missteps.

I recently took a long amount of time to implement some core software in an unfamiliar area, why? My initial assumptions about existing data structures were wrong and I found out a while later. It wasn’t so difficult to pivot to the right structures and then I ran into another issue – the helper functions I relied on didn’t behave as I expected they would.

Looking back, I could have written tests earlier (instead of later) to validate expected behaviours and also clarified assumptions with senior engineers. Lesson learnt.

A respected senior engineer explained it thus to me:

Junior developers might take a day to realize they are on a wrong path while senior engineers have enough experience to avoid such drains.

Good judgment comes from experience which involves making mistakes and learning. It’s OK to stumble and make mistakes, but please review each one and learn from that.

4. Know where you want to go

How do you get to China? Start heading east. Once you have a target, it is easier to calibrate your efforts and see if you are moving in the right direction. Unfortunately, most software developers never set goals and then get caught up.

The software engineering profession is remarkable for having rapidly changing and moving targets. Currently, the big areas include cloud computing, security, big data, artificial intelligence and AR/VR. Are your skills stagnating or positioning you to shine?

Having a general blanket target (e.g. I want to be a 10X programmer) doesn’t work; there has to be some specificity to it. So for the one who aspires to be a 10X programmer, the next question would be in what areas do you want to excel? Are there examples of such 10X programmers to copy?

Think again; do you want to extend the frontiers of computer science? Design the next Hadoop? Or author a widely popular book / open source software piece.

Know where you want to go and then start walking.

5. Task Breakdown and Estimation

You are to build a brand new calculator app; your manager wants it delivered as fast as possible. How do you estimate the amount of work required and when it’ll be ready for the customer?

How do you eat an elephant? One bite at a time…

The calculator project can be decomposed into modules like the core engine, the user interface and the data storage mechanism for calculation history. Each component should be broken down into smaller chunks. After breakdown, the engineer typically knows the pieces of work that needs to be done before he completes the project and can start implementing each phase.

This quickly becomes tricky when there are multiple people working on the same project and you need to coordinate integration efforts, release dates and project quality.

Always keep in mind the end goal of shipping the software and prioritize tasks accordingly. You want to maximize your return on investment so pick tasks that move you closer to the end goal.

6. Quality

Nope nope nope, testers shouldn’t find simple bugs in your code. You only want them to find bugs that occur when you press 10 buttons simultaneously while doing one-handed push ups standing upside down.

If you don’t find and fix your bugs; your customers will. And they won’t be happy.

Do the right thing; test the happy path first. Then, verify that there are no monsters lurking in the dark corners. Take some time, maybe 30 minutes, to rigourously exercise the code.

I typically write unit tests while completing the feature. Then once it’s ready, every discovered bug gets its own unit test. This helps to protect against regressions in the future. Aside, those unit tests provide active ‘documentation’ of the behaviour of the software.

Don’t compromise on quality – it’s the sign of good software and a great programmer.

tl;dr?

Have been writing programs for some time, made mistakes and learnt lots of lessons.

Do share your thoughts in the comments – would love to hear them!