This post challenges misconceptions about chaotic on-call and livesite practices, offering lessons from extensive experience. It introduces common red flags like call hell, hero worship, and the wild west, and provides solutions. These include customer-focused monitoring, monitoring pruning, 1-2-3 troubleshooting rule, follow-the-sun schedules, and repair item deadlines. As services mature, standardized incident response and efficient toil control practices become crucial.
Category: Best Practices
Why most monitoring strategies fail
A team without proven observability and on-call strategies will invariably suffer from reactive disruptions; mitigating outages will be painful, like finding a needle in a haystack while blindfolded.
Using Systems Thinking to craft high-leverage strategies
Most teams struggle with removing friction because they concentrate on surface-level reactionary fixes instead of addressing the fundamental causes of inefficiency.
When sleeping dogs bite: Unmaintained systems breed disasters
The issue with systems that do not 'fail' is that they have no fixes when they eventually fail.
High Leverage Activities for Teams : Documentation
Excellent documentation leads to efficiency gains, insufficient documentation leads to bottlenecks, while poor documentation sprouts confusion.
9 multipliers for boosting your team’s productivity
Multipliers make or mar engineering organizations - teams that invest in boosting the right capabilities at the right time will get more done with less. Teams that neglect these capabilities will eventually get bogged down – they’ll get less done with more.
No Surprises: A framework for Software Quality
When most teams complain about poor quality, they usually mean reliability woes; however, quality spans a more extensive spectrum and can mean many things. If you complain about your software being of low quality, what dimension do you mean? Use the Maslow quality hierarchy to identify the pertinent challenges and make the right tradeoffs.
How to run better stand-ups
There is a high chance that you attend or have attended an inefficiently-run stand-up. I have seen various stand-up styles over the years. Sadly, most of the roughly 2000 stand-ups I attended were unproductive. Mildly put, most were status reports for some manager or higher up.
Five challenges of engineering teams
5 important ideas that engineering teams need to keep in mind to optimize value delivery.
Why you should delete code every sprint
Constantly factoring deletes into your iterations keeps your code base healthy
Lessons learned from running services at scale: 1
Tips for running services at scale with minimal toil
Not all Technical debt is bad
Some engineers believe they have to go to great lengths to eliminate every single piece of technical debt in their codebase. This focus on perfection ignores the cost of fixing debt, the risk of introducing new bugs and contagion (the chances of debt spreading).
Do you want to sleep well at night?
Do you want to sleep well at night?
How to build Resilient Software
One of the most challenging aspects of software development is staging changes without breaking the service. Releasing new features always comes with a risk - bugs might be introduced and existing failure points might become more prone to failure.
5 Tips for consistently crafting delightful User Experiences
Have you ever wondered why some applications always look and feel similar? Why for example does Apple have a unified experience across devices? Why are Google products starting to adopt the material experience?
Things to check before releasing your web application
A couple of things to validate before you press the 'go-live' button on that wonderful web application of yours.
Code complete != ship ready
I used to work for a team where whenever an engineer said he was done, the next question would invariably be are you 'done done'?
Quick estimation tips for engineers
Engineers need to estimate system performance and simulate real-life scenarios. For most engineering fields, there are rich banks of proven theories and mathematical relations to rely upon. Unfortunately, software engineering - the new kid on the block - has a few rigorous rules, most times we rely on heuristics and handed-down wisdom.
What is Semantic Versioning (SemVer)?
Software Versioning Software versioning has always been a problem for software developers, release managers and consumers since time immemorial. For developers, the challenge lies in releasing new breaking changes while simultaneously minimizing consumer upgrade pains. On the flip side; consumers, when they finally decide to upgrade to new-shiny-release-10000, want to be sure they are not buying a one-way-ticket … Continue reading What is Semantic Versioning (SemVer)?
Code is Poetry : 5 steps to bulletproof code
Programmers have to love their craft and put their best into making it stand out.
The Myth of Perfect Software
Programs do not acquire bugs as people acquire germs, by hanging around other buggy programs. Programmers must insert them... Harlan Mills Software breaks all the time: booting issues, corrupt software and files, crashes etc; nearly everyone has had a close shave or two with fragile software. Can programmers write 'perfect' fault-free software? I presume a trip to … Continue reading The Myth of Perfect Software
Design Patterns: PubSub Explained
I actually wanted to write about PubSub alone: it’s a fascinating design pattern to me however, the thought occurred to me, why not write a design patterns’ series? It’ll be good knowledge for me and give good information. So here goes the first: PubSub.
The Singleton Pattern
I stumbled upon the singleton pattern while reading a PHP book and fell in love with its simplicity. I rushed to use it at every single opportunity I got (when you have a hammer, everything looks like a nail, right?) until I got tired of it; now I can't really remember when I used it … Continue reading The Singleton Pattern
Beautiful Code 1 : 5 Symptoms of Software Rot
I used to wonder why people would refer to software development as an art; to me there was absolutely no correlation between programming and art. However, after hacking at software for years and writing all sorts of software: crappy ( I bet I'll would hide my face in shame if I see some of my old code), good and ugly, I believe it is an art.
CodeKraft 