The Black Box Model

The pancake challenge: In this level of maturity I am able to get all the ingredients, the pan, and a stove.

• I can turn the stove on, put the pan in there and cook a pancake following a recipe.

• Because we only focus on the process its normal that we waste ingredients, the pancake is not tasty or that it takes us 2 hours to do one pancake and that the customer complains a lot about it.

The pancake challenge: In this level of maturity I am able to access if the ingredients meet my quality standards, I can easily anticipate how long it will take me to finish my pancake and what does it mean to say my pancake is ready, in more detail:

• When someone asks me to do a pancake plate, it tells me nothing about how many does he want, if it should come with a topping or not and even if he wants it warm or cold.

  • I get tired of complaints, so I start making a set of questions when people make an order so that I can meet their expectations at the end.

• I know exactly how long does it take to make a pancake so at any step of the cooking process I can tell my customer how much more is needed to finish the pancakes.

• I start following a picture of the expected outcome so that my customers know exactly what I am going to put in front of them at the end of the cooking process.

• The pancake challenge: Now that I have the ingredient, control the process and have a recipe, I keep facing the same problem, the pancake is not tasty enough and it looks different every time I do it. In order to fix it, I have decided to enroll in a cooking course so that I can learn the best practices on how to cook a pancake.

• After taking the course I have learned that I was not mixing the ingredients in the right order and was using too much oil to cook it. Now my pancake is tasty and my customers are enjoying it.

• The pancake challenge: How can I make sure that whenever I do a pancake it will take a similar time than before and will taste the same way? 

• The pancake challenge: Imagine one of your ingredients, say an egg, is rotten. This should be your first control check, making sure your base ingredient is not expired and that you can trust its usage. However, I also need to make sure that I am following the recipe properly so maybe I should write it down, follow it carefully every time, weigh the ingredients and test the consistency and looks of the pancake at every time.

• Last but not least, should I try every single pancake before giving it to my customers? I don’t think so, maybe its enough to ask for feedback from them and to do a proof of concept once in a while. What if a customer complaints? In that case, I should be careful in the next pancakes and try a little one before serving to the customer. Once the trust is restored I can start cooking without fear again.

• All this will ensure my pancake is of good quality, can be delivered in time and my customers trust my work.

This level is all about the basics. In order to implement a development process, you need to make sure:

1. Requirements are flowing into the team frequently

2. Your development team has proper tools and hardware so that they can start delivering

3. Deliverables are produced often and feedback from end-users is being collected (Search for content on Agile, Lean, XP and Kanban to improve here)

Basically, there is something coming in, development is being done and something is coming out. It’s making sure the basics are there: requirements, people, equipment, communication and feedback channels.

This level is all about controlling the workflow and is probably the most complex one and, if properly implemented, a game-changer in terms of quality of delivery. As we grow in maturity we need to keep track of what we are working on, split the work across the team and make sure we can anticipate what is going to be developed. Tools like Trello (For simple environments) or JIRA (for more complex ones) play a major role in this step and you should consider having them in place at this level.

If you look at the previous figure you will notice there are some rectangles in the development phase. This is very important to make sure we can assess, at any given step of our scrum sprint or development phase, what is going on, how is it going and if we are going to deliver it with the expected quality at the expected time.

Another important thing at this level is making sure the requirements you are receiving are properly structured and are fully understandable and that whatever you produce is following a set of rules that ensure the minimum quality of delivery.

In terms of items to focus on at this level of maturity:

1. Definition of Ready: To ensure that the requirements are coming in such a way that there are no doubts about what needs to be done, who to contact in terms of doubts, the DoR is the best tool for you. This is a commitment of the development team stating that: “If you guys follow these rules and give us these required inputs, we are able to start working on your request and deliver it with the expected quality at the expected time”.

2. Issue tracking: When our projects become more complex, tracking the work being done by the team can become a complex task. However, it’s crucial that we have a process that states: how to log new work; what work status should exist and how to move between them (and who can do it); how to make visible to the whole team the current delivery status compared to what was promised. Again, tools like Trello or JIRA play a major role here.

3. Monitoring and Tracking: Here I am not talking about systems monitoring, I am talking about work and this means it’s important to implement processes (some tools make this very simple), that allow us to understand where we stand and if something starts getting delayed we can immediately act on it.

4. Definition of Done: When your team says it’s done, what does it mean actually? This is what the DoD answer. The team should agree on the criteria that make them say development is done and can be delivered to your end customer. This is quality assurance as your stakeholders and end-users know that everything that is released at least followed those rules. This does not create bulletproof items but ensures minimum quality.

Hint: DoR and DoD for those using JIRA can easily be added as a new ticket attribute (Checklist) and be used to condition the workflow of a ticket. “It can only move to close if all DoD items were ticked”. 

Basically, in this level, we implement a process that allows us to know: what work is in backlog, what’s being implemented now and what’s the status and what does it mean when the team says it’s done.

This level is all about excellence. For whatever you are doing, are your following best practices, validated in the community or following the proper rules to writing code?

This kind of initiative is important not just to embed quality in your development process but also to ensure a standard communication and understanding or your architectures and implementation across all team members, even new ones.

Some examples of what can be brainstormed with the team at this level are:

• Implementing a coding standards document stating the basic rules to writing code in your development cycles.

• Consider the usage of proper branching strategies to increase the control over your versioning process.

• Consider the usage of infrastructure as code frameworks to ease and give more maturity to the management of your infrastructure.

Although the previous examples focus on technology, also think about the organization of your team, considering things like:

• Do we have the proper hard skills to execute our work? (Consider creating a skillset matrix)

• Should we deliver training programs to our resources?

• Is our team properly setup? Are we missing a required role?

• Is our Agile method efficient?

  • Consider reviewing your agile process and improve it with the team.

Hint: Again, it’s important to mention this is the moment you take a team that works and transform it into a team that shines and achieves high output levels.

When we assemble a team, development or not, making mistakes is important as they allow us to quickly learn how not to do something. However, once you achieve a certain success you must make sure your quality of delivery is predictable and that whatever you produce will be delivered with a similarly “taste” to the previous one.

I like to think of this level as the one where you turn your team into a McDonalds one. Imagine what it would be to go three times to Mcdonalds and your big mac tasting differently every time. What will naturally happen is that you lose trust in the restaurant as you cannot anticipate how your burger will taste.

But the problem at this level is not just the quality of the product, it’s also the speed of delivery. Again, imagine your big mac takes 10 min to deliver today and 1h tomorrow. In order to have efficient teams that generate trust, both quality and time to market should be predictable.

One of the best ways to ensure repeatable quality is through automation because as you know computers are great at following orders and can make it in a very similar way every time.

In development processes this can be achieved in multiple ways:

• Implementation of Continous integration and continuous deployment pipelines (Read about DevOps if you want to know more about it.)

  • This will make your “idea to production” process repeatable, predictable and faster than compared to performing it by hand (You never know if your developer got a bad night, and will make a minor mistake that will cost you a big chunk of your system availability)

• Automation, Automation, Automation, just look at all the manual processes your developers are doing and invest in scripting it.

  • It can go from turning EC2 instances on and off automatically to extracting a certain dataset from JIRA to even smaller things like monitoring the success of your ELT flows.

• Containerization.

  • • A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. (Got it from docker website)

    • This is a great way to implement automation at the application migration between environments level and makes the setup of environments very straight forward and predictable.

    • Consider the usage of containerization for easing the deployment process.

More ways might exist, it’s up to you and your team to assess which ones exist and when and how you want to implement it.

The main message in this level here is, if your team is doing something manually, frequently add it to the action items as an automation opportunity.

This level is all about quality. And not just the quality of the deliverable but the quality of the whole implementation process and of the software that is being operated. We need to make sure as a team that we fully control what is happening in a timely way.

I am not a fan of implementing quality controls everywhere but of implementing the ones that will allow us to trust whatever we develop and deliver. And of course, this is highly related to the complexity of the system we develop as simple systems might require less quality gates than complex ones.

The reason I am an advocate of controlled quality is inspired by how typically a factory works. If a factory is producing thousands of products every day and decides to assess the quality of each one of them, it will take a lot more than if they do quality analysis over a sample of those. This will allow them to assess how much they trust the quality of the products developed in one day and of course if there are many defects they can still act on it. Its a tradeoff between speed of delivery and number of defects that go to waste.

And this example leads me to my point: trust. You should invest more or less in new quality controls based on the trust you have on the quality of your deliverables. If your team is delivering high-quality ones, totally aligned with your business stakeholders that are scaling well, follow a normal continuous improvement pace and keep adding new controls. However, if you are constantly failing and bugs are increasing like hell, spend more time with your team at this level and implement as many controls as necessary. But whatever happens, don’t freak out, working software is still your best quality measure.

In development processes there are many quality controls that you can implement but start by brainstorming the following with your team:

• Unit testing this type of test allows you to test individual units/ components of a software.

  • See that new transformation function your team is working on? It should have a corresponding unit test to make sure the basic behavior is working. This type of test makes sure that atomically all your system components work. However, they don’t test them in an integrated way.

• Integrated Testing is when individual software modules are combined and tested as a group.

  • Say you have a data engineering team and they are developing data pipelines. This is the time of the test that combines all transformation functions into the required pipeline and assesses that the outcome is expected.

• Code coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs.

  • It basically tells you how much of your code is being tested. Say you score 80%, that means there is 20% of your code that was implemented and not tested using a unit test. Meaning it is at high risk of having a bug and potentially break your system

• Logging allows you to keep track of all the events that have occurred in your system (Say all the records you have processed, the number of exceptions thrown in your try…catch block, etc ). You can save it to a file or use a proper logging platform such as Splunk.

  • This will allow you to know exactly what happened and when it happened in your system. Imagine you receive a call in the middle of the night saying your ETL flow as failed, where are you gonna check what happened before relaunching the chain? In your logs. Without it, you should pray that your systems don’t fail because if it does you won’t know where to check that.

• Systems monitoring allows you to know the health of the infrastructure where your applications are running but also the health of your application like the availability percentage, downtime etc.

  • This is critical to make sure your environment is healthy and that your application is performing in accordance with what is expected.

Achieving an Agile Culture