Clara built a habit of continuous improvement – and reduced Change Lead Time by 89% over a year

‍Background

Nancy Barraza, Engineering Manager at Clara, uses Multitudes to understand how her teams are performing and where she should focus her time. Using insights from Multitudes, she identified that <code-text>Change Lead Time<code-text> had increased significantly because of bottlenecks around <code-text>Editing Time<code-text>. 

As she dug into this further, it became clear that there were opportunities to improve how product and engineering collaborated and better scope work. Over the following 9 months, the team built a habit of continuous improvement – ultimately they reduced their <code-text>Change Lead Time<code-text> by 89%, a >20% month-on-month improvement.
‍

‍The challenge

As the fastest unicorn in Latin America, Clara had been experiencing a period of hypergrowth. Like other fast-growing startups, their focus was on keeping up with customer demand, so while they had built quickly and hired rapidly, they hadn’t had much time to consider how they were doing that, and if they were doing it as efficiently as they could. 

When Nancy Barraza started at Clara, one of the first things she did as a new engineering manager was to get an understanding of how her team was doing. Based on some anecdotes, she suspected that there were opportunities to improve how they worked, but she wanted some data to help her understand more about what was going on.

“Multitudes has become my go-to for identifying any blockers that might have arisen. By regularly checking in on the data, I can make more informed decisions over time.”

— Nancy Barraza, Engineering Manager – Clara‍

Our unique insight

Nancy decided to use Multitudes to explore what was happening. When she dived into the metrics, one of the first things she saw was that her team’s <code-text>Change Lead Time<code-text> had been at a high level for months – the 75th percentile (P75) ranged from 100 to 250 hours over the last 3 months. This meant that 1 in 4 Pull Requests (PRs) were taking more than four days to go from first commit to code running in production. The DORA research shows that elite teams have a <code-text>Change Lead Time<code-text> of <24 hours, so they were well above the benchmark.

Note that <code-text>Change Lead Time<code-text> is one of the four key metrics of great software development performance, based on the multi-year DORA research. These metrics are correlated with both financial performance of the company and psychological safety on teams – a win-win.

Curious to understand what was contributing to this trend, Nancy dived further into Multitudes to explore where the bottleneck might be – and saw that <code-text>Editing Time<code-text>, the time spent in rounds of reviews and code edits, was 67% of the overall <code-text>Change Lead Time<code-text>. So reducing <code-text>Editing Time<code-text> was a good first place to start to improve the overall <code-text>Change Lead Time<code-text>.

Given that, Nancy explored <code-text>Editing Time<code-text> further. She saw that P75 for Editing Time had been high, bouncing between 20 and 48 hours over the last three month, sitting overall at 48.5 hours. This meant that 1 in 4 PRs spent about two days in back-and-forth edits, which of course was pushing out the overall <code-text>Change Lead Time<code-text>.

Multitudes surfaced one final part of the puzzle: It looked like the time spent editing PRs might be impacting the team’s ability to get through work. Over this period, the team completed an average of 20.7 tickets per month. For their team of 9, that meant an average of 2.3 tickets per person. That number seemed low so it made Nancy curious to get more context about what might be going on.

Actions taken

After reviewing the data, Nancy’s next step was to get more context from her team. Multitudes pairs the data insights with conversation starters, so Nancy used some of those to discuss the key metrics in a retro with her team, starting with the insights on <code-text>Change Lead Time<code-text>. 

In that discussion, a couple of key points came out –

1. User stories weren’t always clear, and that would sometimes contribute to more back-and-forth in development as the team worked through questions with the product team
2. Other tasks would often interrupt the planned work, from bug tickets to new priorities coming from elsewhere across the business
3. Some engineers were spending up to 4 hours trying to solve blockers themselves before asking for help 

With that additional context, Nancy and her team set to work to improve their processes for scoping and prioritizing work. To support a culture of continuous improvement, Nancy’s team agreed that they would review the metrics regularly in every retro. This way, they can continually identify and address any areas of concern, as well as celebrate the wins along the way.

Over the following nine months, they made a series of changes:

• Nancy started by working with the product management team to create better-designed stories, making it clearer what was needed from the start. 
• To ensure team members wouldn’t be interrupted, Nancy implemented on-call processes – rotations and assigned tickets – so that it was clear who would and would not be interrupted in any given week. Alongside that, Nancy acted as a buffer between engineering and product teams to help with the prioritization of incoming requests, which was met with relief from her team.  
• The team also suggested doing grooming sessions, so that as a team they could discuss, review, and prioritize backlog items, to ensure the most important tickets were being worked on. 
• Since the goal was to reduce <code-text>Editing Time<code-text> – and therefore their <code-text>Change Lead Time<code-text>, the team also thought about other areas where they might be losing time. In the discussion, some people shared that they would sometimes spend 3-4 hours working on the same problem because they got stuck. 
• To make sure that no one was spinning their wheels, they made a team commitment that if someone spent more than 1 hour on the same problem, they would ask to pair program with someone else. This helped team members have a reminder for when they might want to reach out for help. 

As the team made each change, they added up and the team gained momentum – but each of the shifts was small and achievable.

Outcome

The first improvement that Nancy and her team saw was in <code-text>Editing Time<code-text>: Over the following nine months, P75 came down by an incredible 96%. It stabilized at 1.4 hours, putting them well within their target range.

All the effort that the team put in also paid off with a higher volume of work. As the improvements took effect, Nancy’s team was able to deliver 5x more tickets.

“What we love about Multitudes is that it gives us intelligent suggestions to implement to make our teams better. Multitudes helps us make better decisions and take action.”

— Eduardo Antonio Davalos Camarena, Engineering Director – Clara

These wins were all a leading indicator for the success they had with their <code-text>Change Lead Time<code-text>: The P75 dropped by 89% to 19 hours– putting them well within the benchmark for elite teams!

One of the biggest wins here was how Nancy’s team built a habit of continuous improvement throughout the whole year – not only did they have massive initial gains in <code-text>Change Lead Time<code-text>, but they showed continued progress through to the end of the year.

This reinforces the power of continuous improvement – after all, just getting 1% better every day for a year can make you 37x better at the end of it.

What next? 

To keep unblocking the team, Nancy could use Multitudes’s Google Calendar integration to understand if her team is also getting interrupted by meetings. Extensive research shows that changing how we work to include more deep work (or “focus”) time can increase productivity.