What tools should I use? Part 2

Charles Cary
January 15, 2020

We started Shoreline in 2019 to solve problems that we faced as operators. As businesses become more agile and engineering teams push out new features more frequently, it has been harder for operations teams to keep up. Our aim is to make life easier in a CI/CD, multi-environment world.

In our previous post we discussed the unique challenge that we faced as a team of operators attempting to develop a software product: balancing the two competing demands of needing high variability to innovate and low variability to deliver. Both of these demands must be met to effectively deliver a product that is also competitive.

To satisfy these demands we use a two-part control loop—one of which sets targets and the other that drives the system to those targets. In this post we will go into more detail about our tools, processes, and how the two-part loop will continue to influence our growth as a company.

The Tools

To implement these control loops, we use Wrike for Gantt and GitHub Projects for our workboard. Our Wrike project syncs to GitHub and we use GitHub automation via pull requests to automatically close issues when the pull request for them is complete. This source control integration is crucial, as it is not efficient or economical to demand individual contributors spend significant time tracking work. It needs to happen automatically.

We use Wrike because it’s the only Gantt tool we could find with synchronization with Github. To be functional, we needed to be able to batch synchronization, otherwise issues come and go while the plans updates, which is very confusing for engineers. In addition, we needed to have two-way sync: closing an issue in GitHub needs to close the issue in Wrike.

With other tools, we had major problems manually synchronizing the long-term plan with the current work in progress. This would result in the plan and engineering work diverging over time, rendering the plan useless.

Additional benefits of Wrike include its great support for visually laying out a long-term plan, assigning work items to people, producing reports on resource capacity, detailed filtering on work items, and producing a report of whole plan and its status for a board meeting.

The Process

What we do is not exactly agile, and not exactly waterfall. We’re not dogmatic about either methodology and recognize that it takes a mixture of both to develop and ship a competitive product.

The requirements and design phases of waterfall can be a two-edged sword. We need a sufficient amount of time to generate ideas and create variability. But too much time poses a risk. We could have found ourselves in a never-ending design phase if we weren’t careful to break down design tasks and set time limits.

And while we shouldn’t start building with no direction in mind, writing a very large number of design documents at the beginning (i.e., fully spec up front), doesn’t make for where we are as a company. How could we fully spec out the inevitable unknowns that come with bringing a new product to market?

A pure agile approach wouldn’t have worked either. If we just jumped in without allocating enough time for design, we posed the risk of removing too much variability from the system. As we noted in our previous post, that variability will be what sets us apart.

So how are you supposed to iterate enough to find product market fit? We use what we see as  the best parts of both approaches. We see benefits in the planning steps of waterfall especially long-term aspects like estimates and dependencies. However, we use agile-style shipping of many work items, including design. We treat design as work and assign time to it.

Instead of writing full specifications in the beginning, we draft a living document. Once a work item is completed it then becomes part of the document. This gives us the flexibility to iterate if necessary.

The Power of the 2-Part Loop

As Shoreline grows and we look to the future, we see two-part loop as an integral part of our trajectory as an organization, not in just how we ship products.

Scaling large teams

Two-pizza teams work well in an agile system, but there inevitably come situations where you need a team larger than this to complete a project. Adding more people while still adhering to agile methodology will lead to communication breakdowns (team growth is inversely related to output). Dependencies will also be poorly modeled in the workboard because it will be more difficult to estimate task times.

Our two-control loop mechanism works for this fluctuating team size. The time spent manually entering state changes to work items each day is significantly reduced because Wrike and Github sync automatically. In addition, everyone is producing documents and code and one person also owns the long term planning delivery.

Enabling large teams with subteams

Although we might keep individual teams at the two-pizza size, organizational growth leads to an increase in the number of teams. This introduces the challenge of cross-team coordination. While an individual team may work in a semi serial fashion, the whole organization is a concurrent and parallel machine. With the two-part loop, multiple work streams with dependencies between each other can be modelled.

Enabling distributed teams

Currently, a quarter of Shoreline’s engineering team is remote, and we expect that proportion will grow over time. We use the two-part control loop to facilitate asynchronous communication. Always communicating the long-term and short-term plan is a very powerful way of building community in a distributed organization.

It’s not just for engineering

Like most startups we have long-term plans of building out an entire organization to support our products. The two-part loop will be coming along for the ride. As we mentioned in the last post this method already drives our staffing decisions. We know the skills we will need, when we need them, and can develop a hiring plan around it.

When we add sales, marketing, product and other teams we will use the two-part control loop to align plans between all of them. Too many startups make the mistake of siloing these teams early on, and getting in sync becomes more difficult as they grow. We plan on delivering a “whole product” meaning that yes, the software has to be great, but the services, partnerships, and documentation in support of it need to be great as well.

This synchronization is especially critical between product and engineering teams because they are really one thing. Product needs to know what the engineering team is doing in order to prioritize feature requests and engineering needs to understand that what they are building comes from the market and product innovation.

We’re excited to share our journey with you. There’s more to come. Subscribe to our mailing list to stay up-to-date with what’s happening at Shoreline.