The AWS meetup group in New York invited Hudl CTO Brian Kaiser to speak in September. He presented how Hudl implemented agile methods to overcome slowing code deployment time. As his company grew, deploys that used to take 30 minutes could take a few hours or even be pushed to the next day. Brian emphasized that their agile transformation began with the reorganization of teams into Agile Squads. Only later did they transform their “Monolith” application using microservices technology. Hudl’s approach offers insight on how enterprises can successfully transition Big IT to Big-and-Fast IT.
Hudl provides tools for coaches and athletes to improve play and strategy. Their initial tools were for football, a sport that epitomizes the importance of agility. Analyst David Linthicum once wrote about the important of agility in cloud computing using football analogies, including these.
- Agility beats strategy each and every game
- Agility (of a lineman) is more effective than size
- The game can change quickly
In football, a team could be down by 20 at halftime and still win, or vice-versa. Those of us in technology know that our game can change quickly too. New disruptive innovations abound. How do enterprises take advantage of what is new? How can enterprises become more agile?
The Problem at Hudl
Agile DevOps approaches have been pioneered by fast-growing Internet-unicorns like Netflix, Spotify, Etsy and Twitter. Hudl’s approach is inspired by Spotify and they leverage technology from Netflix. What worked in a 20-person startup did not work as well as they grew. These are the problems they faced.
- Growing size of code
- Slowing code deployment times
- Organizational issues included more meetings and lack of autonomy
Action #1. Reorganize
The first action they took was to reorganize into adaptive, self-organization units called Squads. Squads provide their members with autonomy, mastery and purpose – what Daniel Pink lists as the three key elements of motivation.
- Autonomy – Each squad self-organizes
- Mastery – Each squad choses the tools and programming languages they want to use. They become experts in the tools they chose. Developers in this style of organization are often referred to as full-stack developers, who leverage new tools and cloud services to perform a full range of tasks including development, QA and operations.
- Purpose – Each squad owns a product or piece of a product. Each squad has all the necessary skills, e.g. development, test, operations and marketing.
Hudl’s Agile Squads framework is inspired by Spotify. Figure 1 depicts how Spotify deployed Squads, Tribes, Chapters and Guides in late 2012. To put it in a nutshell, this approach turns the traditional matrix organization on its side. Instead of multiple departments that work together with dubious ownership, they organize vertically around products or parts of products.
Squads provide flexibility as they are formed, grown, divided or dispersed as the business evolves. Coordination and communication is facilitated by structures called tribes and chapters. Groups of Squads that are related by product or strategy are called Tribes. Groups of people with similar skills like web developers or backend collaborate as Chapters. Communities of interest that share knowledge, tools, code and practices are called Guilds. They key is that while Squad members can interact across Squads. They are focused on what is most important, the product. For more information on Spotify’s organization, see “Scaling Agile @ Spotify”
Action #2. Transform from a Monolithic application to Microservices
After Hudl reorganized, they then began to adopt their technology. The transformed their monolithic application to a microservices architecture. Figure 2 shows a high level architecture of Hudl’s “multiverse” microservices architecture. Here are some of the challenges they faced as they transitioned their technology.
- Routing and transport among cloud instances
- Database performance
- Low utilization of cloud instances
Brian stated that routing and transport is extremely important. They need to very quickly reroute communication if a server or cloud instance goes down. They also have circuit breakers to prevent cascading failures. Over time, they migrated from SQL server to MongoDB due to performance issues. On AWS EC2, vertically scaling a database was not a great option so they chose MongoDB because, at the time, it was the easiest to shard out horizontally – see this Hudl blog post for more information about their choice of MongoDB. Their microservices approach also does not have the best utilization of cloud instances and are therefore exploring using container technology. For more information on Hudl’s “multiverse”, see this blog post.
Also consider when a microservices approach would be beneficial. Hudl’s CTO noted that starting with a monolithic application was simpler and easier to get started with. Starting with a microservices architecture would have introduced excessive complexity when the company was just getting started. A good question to ask is, at what point do the benefits of microservices outweigh the costs?
What can enterprises learn?
Organizational transformation can be just as important as technology as one journeys towards becoming more agile. Many enterprises, when they pursue agility, begin by implementing technology. Perhaps they start with a pilot of Chef, Puppet, Github, AWS, Softlayer, Cloud Foundry or OpenStack. Before jumping into the new tech pilot, ask this question. Is the organization ready to benefit from the new technology?
In addition to Agile Squads, other agile organization approaches include Holocracy (e.g. think Zappos and Medium) and Self-Organization (e.g. think GitHub).