Ken Schwaber is best known as the co-creator of Scrum, founder of The Scrum Alliance, and head of Scrum.org. As president of Advanced Development Methods (ADM), a consultancy that helps organizations improve software development practices, he has been quietly working on Nexus, a guide for scaling Scrum in large-scale agile projects. His concept of Scaled Professional Scrum (SPS) forms the core of Nexus, which he describes as "the exoskeleton of scaled Scrum."
TechBeacon caught up with Ken to talk about his new emphasis on helping organizations scale agile, Nexus, and what comes next.
TB: What are the key challenges in scaling Scrum?
Schwaber: We thought in 2001 we had crushed that approach with The Agile Manifesto, but ten years later methodologists have seen an opportunity to make money—a lot of it—through the silver bullet appeal of prescriptive methodologists and their inherent command and control approach.
At the executive level we have had maybe a 30 percent success rate with understanding what agility really is, what it requires, what it delivers, and then being willing to do the hard work to build software that way. In this, agile is no different from lean: simple but hard.
Many late adopters start with CIOs that have been told by their CEO that they are going to be "agile" or "scale Scrum." These organizations have succeeded by optimizing large volumes of work through optimizing work and transaction flow, organizing hierarchically with command and control and delegated work, and instilled predictability through detailed upfront planning.
However, agility and agile require that we change the way people are empowered, trusted, and self-organized. That feels weird to CIOs and their managers. The idea of emerging requirements and projects driven by customers also doesn't feel comfortable. These are new ways of seeing work.
When agile is represented by a methodologist in a command and control, prescriptive method that can be bought and implemented, this feels right to these organizations. Buy it, teach it, implement it—it's done. Except it isn't, because the basis of agility is that software development, as complex work, cannot be standardized into one solution. Every organization and project is unique and requires new solutions. So when an organization that is used to delegating and a methodologist says "give me this money and you'll be agile," it feels good, it feels right, it fits in.
TB: What results have you seen from this approach?
Schwaber: We had tremendous success in earlier years in places like Adobe, Intuit, Cisco, Google, Salesforce.com, and smaller companies that were investing heavily in new products, new ideas, change, and being better. The idea of empowerment, change, and lean products were great. The successes created the buzz and momentum of the agile movement. This caused the business press and conferences to laud the agile movement, to identify it as the key to software project success. But even though these organizations see others that have done better, they are comfortable, don't have any critical problems, and are organized and acculturated to predictive, transaction-based, hierarchically managed organizations. The benefits of agile may be desirable, but there is nothing pressing that justifies the needed work and culture change.
These later organizations use the traditional approach. Commit to buying a solution that doesn't require culture or engineering change, only a change of structure and nomenclature. Instead of putting in the hard work of figuring out how agile would work and devising a continuous improvement plan to get there, organizations are sold agile as an implementable template, one size fit all. Instead of the cost being the sweat equity paid by the early organizations, these later organizations try to buy in.
So if I want regularity and prescriptive and I want agile, that's a conflict. Scrum is very easy, but it's also very hard. The hardness is putting people in conflict with old habits to get new results. If the problem the organization faces isn't so compelling or so awful that people know they have to change, they go with what feels comfortable.
TB: What have you been working on lately?
Schwaber: We continually work on improving our courseware, our training, helping people to assess their own skills. The introduction of things like the Scaled Agile Framework, disciplined agile delivery, and enterprise shared services started to reintroduce methodological, prescriptive, one-size-fits-all thinking back into software development, at the same time claiming that it was agile. However, all of these approaches fall apart at the software development level. They typically just say, "use Scrum," but Scrum is a one-team, one-sprint thing. Also, these methodologies don't tell you how to run large, multi-team Scrum projects or programs.
In a successful Scrum implementation, when organizations want to execute large projects or releases for up to 3,000 people, they devise their own techniques for working out dependencies, cause frequent integration to unearth festering dependencies, and adopt continuous integration and build. Several projects or releases are usually required to devise adequate techniques and tooling, and the organization identifies and works out the problems as expertise aggregates.
These enterprise methodologies have a lot in common with the old rational unified process and consulting—business strategy, platform strategy, system strategy and turning those into plans, portfolios, and a product backlog. That's all classic top-down work that is the basis of high-level consulting. I'm not sure how the businesses implemented these things, but the ideas were pretty good for their level. What I don't see in those is much focus on software development, which is my area of interest. What I didn't see them cover is how to use large teams on large projects.
What I see with these companies is that when they get the release train (or equivalent) down to the development level, they get there and sort of scratch their heads.
So we pulled together a program for scaling. We devised a framework that would help organizations run scaled projects without imposing predictive methods that are antithetical to that organization.
We invented Nexus, a simple framework, an exoskeleton on top of Scrum that enables up to a 100-person effort, along with rules and events for how to do so. We include many practices that identify and remove dependencies and continuously integrate work to remove technical debt and the problematic stabilization phase.
Nexus is an instructive framework that resides above Scrum. It rests on the same theoretical base as Scrum, empiricism, lean, and self-organization. It rests on and extends scrums events, roles, and artifacts.
Just as we made a Scrum Guide for Scrum, we created a Nexus Guide. We've been bringing people in who are using these agile transformation processes and teaching them how to run scaled agile development projects. We were being asked for a structured approach for these organizations to take and use.
The Nexus Diagram
TB: What are the pieces and parts that make up Nexus?
Schwaber: We have, say, four, six, eight teams that we want to work together against a common product backlog. Each of these are regular Scrum teams. What kills these teams are the inevitable large number of dependencies between their work. One person's change corrupts someone else's changes, the branches are in conflict, the tests are in conflict ... there is a lot of conflict.
We use a standard product backlog, but we spend more time refining it. We weed out dependencies so a single team can do the work without interrupting other teams. That's impossible, but we try to come close. We have a Nexus sprint planning meeting, where each team picks a backlog, and they communicate and coordinate to reduce those dependencies. At the end of the meeting you have the shared product backlog, the Nexus goal they hope to achieve, and the single sprint backlog for all the teams, tagged by which team is doing what.
The sprint starts. Work is integrated frequently, at least once a day, to identify and resolve dependencies. If people are working in different branches and they haven't integrated into the code base and aren't delivering, you don't know that you have resolved the dependencies. So we use a lot of modern practices, like continuous delivery, so you know you've resolved dependencies on a daily basis. In the Nexus workshop we show how to do scaled development using 40 modern engineering practices, but these and the underlying tools are often replaced by what an organization already uses or prefers. Since our practices are free, there is no organizational resistance to replacing them.
Of course each team has a daily Scrum, but there is also an integration Nexus Scrum, or scrum of Scrums, that covers how things are building, how the tests are reported, and identifies dependencies, which each team takes back.
There is also a Nexus integration team that's responsible for people working together. It has the overall product owner, the Scrum master, and perhaps not programmers, but consultants, coaches, systems engineers who make sure the system can support all the needs for integration.
So one thing I know we need is the build tools and integration tools, the application lifecycle management tools. That's a pretty stiff learning curve for most organizations. These people are there to help them set it up and use it. Jira, Jenkins, Git, Team Foundation Server, all the things you need to be successful with large projects multiple teams.
TB: So how did people do it in the 1980s?
Schwaber: We got to the end of the project and the heroes kicked everyone out of the way and got it done.
TB: You descaled?
Schwaber: Exactly. We have measures to look at productivity, and one option when these measures get worse and become unacceptable is to descale, to say that we aren't succeeding with seven teams, we'd get just as much done with five. We even have something called scrumble where we realize that we are just piling on technical debt.
One client wanted to create a release candidate every three months, with integration every month. It turns out that expectation was never fulfilled. They simply demonstrated their own code branch without ever integrating it. They took another six months, had the heroes go in and fix it again—descaled. Maybe they shouldn't have had so many teams in the first place, but they certainly created a lot of technical debt and crap and cruft by doing that.
That's why we're having so many integration points—to avoid things proceeding as if they were okay when they were not okay. For Nexus, we do not demonstrate what each team did; we demonstrate the integrated increment.
The sprint retrospective is both at the team level, what do we need to change, and Nexus level, what is affecting all teams badly and needs to change.
Nexus is something that resides on top of a bunch of Scrum teams doing Scrum, designed to seek out and delaminate dependencies, to cause continuous integration, and to deliver products at an acceptable, known cost... [T]o delaminate is to take something that is one piece and tease apart each level of lamination by removing the binding glue of dependencies. When we delaminate a product backlog, we tease it apart to remove dependencies that otherwise cause teams to trip over each other and create a product that can't be integrated. The amount of delamination required depends on the amount and type of dependencies. That is driven by the quality of the existing system, the architecture, and the number and skill of the teams involved in the project.
TB: Looking beyond Nexus and scaling Scrum, what new innovations do you see coming?
Schwaber: Software has enough complexity that efforts with more than 100 people are problematic. Efforts that, because of their functional breadth or technology, require more people should have multiple 100-person efforts, multiple maxed-out Nexuses. We are working on an extension of Nexus, Nexus+, that is a platform approach that provides standards through which the individual Nexuses can integrate without creating dependencies. These platforms are architectural, including APIs, operating systems. They even point at the presence of SDKs that further simplify the Nexus development effort.
Integrating platforms are being devised to support complex, large-scale systems and development.
I also intend to emphasize our evidence based management tools for measuring the value that organizations get from their approach to agility and scaling. These rely on measurements of value, time to market, and innovation to assess the organization's software capability, both as a whole and in particular products or systems.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
