Imagine, for a moment, that you are the general manager of a division or the president of a small company. Inevitably, you have more ideas and initiatives than you have time, people, and money. The tough task ahead of you, then, is to select the "vital few" projects to fund. Anything more, and your team will lose focus.
Given that choice, two things come to mind:
- First, the size of the project (smaller is better)
- Second, the value of the project (higher is better)
Weighted Shortest Job First is a technique for a) assigning a weight, or value, to each job, and then b) dividing that by the length of the job, in order to c) determine a relative ranking.
Sort the spreadsheet by ranking, and you get a logical, reasonable priority order. It all makes perfect sense. Right up until you try to do it, which tends to be when the wheels fall off.
From theory to practice
On the surface, WSJF is fantastic. But like the old Cocomo Model for software estimates—which purported to estimate length in person-months of a project—you need to know the number of lines of code, which is just about as hard to estimate. WSJF tells you a relative order of projects, given duration (which is hard to calculate) and weighting (which is harder).
We could use something like return on investment, but ROI also has an implied division-by-cost element, and cost roughly corresponds to "shortest." Not only that, but ROI over what period? How do we compare a project that will generate $1 million in revenue one time versus one that will generate $400,000 a year for five years? 
In his book "The Principles of Product Development Flow," Donald Reinertsen introduces WSJ, and suggests a new calculation: cost of delay divided by duration, to determine WSJF. Dividing by duration is the "shortest job" part, which makes it clear that 12 one-month projects could add up to more value than one 12-month project with six times the revenue potential.
To use the formula, we need the value of the variable, and that's where things get tricky. Determining cost of delay can be extremely tricky, especially on one-time projects. A large retailer like Target, Walmart, or McDonald's might be able to tell the cost of delay for opening a store. Given enough existing stores (to know the average price), some demographic data (population and income), and data on the distance from competing stores, a nuanced data analyst might be able to calculate the sale, profit, and potential for a store. All of that relies on the law of large data—on having data sets large enough that averages are meaningful.
In software, that can only be effective when the largest of companies create a new product that is very similar to what came before. It might work when Microsoft releases the next version of an educational package, or possibly when EA Games releases the next version of Madden Football. But for internal software or even new product development, without strong, financially validated experiments, cost of delay sounds like yet another guess.
Joshua Arnold, author of Black Swan Farming with Cost of Delay, sets aside the objection that the numbers are hard to calculate by referring to the number as simply a "strawman." Interviewed for this article, he commented, "Whether we like it or not, assumptions about cost of delay are being made on a daily basis, based mostly on gut feel. In product development, we are constantly trading off time and money, so implicit cost of delay decisions are embedded all over the place. These hidden assumptions about cost of delay are the current status quo. We can't avoid it. Surfacing those assumptions and sharing them with the development team may invite challenge, but it doesn't fundamentally change the risky/stochastic nature of product development."
Joshua uses the same formula as Reinertsen to calculate WSJF: cost of delay divided by duration, which he termed CD3. That's very different from the manner in which the Scaled Agile Framework, or SAFe, calculates it.
WSJF in the Scaled Agile Framework
I spoke with Dean Leffingwell, lead author of SAFe, about how he understands the concept. In SAFe, WSJF is implemented by this formula:
WSJF = (user/business value + time criticality + (risk reduction (RR) + opportunity enablement (OE) value) ) / job size
While job size remains the same, the "weight" here is the sum of three variables, all on a scale from 1 to 20.
User/business value is a relative score from about 1 to 10. Leffingwell suggested using the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, etc.) mostly because larger numbers are less precise and using Fibonacci makes consensus easier.
Time criticality indicates the importance of doing the project now. If a project is highly time critical, then our competitors are considering it as well, and if we are not first to market, then the value we can capture will be limited. Projects that are not time critical have a longer shelf life.
Risk reduction and opportunity enablement indicate the project either positions the company to make follow-on projects (like a new video game console) or mitigates risk. Moving to the cloud, for example, might not generate revenue directly, but it might reduce data center failure risk while making future work faster.
This sort of math works on any piece of work in a scaled agile project—from helping teams decide what story to work on next to helping the company decide what major initiative to fund, and everything in between.
If it works.
Problems with SAFe-style WSJF
This idea of deriving numbers sounds like sound science, but adding together relative rankings creates problems. For a simple example, try using the children's tale "Goldilocks and the Three Bears." Papa bear's chair was too big (3), mama's was also too big (2), and baby's was just right (1). We can use those numbers to represent 1st, 2nd, and 3rd, but no one would expect that if we added the weight or size of the chairs that mama and baby's (2+1) would equal papa's. These are ordinal numbers, and we cannot add or multiply them in a meaningful way.
Yet that is exactly how WSJF works in SAFe—adding together the relative goodness of customer value, time criticality, and risk reduction. When I asked Dean about this, he pointed out the accuracy problem we mentioned above: There are inherent problems in calculating cost of delay, especially for anything but certain kinds of product companies with certain kinds of research techniques. He sees the value in WSJF in at least getting a real list of priorities, which allows a portfolio management team to fund the project.
Problems with skill sets in play
Which brings up a second problem. If we fund the most valuable project, then the second most valuable, then the third most valuable, eventually we will find we have conflict. We have enough people to do project four, but not enough DBAs. We could do project seven on the list, but that means doing a lower value job because of staffing issues. One "solution" to this is to identify multiskilled people and build implementation teams that have the optimal combination of skills. In my experience, this is a little bit too much like Tetris. When we want to start the next "block," or project, we'll need 10 people, only eight of which are available, so we start the other two part time and ...
... it's a mess. Personally, I would rather develop multiskilled project teams that have all the skills, perhaps a little more or a little less than what this project needs, and get them started on the next project when the last one finishes.
That means some members of a project team might be underutilized (not busy) because they are, for example, DBAs and the project has no database component. In that case, I suggest they pair to learn a skill. Companies that can't accept this will probably end up with the skills matrix, the large portfolio management group, the Tetris, and the multitasking. They will deserve what they get.
What is a "shortest job," anyway?
When I explored this topics with Adam Yuret, an enterprise change coach, he pointed out that WSJF is not perfect. Imagine, for example, that a company has the potential to do four different jobs. Each will take one year. The first three can be done with three people each and generate $300K in revenue annually; the fourth requires all nine and generates $700K. Plug in one year in a WSJF formula and sort, and you'd fund the one that generates $700,000.
Except, of course, you wouldn't. You'd fund the first three projects, because they are shorter as measured in person-years. You might also look to half-life—you might want to know how those $300K revenues decline (or expand) over time—and if they were different, that might make a difference. Leffingwell hints at this with his time criticality.
All this means it won't be that easy to come up with a ranking. Even if you do, slow down. Consider more variables. Use WSJF as a tool to figure out what to build and when.
Give it a try
At this point, I would warn to not make it the tool, but Joshua had a bit more advice for me. That is, don't plan on WSJF saving you. Instead, experiment with at least trying to make it explicit, along with other modern techniques like stable, cross-functional teams, optimizing for end-to-end delivery and building in quality along the way. As Joshua puts it, without those, no scheduling algorithm is going to save you.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
