Development Process

A project manager's job is not to assign tasks but to remove obstacles—which means understanding how work actually flows through a team. Over the past fifty years, researchers and practitioners have described dozens of development processes. Most have passionate advocates and equally passionate critics, but the differences between them are smaller in practice than in theory.

How Most Small Teams Actually Work

Most small research software teams don't follow any named process. They have a list of things to do, people pick items off the list, and eventually the list gets shorter. This works until the team grows past three or four people or the project reaches a level of complexity where nobody can keep the whole thing in their head at once.

The problem isn't chaos—it's invisible chaos. When Jess asked Ren what he was working on and discovered he'd been stuck for two days on a dependency issue, the team had already lost time that better process would have surfaced earlier. What most teams need isn't a complete methodology overhaul: it's a few lightweight practices that make progress and blockers visible.

Agile for Research Software Teams

Agile development broke down work into short cycles, typically one or two weeks, each of which produces something demonstrable. The most widely used flavor is Scrum, which organizes work around sprints. At the start of each sprint, the team agrees on what they will finish. During the sprint, a brief daily check-in surfaces blockers. At the end, the team reviews what got done and adjusts their plan.

Scrum works well for research software teams when:

• Requirements change frequently, which is true of most research software.
• The team is small enough that everyone can be in a single stand-up.
• People can make meaningful progress at least a few days a week.

The danger of adopting "agile" without discipline is that it becomes an excuse for not planning at all. The stand-up meeting only works if each task is small enough to report on meaningfully in a day. "Still working on the climate parser" every day for a week is not a stand-up; it's a warning sign.

Planning and Estimation

If you have a rough sense of what needs to be done over the next few months, a planning session at the start of each sprint is worth the time. The product manager (who in a small lab is often you) brings a prioritized list of desired work. The team discusses what's feasible in the sprint, makes estimates, and commits to a subset.

The hardest part is estimation. Nobody is good at it without practice, and nobody gets better without comparing estimates to outcomes. Keep track: at the end of each sprint, note which tasks took longer than expected and by how much. After three or four sprints, your estimates will improve and you'll have data to show stakeholders when they ask why something is taking so long.

A few things help:

Break tasks until each one fits in a day.

If you can't describe what "done" looks like for a task in a single sentence, the task is too large. Split it until you can.

Separate feature work from maintenance.

Reserve part of each sprint for bug fixes, documentation, and technical debt. If you don't, these accumulate until they dominate everything else.

Add a buffer.

Unplanned work always arrives. Someone files an urgent bug report. A collaborator needs help running the code. Leave slack in the schedule rather than pretending the sprint will go exactly as planned.

Triage

When the backlog is longer than the team can address in any reasonable time, you have to triage it. Prioritize ruthlessly using two axes: how much do users need this, and how hard is it to do?

High-need, low-effort items go first. High-need, high-effort items need scheduling and possibly scoping discussions. Low-need items, regardless of effort, should be closed or deferred unless someone steps up to own them.

[Forsgren2018] found that the highest-performing software teams deployed frequently and recovered quickly from failures. Those outcomes came not from a particular named process but from short feedback loops and good measurement. Whatever process you choose, design it around those two things.

Cutting Scope

A schedule's main purpose is not to tell you what to do each day but to tell you when to stop adding things. If you are halfway through a planned sprint and have completed a third of the work, the right response is to update the plan, not to work nights and weekends.

When the math says you won't finish, drop features. Announce it early: "We won't have the multi-site comparison tool for the October release, but it will be in January" is useful information. The same news delivered on October 30 is not. Real users and collaborators will respect the honesty if it gives them time to plan around it. They will not respect a release that includes a broken version of a feature you promised.