Checklists

Managing Research Software Projects

Greg Wilson

2026

Learning Objectives

  • Reframe individual-contributor technical practices as management responsibilities.
  • Distinguish between claiming a practice is followed and having evidence it is followed.
  • Identify the compliance and safety obligations most likely to apply to your project.

Trust, But Verify

  • As team lead, your job is to make the team follow practices consistently.
  • Individual contributors can rely on their own habits.
  • Managers cannot rely on anyone’s habits, especially under pressure.
  • A checklist protects against the predictable effects of stress and distraction.

Claims vs. Evidence

  • “We use code review” is a claim.
  • “Every PR has an approval before merge” is checkable evidence.
  • Funders, compliance officers, and collaborators may ask for evidence.
  • An audit trail prevents mishaps; it is not surveillance.

Practices to Verify

  • Version control: commits explain why, and no secrets or credentials are committed.
  • Code review: every merge has a reviewer who is not the author.
  • Testing: unit tests run before merge and a failing CI build blocks the merge.
  • Dependencies: versions are pinned and vulnerability reports are acted on.
  • Documentation: every release has a changelog and a working README example.

Safety and Compliance

  • Data privacy: touching human-subject data is data privacy territory even if the software is open.
  • Ethics review: collecting or processing human-subject data may need review board approval.
  • Data-use agreements: they may restrict who sees, stores, or redistributes data.
  • Secrets in version control: rotation, not deletion, is the fix when accidents happen.
  • Licensing: a GPL contribution can change the license of an MIT project.
  • Export controls: accepting contributions from some countries may be regulated.

What about AI?

  • LLMs can draft onboarding checklists from existing documentation
  • LLMs can catch gaps between what docs say and what the repo requires
  • Contributors may fill PR checklists with LLM-suggested answers they did not run

How to Check (5 min)

The list below gives specific things to look for when doing an audit of practices. Add three more, two of which cover areas that these three don’t.

  • Version control: a branch protection rule in GitHub.
  • Code review: a rule requiring reviewers on PRs before merging.
  • Testing: coverage reports checked into version control.

Practice Audit (8 min)

Go through the practices checklist provided earlier. Mark each item: yes (with evidence), probably, no, or don’t know.

  1. For each “yes”, write one sentence of evidence: Not “we use CI” but “CI is configured in .github/workflows/ and the badge on the README shows current status”.

  2. For each “don’t know”, write the name of one person on your team who does know. If you don’t know who knows, write that too.

Compliance Check (7 min)

Repeat the audit using an LLM:

Review my project’s practices against this checklist: [paste checklist items].

Based on what I tell you about my project [two or three sentences describing the project, its data, and its contributors], identify which compliance issues I should investigate.

Run the prompt. Note what the LLM confidently asserted that it cannot actually know, such as your institution’s specific policies, or your jurisdiction’s legal requirements. (It may be able to say useful things about data sharing agreements if those are in the repository.)