← MethodsSCREEN READING EDITIONDownload the PDF

How Much Evidence Is Enough?

Compare action, inquiry, delay and staged commitment for one real choice

1

Enough evidence for which next step?

Your team is considering a change to a service. One colleague wants another study, and another says you already know enough. What does this next step need, what could change the choice, and what happens while you wait?

The short answer: compare the practical options of acting, making a focused inquiry, waiting, and committing in stages. For each one, look at how it would bring about the change, what conditions it needs, what burdens it creates and what its consequences would be. A proposal that can be reversed can be sensible even without proof that the program has an effect. But some actions that look small can still expose people to serious harm.

What you end up with is an evidence-and-action brief. It sets out the choice, the alternatives, the strongest account the evidence supports, the uncertainty that could change the choice, any inquiry worth considering, and the conditions for approval and review.

Where should you start?

For a meeting today, use the short first-pass companion and the blank brief. For a fully worked example of the reasoning, start with the advice service's decision in the section “What is the advice service deciding?”. No score and no universal threshold of evidence can give the institution its answer.

2

Which distinctions make the choice clearer?

Keep four questions apart. What do the sources show? How could each option bring about the change you want? Which conditions must hold for it to work? Who can approve the choice, and who will live with its consequences? Better data can help with some of these questions while leaving priorities and authority unsettled.

A pivotal uncertainty is one that could reverse the choice or change it a great deal. If an evening service cannot be run because there are no staff or safe premises, another satisfaction survey will not change that. If people need help during working hours, opening in the evening may not solve the access problem, even if the idea is popular.

Weigh the uncertainty against the alternatives. Could existing sources settle it? Would a focused inquiry provide relevant information before the decision? Would acting in stages let you learn something useful while limiting the risk? Would waiting protect people, or would it drag out an important need that is going unmet? Say who would feel each consequence.

Do not assume that a choice is "low stakes" because the budget is small, or that it is "reversible" because the trial is short. Lost access, breaches of confidentiality, staff exhaustion and damaged relationships may not come back when a schedule is changed back. The conditions for acting safely in stages need to be checked openly.

Keep the conclusion about the evidence separate from the proposal. "Some callers report that the hours are a barrier" can support looking at opening hours. It does not show that evening appointments improve outcomes. A leader may still approve a limited trial on access for reasons that can be defended. The brief should record those reasons, and what would make a different option better, rather than expecting a favorable result to justify everything.

3

What is the advice service deciding?

In this case, Bridge Advice must decide next month's appointment schedule within ten working days. It can keep the current hours, commission a focused three-week inquiry into access, or propose an eight-week trial of evening sessions. A permanent expansion would need ongoing funds that have not been approved, so it is not a realistic option at the moment.

The people, places, organizations and figures in this document's examples are fictional.

The existing booking record holds 60 recent requests from different callers. Twenty callers said directly that the current weekday hours made it hard for them to attend. Thirty could use the present schedule, and ten have no information recorded about when they are available. These counts describe these requests. They do not show how common unmet need is across the community.

The proposed trial moves twelve staff-hours a week from routine weekday appointments into two evening sessions. Two workers covering two three-hour sessions take twelve staff-hours. Staff, premises and supervision are confirmed for eight weeks. Urgent daytime triage, the sorting of cases by urgency, is covered by separate shifts and would continue, but waits for routine daytime appointments could still grow.

The team expects the trial to work in a specific way: evening appointments could help some callers who cannot come in during working hours to get advice. Travel safety, caring responsibilities, work schedules and the kind of problem people have could limit that. The record does not show that there is demand for evening appointments, that people would attend, or that more problems would be resolved.

The manager can propose changes to the schedule, but the director must approve the trial and the staffing arrangement. The team needs a reasoned choice before the deadline, not a declaration that Bridge is effective. Because the access question is narrow and the operating conditions are confirmed, the evidence that is available is relevant to a possible next step.

4

How could the available options work?

Bridge Advice optionWhat must hold for it to be feasibleMain uncertainty or consequence
Keep weekday accessExisting commitments stay workableWeekday barriers may continue
Run the limited evening trialConfirmed staff time, approval, protected urgent access and enough people actually attending in the eveningTwelve staff hours a week are taken from daytime work
Look into it before changing hoursA useful check can be finished before the decisionThe delay and the burden of collecting information may outweigh what is learned
Make a larger commitmentFunding, capacity and the underlying case on access support itA small inquiry into access does not justify an unfunded lease

For the planned eight-week trial, twelve staff hours a week comes to 96 staff hours over the trial. This is a planning assumption, and those hours are a real cost.

5

Why choose a bounded trial rather than wait?

Bridge prefers the eight-week trial for three reasons. It responds directly to a barrier that callers reported, it uses temporary capacity that has been confirmed, and it lets the team learn whether people actually receive advice without committing to a permanent expansion. It also has a real cost, because twelve staff-hours of routine daytime work are moved each week. The proposal must look at that burden, instead of treating the trial as free because no extra payroll is planned.

Keeping the schedule protects the current service but does nothing about the barrier people reported. A three-week inquiry could make people's preferences and transport problems clearer, but its report would arrive after the ten-working-day deadline for the schedule, unless the institution deliberately moves the decision date. An inquiry into whether problems are eventually resolved would take even longer, and it would not directly show which opening hours are workable.

So the proposal starts with a scheduling check with the twenty callers who reported the barrier, using a way of contacting them that is accessible and permitted. This check confirms whether they could use the proposed sessions. It is not a survey of demand across the community. If their circumstances do not fit evening sessions, the team changes the hours or turns to another option before the trial starts.

The director gives approval only after confirming that the sessions fit in practice and that the stated arrangements for staffing and triage are in place. During the trial, the service records the appointments offered and the appointments that took place, waits for routine daytime appointments, staff workload, and relevant accounts from participants. A review at four weeks can adjust the sessions or stop the trial, and a review at eight weeks considers whether to continue.

The evidence supports a limited proposal with conditions attached. It does not support a permanent expansion, or a claim that the change causes more problems to be resolved.

6

What would make another option preferable?

The brief should set out the conditions that could overturn its preferred recommendation. Suppose the check shows that most of the twenty callers need help during the day to contact public agencies while those agencies are open. An evening appointment might be more convenient, but it would not let them take the step they actually need. A daytime callback arrangement could suit them better.

Or suppose the premises are not available without an expensive new lease. Then the trial's basis in confirmed capacity disappears, and a three-week inquiry or another way of delivering the service may be better. A plan should not keep its recommendation after the condition it depends on has failed, just because the original evidence looked encouraging.

During the trial, waits for routine daytime appointments might rise a lot, or staff might report that their workloads are unsafe. Agree before the trial starts how the responsible person will respond. The team might bring back some daytime hours, reduce the evening sessions, or stop and redesign. One automatic numerical rule cannot capture urgency, access and workload all at once, although specific operating limits can be agreed where they make sense.

A good-looking trial result also needs a careful look. Higher attendance may come from existing users switching times, rather than new people getting access. People who book appointments may be different from callers who still cannot attend. And receiving advice does not show that people benefited later. Ask whether the trial worked in the way intended, and what else changed.

These checks are there to support action, not to cause endless hesitation. They make the proposal easier to oversee and to revise. The institution's choice includes judgments about access, continuity and how much burden is acceptable. Record any serious disagreement and the director's reasons. Agreement in a meeting is not evidence that the account of cause is correct.

7

Can the review actually change the work?

Before the trial starts, turn "we will review the trial" into an arrangement that someone can actually carry out. In Bridge's proposal, the supervisor brings figures on appointments received, daytime waits and staff concerns to the director at week four. The director can bring back daytime hours or stop the evening sessions, and staff know how to raise an urgent problem before that meeting. Confirm that the timetable and funding allow those responses.

Also record the reasons that could support carrying on. These are that the intended callers can use the hours, that people actually receive advice, and that the daytime work given up and the staff workload stay acceptable under the agreed arrangements. Each of these is a question for judgment.

If something important changes, such as the premises becoming unavailable, look at the proposal again and record why. Agreeing the review conditions in advance helps you resist reinterpreting results in a convenient way. But it does not make every change of plan dishonest, and it does not override lawful authority. Keep the original expectation, the new evidence and the reasons for acting differently. A review with no owner, no usable information and no realistic response gives little protection just by appearing in a plan.

8

When does the same uncertainty demand more inquiry?

Now consider a different proposal: to close the only local advice office and replace it with remote support. The saving in running costs may be clear, but closing the office could cut off people who have no device of their own or no reliable internet connection. Reopening premises and rebuilding staff relationships could be hard.

A good satisfaction result for remote services among people who already use them online does not show that the people affected by the closure could use the replacement. The uncertainty that matters is whether access and support would carry over for that group, with the consequences for continuity. The institution needs more evidence here than it needed for a short evening trial that people could choose to use.

A focused inquiry into access could combine suitable administrative information, careful and planned conversations with people who do not use online services, and a look at workable alternatives. It needs enough time, permission to use the sources, and the right skills. If the funding deadline comes sooner, the institution should look at funding a bridge period or another lawful and workable transition, instead of pretending that the inquiry will be ready in time.

Safety sets another limit. A trial that changes clinical treatment, or exposes people to serious safeguarding risks, cannot skip the proper professional and institutional requirements just because it lasts only eight weeks. The right expertise and approvals may be essential conditions for any action.

The point is to match the evidence to the actual option and to the risk people would be exposed to. Stronger inquiry can be worthwhile when the consequences would be hard to contain and the uncertainty could change an important choice. Waiting also needs reasons, someone responsible and a workable plan to keep services going. Waiting is an action with consequences. It is not the same as making no decision.

9

Which uncertainty could actually change the proposal?

Test the proposed evening sessions against the conditions they depend on. Can the people who reported weekday barriers actually use them? Can staff run them while protecting urgent and daytime support? Can the person with authority approve the arrangement? A "no" to any of these changes or blocks the proposal. Another satisfaction percentage would not settle any of them.

Write down what the owner would do in response to different credible findings. Keep the acceptable level of burden and the service's priorities visible as judgments. Evidence does not make those judgments for you.

10

What does statistical uncertainty contribute?

A result can be statistically inconclusive and still be consistent with a benefit or a harm that matters in practice. That is different from an estimate that is precise enough to show the effect is close to a change too small to matter. Before you ask for a larger study, understand what the estimate and its uncertainty allow you to say, given the study's design and the assumptions behind the analysis.

A difference that is statistically detectable may still be too small, too costly or too poorly matched to the population and outcome to justify the proposed action. A confidence interval does not account for every bias in measurement, selection or the way the program was run. Statistical precision on its own cannot show whether another setting could recreate the conditions that produced the result.

Bridge has descriptive booking information, not an estimate of causal impact. Adding a confidence interval to the percentage of callers would not answer the central questions about access and practicality. The check that matters is whether the proposed hours can reach the people who reported the barrier, and whether moving daytime capacity has consequences that are not acceptable.

If the choice really does need an estimate of the program's effect, get a suitable design and advice on methods. A convenient study of delivery cannot take its place. On the other hand, do not demand an impact study just to decide whether an approved, limited scheduling trial is worth looking at.

Formal value-of-information analysis can help with some clearly defined decisions, when believable probabilities, consequences and costs are available. Without those inputs, any score would be made up. Its comparison in words makes the assumptions and practical consequences visible. Any further numerical analysis should improve that reasoning, rather than hide unsupported assumptions inside a formula.

11

Can you finish this first-pass proposal?

Here is a practice case. A reading club could add a Saturday session for six weeks, using volunteer time that has been confirmed and premises it already has. Ten parents have said that their work clashes with the weekday sessions. Based on those ten accounts, the coordinator proposes opening on Saturdays permanently and ending one weekday group. There is no evidence that all the families affected could come on Saturdays.

The partly completed brief reads:

FieldFirst attempt
Desired changeMore children can attend
EvidenceTen parents report weekday conflicts
Proposed optionPermanent Saturday opening; close a weekday group
Pivotal uncertaintyWhether Saturdays work for affected families
Next checkAsk current Saturday volunteers if it sounds good

Before you read the feedback, propose a limited alternative that keeps the fair aim of widening access and looks at the consequences for families who use the weekday sessions now. Identify whose information could change the choice. Keep the check on whether volunteers can run the session separate from the question of whether families can attend.

Then identify the conditions under which waiting would be better. The six weeks of capacity may not be safe or usable if supervision, transport or consent arrangements are not settled. Calling something "a trial" does not settle those conditions.

A good answer compares the options that are actually workable and links its check to an important uncertainty. It does not have to agree with permanent Saturday opening, demand a randomized study of reading outcomes, or assume that every parent with a weekday clash would prefer Saturday.

12

What makes the worked response defensible?

One proposal you could defend is an optional six-week Saturday trial that keeps the weekday group going, if staffing, supervision and access arrangements allow both. First check with the ten parents whether the proposed time removes their barrier. Also look at whether families who use the weekday group would be pushed out or lose support under any other schedule.

Volunteers' enthusiasm tells you whether they are willing to run the session, not whether families can come. Attendance during a trial would show whether people received the session, and would help test whether the change improves access. On its own, it would not prove that reading improved or show the best permanent arrangement.

If running both groups needs capacity the project does not have, that proposal is not workable. A shorter schedule that alternates days might be an option, but only after looking at the disruption it would cause and getting approval. Another answer you could defend is to keep the current sessions while commissioning a focused check on when families are available, with a clear decision date. Waiting is useful when it can settle an important uncertainty at an acceptable cost.

The weakest answer jumps from ten reported barriers to a permanent replacement. It changes the commitment and puts other families at risk without checking whether the proposed action fits their circumstances. The strongest reasoning says what is known, what must hold, who might lose out and what would change the proposal.

No single answer is right in every setting. Evidence alone does not decide the acceptable balance between access and continuity. The person with approval authority should explain that judgment. The practical result is a next step that is properly overseen, with workable alternatives and reasons someone else can follow, rather than a declaration that the project has passed some abstract threshold of evidence.

13

What can your team compare in fifteen minutes?

Bring an open choice, its deadline, a summary of the sources and someone who knows the work. This first pass sets the scope of the decision.

In minutes 0–3, name the change you want, who has authority and the practical alternatives. Include carrying on with the current service, a focused inquiry, waiting, or committing in stages, where these are real possibilities. Remove any option that lacks the permission or resources it needs.

In minutes 3–6, explain how each option could bring about the change. State the conditions it needs to work, and whose access, time or wellbeing could be affected. Separate the conditions that are confirmed from those that are assumed.

In minutes 6–9, write down the strongest account the evidence supports, and the uncertainty that could change which option you prefer. Separate a gap in the facts from a disagreement about priorities. What observation would weaken the way you expect your preferred option to work?

In minutes 9–12, compare a practical check with acting and with waiting. Would the check answer the key question in time? Who bears its cost and burden? Can acting in stages limit the risk and produce useful learning, or does the change only look reversible?

In minutes 12–15, record a proposal, the approval it needs, the first action, the owner, the resources and the review conditions. If no workable option protects essential continuity, point out that limit to the responsible authority rather than inventing an easy trial.

Keep the reasons next to the recommendation. If another reasonable person would strike a different balance, record that. The brief should let the institution see the judgment it is making, instead of disguising that judgment as a calculation.

14

What does a completed evidence-and-action brief contain?

Here is the Bridge model. Use the blank brief.

FieldCompleted entry
Choice, purpose and authorityNext month's advice hours; the aim is access; the director approves within ten working days.
Feasible optionsThe current schedule; a three-week inquiry, with the decision date moved; an eight-week trial. A permanent expansion has no funding.
Evidence and scopeTwenty of sixty callers report weekday barriers; this is not an estimate of how common the need is in the community, or of any effect.
Mechanisms and conditionsEvening hours may help people receive advice; staff and premises are confirmed for a limited time; whether the sessions suit callers in practice is still being checked.
Pivotal uncertaintyDo the proposed sessions fit callers' circumstances without moving an unacceptable amount of daytime work?
Inquiry, delay and burdenContact the group who reported barriers before the trial starts; a wider inquiry would miss the deadline; the routine capacity that is moved matters.
Proposal and first actionA trial with conditions; the supervisor confirms that sessions suit callers, and checks triage and staffing, before approval.
Custody, review and alternativesThe data lead holds the booking records; the director reviews at week four and week eight; revise, stop, or propose continuing, with reasons.

At the next meeting, try to recall what would reverse the proposal before you read the model again. Then apply the same question to closing an office. Does the evidence needed change because the option and the risk have changed?

Keep the original brief together with what you observe later. Record whether the expected improvement in access appeared, what evidence pointed the other way, and why the institution changed or kept its choice. That builds learning that goes beyond one confident decision.

15

Further reading

  1. 1
    GAO, Designing Evaluations (2012), printed pages 6 and 22–23. Covers useful questions, timeliness, burden and suitable sources.
  2. 2
    HM Treasury, Magenta Book (2026), sections 1.9–1.10 and 3.2. Covers proportionality, purposes, what methods can do, and tradeoffs.
  3. 3
    CDC Program Evaluation Framework (2024), steps 2 and 5–6. Covers pathways, supported recommendations, practical options, responsibilities and use.
  4. 4
    World Bank and IDB, Impact Evaluation in Practice (2016), chapter 3. Observed outcomes and causal impact are different questions.
  1. 1
    Greenland and colleagues, Statistical tests, P values, confidence intervals, and power (2016), the discussion of confidence intervals and the conclusions. How to read a result depends on the assumptions behind the analysis, the size of the effect and the practical context.

Method and use notes

A short trial is not automatically safe, reversible or legally allowed. Report statistical uncertainty under the assumptions that actually apply, and do not treat an inconclusive result as showing no effect.

Source links need internet access.