← MethodsSCREEN READING EDITIONDownload the PDF

Why Do People Disappear From the Record?

i

Examine trust, exposure, burden, and refusal before treating missing responses as a technical problem

For:program teams, evaluators, funders, commissioners, participant-engagement leads, data staff, and reviewers
Use when:people decline measurement, stop responding, leave a program, or may not feel able to answer candidly
Includes:a trust-and-missingness review, burden map, worked example, practical safeguards, and a 15-minute team exercise
Brief paths:clarify who is represented; review participant feedback; decide what to measure; understand disappointing results

A follow-up survey reaches only half of participants. The response plan calls for more reminders. Staff add text messages, phone calls, and another incentive. Response improves slightly, but nobody asks why people did not want or feel able to answer in the first place.

Missing responses can be a logistical problem. They can also be information about trust, burden, accessibility, exposure, and the relationship between the program and the people it measures.

ii

The short answer

Investigate the conditions around refusal and disappearance, not only the rate. Ask what people were told, what they believed could happen to their answers, whether participation felt optional, how burdensome the request was, and who was least able to respond. Check contact, accessibility, trust, burden and protection conditions and narrow the claim before intensifying collection. The cause of missingness remains unknown unless investigated.

The people missing from the record may be telling you something about the measurement system, even when they leave no completed form.

iii

Why people may not appear

Contact barriers

Phone numbers change, mail is unstable, work schedules conflict, transportation is limited, internet access varies, and follow-up occurs at inaccessible times or in inaccessible formats.

Burden

People are asked the same questions repeatedly, surveys are too long, the emotional cost is high, or the value of responding is unclear.

Exposure

Questions concern housing, income, health, family, immigration, justice involvement, or other circumstances that can feel risky to disclose.

Power

Participants may believe answers affect eligibility, staff treatment, benefits, legal status, or future access. A consent statement does not erase that belief.

Experience of the service

People who left early, felt poorly served, or were harmed may be least willing to remain in contact. Their absence can make the visible result look more favorable.

Success or changed priorities

Some people no longer need the program, are busy with work or family, or see no reason to continue contact. Do not assume every missing outcome is poor.

The key question is whether the reasons for missingness could be related to the outcome or experience being reported.

iv

A worked example

A peer-support program uses a wellbeing survey at intake, every month, and exit. People who stop attending are contacted up to six times. Staff notice that participants with unstable housing and people who reported negative early experiences are less likely to complete exit surveys.

The people, places, organizations and figures in this document's examples are fictional.

The program shortens the monthly survey, stops asking two sensitive questions that have no current use, explains the actual access consequences of answering or refusing, and removes a linkage to service eligibility where authorized, offers phone, text, paper, and supported completion, and asks a separate optional question about why someone leaves. Follow-up is conducted by a person outside the direct service relationship where feasible.

The program also changes its report. It shows outcome coverage and the known differences between respondents and nonrespondents. The visible wellbeing result becomes less sweeping but more trustworthy.

v

Review the measurement relationship

Ask participants and frontline staff:

Do not ask only the people who continue responding. Seek routes to hear from people who declined or left without making renewed participation feel compulsory.

vi

Practical safeguards

Explain the use plainly. Say who will see the information, what decisions it affects, and what it will not be used for.

Make refusal real. Separate measurement from service eligibility where possible and train staff not to pressure participation.

Reduce the request. Remove questions without a real decision, obligation, or learning use.

Offer accessible routes. Use appropriate language, format, timing, technology, and support.

Separate roles when useful. People may answer differently when the person measuring does not control their service.

Return value. Share what was learned and what changed because people gave their time.

Track reasons carefully. Record refusal and loss to follow-up without turning the reason into another compulsory disclosure.

vii

What you can say

viii

What you cannot say yet

ix

What to do next

Minimum useful step: Report response counts and rewrite claims to name respondents rather than all participants.

Stronger step: Examine response patterns and conduct a trust-and-burden review with people affected, including those who stopped participating where appropriate.

Prospective step: Redesign the collection purpose, consent, roles, timing, access, and return of findings. Set a review point for whether the new approach improves both coverage and candor.

x

A 15-minute team exercise

Choose one survey, interview, or follow-up process.

  1. 1
    Write what decision or obligation each requested item serves.
  2. 2
    Imagine the process from the participant's position.
  3. 3
    Mark where refusal may not feel safe.
  4. 4
    List groups most likely to face access or exposure barriers.
  5. 5
    Remove one low-value question or contact.
  6. 6
    Improve one explanation, format, or boundary.
  7. 7
    Decide how participants will hear what changed because of the information.

Do not end with a plan to contact people more often unless the team has first addressed why contact may be unwelcome or difficult.

xi

A note on surveillance

Measurement drifts toward surveillance when information is collected because it can be, follows people beyond a proportionate use, or is used for consequences they did not reasonably expect. Programs operating in child welfare, justice, immigration, health, housing, or benefits systems should be especially careful.

Ask whether the system would feel fair if the person being measured could see, correct, and question the record. Trust is not a soft addition to data quality. It affects who appears, what they say, and whether the evidence can be believed.

xii

When to bring in help

Outside support may be needed when collection involves sensitive information, legal or safeguarding consequences, strong power imbalances, persistent differential missingness, or evidence of harm. Participant-led expertise should sit beside methodological and privacy expertise.

The useful output is a measurement arrangement people can recognize as proportionate, safe enough for candor, and connected to a real use.

1

Questions to carry forward

14

Further reading

  1. 1
    American Evaluation Association, Guiding Principles. Principles A2–A5, C1–C7, D1–D4 and E1–E4: inquiry, integrity, people and equity.
  2. 2
    GAO, Assessing Data Reliability, GAO-20-283G (2019). The official overview: data accuracy, completeness and applicability to purpose.
  3. 3
    Spencer, Ritchie, Lewis and Dillon, Quality in Qualitative Evaluation (2003; Treasury republication 2012). Appraisal questions 1–2 and 12–14: evidence, contrary accounts and traceable interpretation.