← MethodsSCREEN READING EDITIONDownload the PDF

Worked Case: A Tenants' Survey From Question to Brief

Eight decisions to practice before you make them for real

For
the people who will lead a community inquiry, working through it together before gathering any real answers
Includes
one case in eight stages, a decision to make at each stage, a worked answer to compare against, and questions for your group to discuss

01How to use this case

Work through one stage at a time. Read the situation, cover the worked answer, and decide together what you would do and why. Then compare. The worked answers are not the only reasonable choices. Where your group would decide differently, say why. That discussion is the practice.

Allow about two hours for the whole case, or take one stage per meeting.

02The case

Alder Court is a three-building rental complex with 96 apartments. The Alder Court Tenants' Association has a volunteer committee of seven, no paid staff and a small budget from member dues. For two years, members have complained at meetings about repairs: leaks, broken heating, pests, doors that do not lock. The landlord's office says repairs are handled promptly.

The people, places, organizations and figures in this document's examples are fictional.

The city housing department is reviewing its inspection schedule. It will decide at a public meeting in four months which complexes get more frequent inspections. The committee wants that decision to take account of what tenants live with.

03Stage 1. Sharpen the question

Committee members propose the question: "Is our landlord neglecting this building?"

Your call: Is this a good question for the inquiry? If not, what would you ask instead?

Worked answer: Not as written. "Neglecting" is a judgment about the landlord's intentions and responsibilities. A tenant survey cannot establish intent, and the landlord can dismiss the finding by disputing the word. The housing department is deciding inspection frequency, so it needs to know what tenants are experiencing with repairs, how long they wait and how serious the problems are.

A better question: "Over the past 12 months, what repair problems have Alder Court households reported, how long did they wait, and which problems are still unresolved?"

This question has a decision-holder (the housing department), a date (the public meeting), a defined group (households at Alder Court), a defined period (12 months) and a kind of question the association can answer (how many, how long, what is it like). Members can still describe the problem in their own words in the answers, and the brief can report how they describe it.

Discuss: What does the committee lose by giving up "neglect"? What does it gain?

04Stage 2. Decide who to ask

There are 96 apartments. The committee knows of 4 that are empty. Several committee members suggest surveying only association members, because they are easier to reach and already trust the committee.

Your call: Who should the survey try to reach?

Worked answer: Every occupied apartment: 92 households. The question is about Alder Court, and the housing department will ask whether the answers represent the complex or just the association's most active members. Members may differ from non-members in ways that matter. For example, tenants who have had the worst experiences may be more likely to join, or tenants who fear the landlord may be less likely to.

The committee decides to ask one adult in each household, to knock at each door at least three times at different times of day, to offer the survey in the three languages most spoken in the complex, and to record every attempt in the log.

Discuss: Which households are likely to be hardest to reach at Alder Court? What could the committee do about each? (The companion guide The Missing People Problem goes further into this.)

05Stage 3. An offer of help with consent

The landlord's building manager hears about the survey and offers to help: staff could hand out the forms with rent receipts, and tenants could return them to the management office. This would save the committee weeks.

Your call: Accept, decline or accept in part?

Worked answer: Decline the offer to distribute and collect forms, and thank the manager for it. Tenants who know their answers pass through the management office may not feel free to describe problems, or to say no. Some may feel they have to take part to stay on good terms. Either way, the answers would be weaker and the consent would not be freely given.

The committee writes to the manager to say the survey is independent, that answers go only to the association, and that tenants are free to take part or not. It adds a line to its consent terms: "Your landlord will not see your individual answers. Taking part or not will not be reported to anyone."

Discuss: Is there any help from the landlord the committee could accept without affecting consent? (For example, permission to post notices in the lobby.)

06Stage 4. Two requests for the data

Before collection starts, two requests arrive.

  1. EXHIBIT 1
    A city council member's office asks for a copy of "the full survey results" to use in a housing campaign.
  2. EXHIBIT 2
    A graduate student at a local university offers to analyze the data for free, if she can also use it in her thesis.

Your call: How should the committee respond to each, and when?

Worked answer: Both requests should be settled now, before anyone answers, because what tenants are told must match what will happen.

Council member's office: Offer the published brief when it is released, and a meeting with the committee. Do not offer individual answers or a list of who took part. The committee does not know how the results will be used in a campaign, and tenants have not agreed to that.

Graduate student: This could be valuable, but only on written terms agreed first. The committee completes the partner agreement in its study plan: she receives answers with names, apartment numbers and any identifying details removed; she may use them only for this analysis and, with each tenant's separate agreement, her thesis; quotations require the tenant's separate agreement, and the committee reviews any draft that quotes tenants for identifying details; her university's ethics review must approve her use before she receives anything; she destroys her copy by an agreed date. The consent terms then tell tenants this: "A student from [university] will analyze the answers for our report. We will ask separately whether you agree to her using your answers in her university thesis, which may be published. You can take part in our survey without agreeing to thesis use. She will receive a copy with names, apartment numbers and other identifying details removed. Only [named university roles] may see that copy, for the purposes you agree to. We will ask for your separate agreement before quoting your words in the report or thesis, and the committee will check quotations for identifying details. She will destroy her copy by [agreed date]."

The committee records each tenant's choices about thesis use and quotations and makes sure the student follows them. If the student or her university cannot accept these terms, the committee does its own counting.

Discuss: What should the committee do if the council member's office objects publicly to being refused?

07Stage 5. Something goes wrong during collection

Halfway through, a volunteer leaves a folder of completed forms on a laundry room bench for about an hour. Another tenant finds it and returns it to the committee.

Your call: What should the committee do?

Worked answer: Treat it as a breach, even though the folder came back. Following its data-ownership terms, the lead person tells the committee the same day and records what happened in the log. The committee works out which forms were in the folder. Because names were kept on a separate sheet, the forms show answers but not names; however, some answers mention apartment numbers or details that could identify someone. The committee contacts those tenants, explains what happened and offers to remove their answers. It changes the procedure so that completed forms go straight into a sealed envelope, and retrains volunteers.

The committee mentions the incident and the change in the method notes of its brief. Being open about a mistake handled well does more for the inquiry's credibility than silence would.

Discuss: Would your organization's current practices have prevented this? What would you change?

08Stage 6. Reading the results

After six weeks, the log shows:

HouseholdsCount
Occupied apartments92
Took part61
Declined13
Not reached after three attempts18

Of the 61 households that took part, 44 said they had reported at least one repair problem to the landlord in the past 12 months. Of those 44, 27 said at least one reported problem had taken more than 30 days to fix or was still not fixed. Heating and leaks were the problems most often described as unresolved.

At the sharing meeting, tenants who took part say the results match their experience. Two tenants say the survey missed something: several households fixed problems themselves rather than reporting them, because they did not expect a response.

Your call: Which of these statements can the committee make in its brief?

Worked answer: b) and d).

The committee also writes down, before deciding what the numbers mean, that it does not know whether households that were not reached had more repair problems or fewer. The brief will say so.

Discuss: The two tenants' point suggests the survey may undercount problems. Should the brief say that? How, without claiming more than the committee knows?

09Stage 7. What to ask for

The committee drafts its request to the housing department: "Shut down the landlord's operation until all repairs are done."

Your call: Is this the right request for this brief?

Worked answer: No. The housing department is deciding inspection frequency, not enforcement against one landlord, and the evidence describes tenant reports rather than code violations. A request outside the decision is easy to set aside, and it takes the reader's attention away from the findings.

A request that fits the decision and the evidence:

  1. 1
    Include Alder Court in the more frequent inspection schedule for the coming year.
  2. 2
    Inspect the heating systems and the apartments where tenants reported leaks, with tenants' agreement.
  3. 3
    Reply in writing to the association within 30 days of the public meeting, saying what was decided and why.

The committee can still raise enforcement elsewhere, with evidence suited to that question.

Discuss: What would a request look like if the department were deciding something else, such as a new repair complaints process?

10Stage 8. The reply

Five weeks after the public meeting, the housing department replies. Alder Court will not move to the more frequent schedule, because its inspection record shows few violations. The department will carry out a one-time inspection of heating systems. It does not mention leaks or the request for a written reason.

Your call: What should the committee do with this reply?

Worked answer: Record it, share it and follow up on what was not answered.

A partial answer is still an answer. The inquiry has put tenants' experience on the record, and the department has had to respond to it.

Discuss: At what point should the committee stop following up and decide the inquiry is complete?

11What this case practiced

StageThe decisionThe principle
1Replace a judgment with a question you can answerAsk what the evidence can show, for the decision at hand
2Ask every household, not just membersThe question defines who must be asked
3Decline the landlord's help with formsConsent is only free when people can say no safely
4Settle data requests before collectionWhat people are told must match what will happen
5Treat the misplaced folder as a breach, even though it came backKeeping promises includes admitting when one was broken
6Report counts, bases and limitsClaims no bigger than the evidence
7Ask for what the decision-holder can decideA request that fits the decision is harder to set aside
8Record and follow up on a partial replyThe inquiry ends with a record, not just a report

Related: Running a Community Inquiry That Institutions Can Act On; Community Study Plan; Two-Page Findings Brief; The Missing People Problem.

12Further reading

  1. 1
    Pew Research Center, Writing Survey Questions. Plain guidance on question wording, order and testing, from a large survey organization.
  2. 2
    AAPOR, Standard Definitions. How survey researchers count people who were asked, answered, declined or could not be reached.
  3. 3
    University of Kansas Center for Community Health and Development, Community Tool Box, Chapter 3: Assessing Community Needs and Resources. Practical sections on running surveys and listening sessions in a community.