"So What Do I Get From This?"

"Kya sir, 'log survey karne aate hain, karke chale jaate hain aur hame kuchh nahin milta', voh budda bol raha tha", the surveyor said. "usko ID card bhi nahin diya. Ab uska kya javaab de?"

Translation: "What sir, 'people come, conduct surveys and leave, while we get nothing out of it", the surveyor said. "That's what the old man was saying. He didn't even get an ID card. What can I tell him?"

He's just hit a nerve here, forcing me to try and explain what the respondents get from our survey. PIs seem to have good explanations for this sort of thing: policy relevance, extending an analogy, ensuring better delivery of services or a better allocation of resources.

It's hard for me as an RA to be able to step back and look at larger picture like that. Generally, my focus is limited to the study: the respondents, the survey and the surveyors. I have neither the knowledge of the current state of policy nor a guiding philosophy for what research should achieve; something that PIs tend to have. I am then left with this uphill struggle, trying to find something to justify the respondent's time given to the survey.

I suppose the traditional argument still holds: that by evaluating the programs we can get a better understanding of their impact and hence help promote more effective development programs. This is then helpful for the recipients of these schemes and since the respondents are typically recipients as well, they should see the benefits of the study too. Whether the costs are higher than expected benefits is not immediately clear.

It is clear that this is something we must consider and that the least we can do is to make our surveys as short, painless and unobtrusive as possible. Let's make it a point to push our surveys and PIs in this direction and make this a standard of survey design.

A surveyor engaging a respondent in a behavioural game

Stories from the Field 6871916531924152654

Post a Comment

  1. Well said Amanbir. During my two years of tenure as a research consultant 'survey length' was one of the issue I always fought for. 80 pages, 3 hrs survey doesn't make any sense.

    Not only from respondent's point of view but even surveyor tends to lose attention and focus after a certain stage. (I'd say 1.5 hour!) And after that stage everyone tries to finish questions asap rather than gathering quality data.

  2. Amanbir,
    I'd encourage you to look at the literature on reducing respondent burden through questionnaire design and question wording. The World Bank has done a lot of empirical work in this area, with some really helpful suggestions. Often, it's not the length (duration) of the survey that's the problem. In the rush to get surveys to the field and ask novel questions, investigators can design questionnaires that are inappropriate for the context they are surveying in (ie, haven't been piloted well, or have been adopted directly from questionnaires/tests used in US behavioral studies). These types of surveys place a very high cognitive burden on both investigators and respondents. Bhaumik is correct--very few investigators or respondents can manage the cognitive demands of a 1.5 hour survey on financial literacy, tests of discounting, loan and detailed employment/business details, etc. However, it is possible to get excellent data quality (and high respondent satisfaction) from much longer surveys when questions are simple and salient to the respondents. Again, I encourage you to explore the extensive literature on this (including experiments on how interview length and compensation affect refusal and data quality, etc). Best of luck.



Home item

Sign Up for Updates

Twitter Updates


Flickr Photostream