Skip to main content

"So What Do I Get From This?"

"Kya sir, 'log survey karne aate hain, karke chale jaate hain aur hame kuchh nahin milta', voh budda bol raha tha", the surveyor said. "usko ID card bhi nahin diya. Ab uska kya javaab de?"

Translation: "What sir, 'people come, conduct surveys and leave, while we get nothing out of it", the surveyor said. "That's what the old man was saying. He didn't even get an ID card. What can I tell him?"

He's just hit a nerve here, forcing me to try and explain what the respondents get from our survey. PIs seem to have good explanations for this sort of thing: policy relevance, extending an analogy, ensuring better delivery of services or a better allocation of resources.

It's hard for me as an RA to be able to step back and look at larger picture like that. Generally, my focus is limited to the study: the respondents, the survey and the surveyors. I have neither the knowledge of the current state of policy nor a guiding philosophy for what research should achieve; something that PIs tend to have. I am then left with this uphill struggle, trying to find something to justify the respondent's time given to the survey.

I suppose the traditional argument still holds: that by evaluating the programs we can get a better understanding of their impact and hence help promote more effective development programs. This is then helpful for the recipients of these schemes and since the respondents are typically recipients as well, they should see the benefits of the study too. Whether the costs are higher than expected benefits is not immediately clear.

It is clear that this is something we must consider and that the least we can do is to make our surveys as short, painless and unobtrusive as possible. Let's make it a point to push our surveys and PIs in this direction and make this a standard of survey design.

A surveyor engaging a respondent in a behavioural game

Comments

  1. Well said Amanbir. During my two years of tenure as a research consultant 'survey length' was one of the issue I always fought for. 80 pages, 3 hrs survey doesn't make any sense.

    Not only from respondent's point of view but even surveyor tends to lose attention and focus after a certain stage. (I'd say 1.5 hour!) And after that stage everyone tries to finish questions asap rather than gathering quality data.

    ReplyDelete
  2. Amanbir,
    I'd encourage you to look at the literature on reducing respondent burden through questionnaire design and question wording. The World Bank has done a lot of empirical work in this area, with some really helpful suggestions. Often, it's not the length (duration) of the survey that's the problem. In the rush to get surveys to the field and ask novel questions, investigators can design questionnaires that are inappropriate for the context they are surveying in (ie, haven't been piloted well, or have been adopted directly from questionnaires/tests used in US behavioral studies). These types of surveys place a very high cognitive burden on both investigators and respondents. Bhaumik is correct--very few investigators or respondents can manage the cognitive demands of a 1.5 hour survey on financial literacy, tests of discounting, loan and detailed employment/business details, etc. However, it is possible to get excellent data quality (and high respondent satisfaction) from much longer surveys when questions are simple and salient to the respondents. Again, I encourage you to explore the extensive literature on this (including experiments on how interview length and compensation affect refusal and data quality, etc). Best of luck.

    ReplyDelete

Post a Comment

Popular Posts

Vocationalisation of education in India: Current Scenario, Key Challenges and New directions

“Every handicraft has to be taught not merely mechanically as is done today, but scientifically. This is to say, the child should learn the why and wherefore of every process.” - Gandhi’s Philosophy of Education

The greatest challenge in Indian education system today is to provide skill based education to the youth. This is exacerbated by a mismatch in demand and supply for the skilled workforce. The penetration of vocational education and training remains poor not only in rural areas, but also in urban regions where there is a higher installed capacity to impart the same. This post is an attempt to make the readers understand the need of vocational education in India. Also, this is an attempt to summarise a few recommendations on the same. 
A recent survey (61st round) conducted by the NSSO found that:

1. The percentage of population that completed primary education was 70%, but less than 10% went on to complete a graduation course and above. Almost 97% of individuals in the age bracket…

Rockstar of Financial Inclusion: Business Correspondent Model of India

About Author:  Jatinder Handoo is a social business enthusiast and a branchless banking practitioner. Currently works at FINO PayTech Ltd and is based out of Mumbai. He is reachable at jatinder.handoo@fino.co.in
India is a hot bed of financial exclusion. A country which houses nearly 16% of the global population  has more than 65% of its people outside the formal financial system (Global Findex 2012). The Indian banking system has adopted multiple approaches to make universal financial inclusion a reality right from early days Indian post-independence banking system. Be it bank nationalization in 1969 or formation of Regional Rural Banks. Formation of NABARD or fostering microfinance through Bank-SHG linkage programme in early 90’s. A shimmering ray hope was rekindled with the growth of JLG based microfinance, however later studies made it clear that the model is credit led, concentrated predominately in the southern region of India thus could not be seen as painting complete financial…

A Platform for Knowledge - Enabling people to learn ..

I received a rather interesting link/website via my email today. The link read as MR University and all I could think of was, "Ok, this must be another website portal of some university or college". Well, on clicking the link and looking through the contents of the site, I was pleasantly surprised. The site http://www.mruniversity.com/ is an online education portal or platform that allows users or teachers to upload short videos on topics or lessons they wish to impart. First topic that I come across is Development Economics.
The intent of the website is eloquently put out by the two economists, Tyler Cowen and Alex Tabarrok in the intro video. What started as a blog focusing on economics and its various implications in understanding why things are the way they are around us, has now an interesting addition. A video portal titled MRUniversity or Marginal Revolution University that focuses on online education with subjects pertaining to economics. It brought back to my mind,…