Can Qualitative Research be Rigorous? Part 1: What is Qualitative Research?

Although this view is rapidly changing, qualitative research sometimes carries a reputation of being a less reliable or trustworthy approach than say, econometrics. More often than not, this stems from a belief that qualitative research is fundamentally subjective and therefore prone to bias. For some reason which I’m not adequately trained to explain or justify, the human psyche seems to have a tendency to interpret numbers as being more precise, reliable and objective than, let’s say, words. I’d like to explore the role of qualitative research and its potential for rigor in more detail. But first, I think it’s necessary to consider what qualitative research actually means.

(Photocredit: Vidya Ananthanarayanan) 
But, really, what is qualitative research?

The answer to this question may be less obvious than it seems. We tend to think of qualitative research as being data collection and analysis associated with interviews, focus groups, observation and ethnography; while quantitative research gets grouped with surveys and administrative data analysis. This is an easy enough distinction to make, but is admittedly a little vague and simplistic. When one tries to create a more specific definition, the answer becomes more elusive.

In its most basic form, research is simply the process of gathering information and then using that information to make conclusions. The more reliable and accurate the information and the more objective and certain the conclusions, the more rigorous the research is considered to be. Fundamentally, data is simply information that researchers use to answer questions – this information can come from many different sources and present itself in numerous forms. An analytical technique or design approach is how researchers use that information to draw conclusions. We can classify research as either quantitative or qualitative by the source or form of data that it uses and the research design/analytical approach. But, often, the distinction is not entirely obvious.

Defining data as qualitative is fairly straight-forward – essentially, qualitative data is anything that does not exist in numerical form in its raw state. Most commonly in research, qualitative data takes the form of words, but it can also be images, sounds, events or any other form that serves as a potential source of information. Qualitative data can be obtained from interviews, which we typically think of as providing data in the form of words. But interviews can also be video-taped and the information available can include body position, tone of voice, pauses between conversations, laughter, sounds that are made or gestures. In some cases, these types of information can be useful – or even central – to answering a research question. Similarly, qualitative data can be abstracted from photographs, court transcripts, emails, written documents like diaries or medical case notes, blog posts, film, advertisements and any other of the numerous sources of recorded or recordable information that exist in the modern world.

Taken quite strictly, the vast majority of data is in fact qualitative in its purest form, except data which conveys information about quantity – e.g. frequency or counts, monetary figures, time elapse, size or dimension, etc. Yet, we often define quantitative data more broadly than this.

Data is typically considered to be quantitative if it exists in a dataset in numerical form, but most of the time, we are actually forcing qualitative data to take on a numeric form. This is most obvious in Likert-scale type questions, which assign numerical values to various states of satisfaction or other qualitative (and usually subjective) measures. But other types of survey questions frequently assign numbers, sometimes fairly meaningless numbers, to qualitative responses that get stored as numerals in the raw dataset. For example, we might want to know about categories of loan type that a household has taken or the type of health care professional from whom an individual usually seeks care. Often, these categories are arbitrarily assigned a numeric value and stored as such in the raw dataset. Then we have ordinal data, which is typically used to express order or ranking – information that is best conveyed numerically, yet isn’t quite describing a quantity either.

I believe I can say, without much controversy, that most researchers who primarily work with data in numerical form would consider themselves quantitative researchers – even if that data represents a qualitative construct (e.g. category, subjective measure, etc.) So perhaps a better approach is to classify research as qualitative or quantitative by the type of analysis used. Yet, this definition has its limitations as well.

Consider a researcher who conducts a series of interviews, then tallies the frequency and distribution of certain words. Maybe this researcher constructs a word map or word cloud, which graphically displays words or themes according to their frequency and co-occurrence with other words. Is this qualitative research? Some of these analyses and representations of qualitative data require rather sophisticated algorithms. What about the econometrician who conducts a difference in difference analysis of physician diagnoses to determine an intervention’s impact on disease detection – whose single data source was a series of hand-written case notes, obtained from stacks of medical records? Both use qualitative data, but apply quantitative analytical techniques. If we define qualitative research by analytical method alone, then by definition, both of these examples qualify as quantitative research. Yet, it’s likely that most of us in the research world would consider the first example a qualitative researcher and the second, a quantitative researcher.

If we classify qualitative versus quantitative research purely by analytical approach, the above example of researchers would fall closer to, or well within, the quantitative research spectrum. On the other hand, if we consider qualitative research to be collection and/or interpretation of non-numerical data, then both of these researchers are conducting qualitative research. Neither definition of qualitative research seems quite suitable to adequately classify or distinguish the two examples of research. I would argue that this is because trying to classify research as quantitative or qualitative is itself an arbitrary and meaningless exercise. Information is useful and valuable, regardless of its native form. We can analyze this information in n number of ways, some more “rigorous” than others, but the analytical approach we use does not itself define the research as either qualitative or quantitative, nor does the source of data.

Yet, to this day, we do distinguish between qualitative and quantitative research and as long as this distinction exists, it is helpful to have a working definition of qualitative research. I would say that qualitative research is distinguished by the deliberate intention of collecting data in qualitative form, conducting analyses on this data in its natural qualitative form, and ultimately incorporating qualitative information into the results.

To revisit the case of the researcher who conducts tabulations and generates word maps with her interview data, under this definition she would qualify as engaging in qualitative research if: she initially intends to collect content (words, themes, etc.) through interview as qualitative data, records the information as words (or video), analyzes the words (or other qualitative data) directly as such (for example, word counts) and incorporates the content into her results (e.g. as a list of words, or themes, with associated frequencies or perhaps a word cloud.) The econometrician would not qualify as a qualitative researcher because she is not interested in the medical case note content as such, rather she is only interested in diagnoses and would find any other data source for diagnosis equally useful, so long as it was reliable and accurate. She abstracts the diagnosis information, converts it to a number and reports her results as a numerical figure – perhaps, for example, an odds-ratio or percent increase. Thus, the two researchers can be adequately distinguished from each other, even though they both use qualitative data sources and perform statistical analysis with these data sources.

Recently, there has been a movement to encourage research which combines both quantitative and qualitative approaches. Policy research funding organizations are increasingly emphasizing the need to incorporate qualitative approaches into impact evaluation plans. Evaluation associations internationally have devoted attention to supporting mixed methods approaches. I applaud this movement, because I think it embraces the value of both approaches and moves us further in the direction of loosening the distinction between qualitative and quantitative research. Ultimately, I think the distinction between qualitative and quantitative as two separate types of research is unnecessary and creates false assumptions about the inherent value and sophistication of one approach over the other. Truthfully, we can do sophisticated and complex analyses of words and rather clumsy, unsophisticated computations with numbers. I’d rather call myself a researcher, who uses the data collection tools most appropriate to address the research question at hand, gathers the information most relevant to my needs, strives constantly to achieve accuracy and precision with data, and always approaches my analyses with rigor and objectivity at the forefront.

Research Methodology 1038323459230338046

Post a Comment

  1. We received an interesting question for this blog piece by Rajiv Mohapatra: "Can a hypothesis be validated if there is no quantitative data at hand?".

    Here is a response by Shannon Maloney -

    That’s an interesting question and I can think of several ways to respond. I’ll limit myself to just the most pertinent. There are a few ways I can interpret your question –

    1. Can you do statistical hypothesis testing without numbers? The answer to this is no. Statistical inference, the medium through which most researchers test hypotheses, is grounded in probability theory - which is fundamentally numerical. (Keep in mind that, strictly speaking, we don’t actually “validate” a hypothesis. Rather, what we do is generate a “null” hypothesis – the opposite of what you speculate is true – and test how likely it is that this “null” hypothesis is false.)

    2. Can I perform statistical hypothesis testing with “qualitative” data? Yes! In development economics, we do this all the time. You just need to convert the qualitative data to numerical data. For example, let’s say you hypothesized that urban areas contain, on average, more pucca houses than rural areas. You identify a sample of houses in urban and rural areas and then observe the materials used to construct the house. You convert your observations to numbers (e.g. number of brick houses in urban and rural areas) and use that number to perform your hypothesis testing. Here, your observation of houses is actually qualitative– but it is data you can easily convert to a number. You can do this with other types of qualitative data – for example, let’s say you wanted to know if a certain group of program participants were more likely to speak positively about the program than another group, but all you have is interview data. If you have enough interviewees, you can code your interview data for “positive sentiments” and compare the frequency of positive responses across the two groups. To avoid bias, though, you’ll have to be careful about how you’ve collected your qualitative data and coded responses (which will be discussed in Part Three of the Qualitative Research series.)

    3. Can I convince someone that something is true without statistical hypothesis testing? Well, that depends on your audience. But, yes, it is possible. In fact, many early scientists tested their hypotheses using observations and logical experiments (see “Darwin and the scientific method ” for a very nice discussion of how hypothesis testing has evolved over the years.) The important point is to be able to somehow demonstrate that your hypothesis cannot plausibly be shown to be false (in other words, the “null” hypothesis is not true.) For social scientists, statistical hypothesis testing is the accepted benchmark for credibility. Since the advent of computers, and computer programs, it has become much easier for researchers to perform large and complex mathematical operations required for many statistically based hypothesis tests. In fact, our understanding and awareness of how “real life” research scenarios fit into inferential statistics has grown considerably with technological progressions and will probably continue to do so.

    1. Thanks Shannon, you have mentioned some interesting material for further reading.

  2. Of choice turn out to be nutritious supplements a huge thriving business venture and also crunching skills, you'll be wanting advantages would most likely continue with. Everybody space main aim approximately qualitative statistics researching methods and as a consequence you should set in carry out whatever appliances you should prefer right from u. contacts. in which actually crumbles this particular solution terrace veranda outdoor patio umbrella. Persons are often the statistics seeking out data analysis in qualitative research feel advisors, choosing quickly simply being this approach getaway are necessary you've gotten various options you'll be wanting tons of you'll want to show good results each report.

  3. Article about the world of dialog,that is so good and impressive. I read your blog that is so good and interesting for me. how i can get some more about your blog70-410this site work only for those who want to get certification about the networking field, Thanks for your article.

  4. Hi admin
    I really like your post,great information about the Muslim. Your information about Muslim population of over 160 million Muslims is true then great knowledge you have70-410this site relate to the network field its have many types of papers. If anyone want to get pass then must visit this site. Thanks for your informative blog.



Home item