Navsarjan reply to objections related to methodology raised by CEPT University review of “Understanding Untouchability”

untouchRecently, the Gujarat government sponsored a review report of Navsarjan Trust study “Understanding Untouchability”, which is based on comprehensive census of untouchability across the state’s 1,589 villages. Prepared by the CEPT University, the review report, titled “Impact of Caste Discrimination and Distinctions on Equal Opportunities: A Study of Gujarat”, finds fault with the methodology of the Navsarjan survey, which was carried out with the support of Robert F Kennedy Center for Justice and Human Rights (2010). Following is Navsarjan’s collective response* to the CEPT report:

The CEPT report identifies several issues related to the actual implementation of the untouchability census in their section titled “Numerator analysis” (page 3).

First, the enumerators being Dalit community members, which could result in over-reporting. If this were true then we should have seen high discrimination scores across all of the 1,589 villages. But we do not. Instead we see variation. The minimum score on the index is 0.125 and the highest is a 1. About 118 villages or about 7.4 per cent of the villages have score less than 0.5. About 223 or 14 per cent have scores greater than 0.5 and less than 0.75.

Secondly, and relatedly, while the Navsarjan study is transparent about who the enumerators are, it is not very clear as to who the “participant observers” are in the CEPT report. They say “qualified and experienced field observers were stationed in the study villages…”  (page 7).  This does not tell us anything about how their own observers are better than the enumerators in the Navsarjan study.  Additionally, we do not know their backgrounds such as caste and gender. For instance, observations could be different contingent on whether the observer belongs to a higher caste or a lower caste or a man versus a woman. On what grounds the CEPT report is assuming the uniformity of observations in its own study?

Thirdly, one finds it problematic that the senior members of their teams discussed their project with members of the village panchayats “to seek their cooperation” (page 7). They do not discuss the implications of the caste make-up of the panchayats. For example, one does not see why a panchayat composed of upper caste members would cooperate with those interested in studying discriminatory behavior by members of their own group.  Also, as the local panchayats can have considerable influence, it is plausible that they were able to constrain what the observers were able to see.

Fourthly, regarding the effect of pre-testing on the target group, the CEPT report says that “a bias has already been established among the community at the onset of the survey by mobilizing them through questions regarding a collective disadvantageous position” (page 4).

  • First of all, it is not exactly clear as to how this bias is introduced. The survey instrument has questions about whether certain practices exist or not. It is essentially probing how people live on a daily basis. How this translates to the community feeling that they are disadvantaged is not clear especially when they have been living in comparable ways for about 3000 years.
  • And secondly, we know from the Navsarjan report that there were 56 pre-tests in 56 villages that is one pre-test per village at the community level. There were a total of 1,344 villages where the census was conducted at the community level. Even if there was a bias introduced through sensitizing the target group, it affected about 4 percent (56/1344) of the total number of villages surveyed. How would the discrimination be explained in villages where such a pre-test was not conducted?

And, lastly, this is just a clarification, but on page two the CEPT report states that the Navsarjan study involved 1,589 villages and 5,462 respondents. The first number is correct but the latter is not. There were 5,462 census forms filled out. Each form had information collected from 18 distinct individuals. In total there were 98,316 respondents.

We would also like to highlight certain other aspects:

  • Are restrictions on temple entry, touching worship articles and participating in religious services all the same, as the CEPT report seeks to assume? They are closely related, but they do not have precisely the same information. This is the difference between theory and practice. As regards temple entry and religious services, about 8.5 percent of villages experienced one restriction, but not the other. As regards temple entry and touching religious articles, about 7 percent of villages experienced one restriction, but not the other. As regards religious services and touching religious articles, about 4 percent of villages experienced one restriction, but not the other. Across the three events, 10 percent experienced some, but not complete discrimination.
  • The CEPT report suggests that if Dalits cannot drive through or carry funeral processions through non-Dalit areas, then they should also not be able to rent houses there, thus that is redundant. But in our sample 15 per cent of villages experienced one form of discrimination and not the other.
  • The CEPT report says that considering all possible pairs of variables, from 98 variables, there are (98*97)/2 = 4753 different unique pairs of variables. Of those, precisely zero (0) have exactly the same information. In our survey, nearly 70 percent of the pairs of variables have fewer than 50 percent of observations with the same profiles. Around 95 percent of our variable pairs share fewer than 80 percent of observations with the same profile. The idea that we are double counting variables, while perhaps theoretically appealing to our critics, is simply empirically false.
  • The CEPT report argues that enumerators belong to the SC, so that could result in bias in that they will probe for answers. It also indicates that the participant-observers were carefully trained. We also trained enumerators, why do they believe their participant-observers can follow the rules they set out and the SC enumerators cannot?
  • The CEPT report believes that having non-SC participant observers could also impose bias in that it could encourage some people to modify their actions; and also that we did not consider the situation of non-Dalit discrimination at the hands of the Dalits. No, we did not; that does not invalidate our study. Not many people study the extent to which slave owners were discriminated against by blacks either, but that does not mean that studies of exploitation of black slaves prior to emancipation were not meaningful.
  • The CEPT report says that age-group stratification is inappropriate because different age groups experience their surroundings and interactions in different contexts; and that our survey focuses on the private domain, but should look at the public domain where government has the ability to make changes. True it does focus on the private domain; false that it should look at the public domain. Murder is also an activity that happens in the private domain, but on which the government brings considerable resources to bear.

Our goal was not to account for the particular magnitude of untouchability/ discrimination in any particular place. Rather, it was to develop a reliable measure of discrimination that could tell us about relative discrimination across villages. To do otherwise, we would have had to claim we had an exhaustive (or at least randomly sampled) list of practices (random with respect to severity) and we don’t think we did/ wanted to.

* Collective response to the CEPT report has been prepared by the Navsarjan Trust, Ahmedabad, in collaboration with two professors involved in the Navsarjan study, Dave Armstrong, assistant professor, department of political science, University of Wisconsin, Milwaukee, and Christian Davenport, professor of political science, University of Michigan


One thought on “Navsarjan reply to objections related to methodology raised by CEPT University review of “Understanding Untouchability”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s