Friday, April 14, 2006

Real Research Versus Horowitz "Research"

According to Adam Smeltz, writing in the Centre Daily Times regarding David Horowitz, who was on campus at Penn State yesterday:
College Democrats President Alex Smith, said Horowitz does not poll students to determine their true sentiments. "He listens to those few students who tell him what he wants to hear," Smith said.

This is a key point in understanding just what it is Horowitz does, and why it is not research. It is related to Horowitz’s ridiculous claim (backing by an anonymous “Professor X” notwithstanding) that he is using a methodology called “prosopography” in his book on college professors. In neither case can any reasonable conclusions be drawn from the “research”: not only is the sampling biased before the start (Horowitz has selected the professors to examine beforehand, based on certain political criteria, obviating any possible “prosopography” value—and only right-wing students talk to him), but no attempt is made to collect primary data to verify the original sampling.

All Horowitz does is amass secondary information and sort it for nuggets that support his initial assumption. There is no real research involved and no attempt at rigor in analysis.

Horowitz, however, does try to justify his limited exposure to students by arguing that they don’t know what’s in their best interest, anyway. According to Smeltz:

Horowitz suggested that victims of political bias may be unaware of their rights and under-reporting the bias incidents as a result.
Several times, Horowitz verbally assailed students who posed critical or repeated questions.
"You do not have the mental capacity to understand," he told one. To another, he said: "You are deaf and brain-dead."

As most students, in Horowitz’s view, don’t have the ability to understand or defend their rights, Horowitz has to do it for them. He doesn’t need to listen to them or find out anything about them—or about the professors who are “indoctrinating” them.

This is not a good attitude to have when promoting oneself as a “researcher,” as Horowitz does.

There’s an easier way to discredit the Horowitz and the research behind his recent book, however, than simply looking at his attitude and the very lack of credibility it evinces. And that is to show what a reasonable research plan might be, if one were really going to study bias in the classroom.

You could even start with Horowitz’s own hypothesis: certain professors use their college classrooms to indoctrinate students into leftwing ideological patterns of thought (creating, as Horowitz himself says at the end of his introduction to The Professors, that supposed "enormous damage that several generations of tenured radicals have inflicted on our educational system" (xlvii)). From that, you could even identify (as Horowitz does) a list of professors to be examined for verification (or falsification) of the hypothesis.

So far, no problem.

It’s at this point that a certain amount of academic rigor becomes necessary, and it is this rigor that is lacking (among other things) in what Horowitz has done. Research—real research—needs to go beyond the prima facie cases Horowitz builds from secondary data.

Design for a project of this sort is not particularly complicated, nor does it require a great deal of sophistication or training. Most of it is simply common sense.

The first thing to do is to collect data—primary data (and not hearsay). The list of professors has been assembled, mostly through anecdotal evidence coming from disgruntled students, from personal interaction, from Internet searches, and from news reports. Instead of simply assuming the accuracy and completeness of this secondary data, the researcher needs to move to a real research phase. This is the step—a necessary step in research—that Horowitz does not take.

What might the primary data be? First of all, every professor leaves a paper trail, and much of that is easily accessible. Some of the information is even available on the Internet, other from department or university offices. Other pieces might have to be collected from students enrolled in the classes. It may not be easy to get hold of every piece of relevant paper, but good researchers (and reporters) find ways.

The first items to collect are the syllabi of every class the professors under consideration have taught for the last (say) five years. If the courses were designed by these particular faculty members, they had to go through a process of approval. The paperwork for that will be available, too. If handouts can be found, those should be collected, too. Also, paperwork pertaining to other professional activities might be found, and might prove relevant to the classrooms. Once this data is organized, a great deal of information can be extracted. What are the reading lists like? Who are the publishers? Who else uses these texts? What types of assignments are expected of the students? What are the stated goals for the courses? How are they organized to meet those goals? All this and much, much more.

The next step would be to try to interview each of the professors, using the both the data that sparked their inclusion and that gained from examination of the paperwork associated with their classes. Some professors would refuse, and that would be too bad, but most would welcome the chance to expand upon their pedagogical choices. Once this was accomplished, an attempt would be made to interview colleagues, especially those who have observed the classes of the teachers under question. Again, this might not always be possible, but the attempt should be made.

Next, students in the particular classes (along with those who have taken them in the past) would need to be surveyed. The protocols for managing this might prove difficult, but it can be done.

None of this would be easy, and bureaucratic barriers would have to be breached, but most of it could be accomplished—even by David Horowitz—if the design were clear (it is important for people to recognize that the research is honest; a good design will help indicate that).

Once this primary data had been assembled, each case would have to be examined against a carefully constructed list of attributes indicating possible indoctrination in the classroom. To be viable, this list would need to be vetted by outside figures, including people from extreme political positions. In other words, it could not itself show a political bias (it could not, for example, include questions like, “Is Marx studied in the course?”). The list could be assembled, in part, through study of indoctrination techniques ranging from the “mass line leadership” programs of Maoist China to the methodologies of the Scientologists. This study should be part of the project and presented with the results.

Only then could real conclusions about “the professors” start to be reached.

Horowitz hasn’t even attempted such a process. He has simply taken data that could possibly be used as the starting point for such research and called it “conclusions.” Clearly, he never intended real “research” at all. Most likely, he is simply trying to foment distrust and antagonism towards American universities—for motives that have nothing to do with the true state of affairs on our campuses.

Again, Horowitz doesn’t do real research for a reason: he isn’t interested in the truth, but in furthering a certain political agenda. If he did do the research, and it did not come out the way he wanted, he would feel he had wasted his time.

So he doesn’t bother.

No comments: