Saturday, April 08, 2006

Are Databases Killing Research?

Outside of the sciences, the answer may well be “Yes.” But it doesn’t have to be.

The shift away from library research, from actual examination of books and journals—or of microfilm that shows the whole of a particular periodical—has caught my profession (“teacher, higher education”) unaware, at least in regards to the ways we expect our students to perform on research projects. Without thinking much about it, we have accepted database searches as replacements for the activities we earlier expected as part of student research. Many of us, not conversant in the latest database tools, even relegate instruction in these new research methods to others, to librarians or technical experts with a grasp of how to use them but without, too often, an understanding of the needs of the particular course—of what the professors are trying to do through them.

Because our students are able to produce results that look like the results of the past, but without acquiring the knowledge once gained through a hands-on research process, these students, more than ever before, are graduating without the ability to do even the most basic research—beyond, that is, a keyword search through a database. In fact, few of them, today, seem able to recognize research at all. They mistake simple “data mining” for research and have developed almost no ability at all to judge its value. They rarely can, among other things, rate the relative importance of publications or recognize that one should judge the whole of a work instead of simply relying on a part.

This is showing up today in an inability on the part of readers to evaluate data presented—as well as in a great deal of scandalously poor research presented as “scholarly” by more than a few American publishers. Sometimes it even seems that the simple use of footnotes—required for most undergraduate research papers but meaningless in itself—has come to signify “research” to most people, allowing readers to abandon their own duties to verify for themselves. The footnotes wouldn’t be there, many assume, unless the data had been carefully vetted.

Some of the most egregious misuses of research today may come from Regnery Publishing, presenter of works by the likes of Ann Coulter and David Horowitz, the later being perhaps the most famous user of faulty “research” in the country right now. Horowitz will take a snippet from something someone else has written and build what looks like a careful “research” apparatus around it, completely twisting what that person was trying to do in the first place so that it fits the point Horowitz is trying to make. For example, in his Book The Professors: The 101 Most Dangerous Academics in America, Horowitz identifies Priya Parmar of Brooklyn College as teaching that “proper English is the language of white ‘oppressors’" (296). The truth of the matter, according to Free Exchange on Campus, is that the comment in Parmar’s class was in regard to something someone else (bell hooks) had written concerning a poem by a third person (Adrienne Rich). Technically, Horowitz may be right that Parmar has taught that some people may see “proper” English as the language of oppressors, but that gives no accurate indication of just what Parmar was trying to teach.

Not only is this sort of misleading “research” increasingly common, but the populace is less and less able to spot it, for the training in research that should be leading them to the necessary knowledge just isn’t there when student research, done in exactly the same way with exactly the same myopic concentration on detail at the expense of the larger picture, is as poorly executed as it is today.

Bad research isn’t new, of course. One of the weaknesses of too many college research projects was always an over-reliance on libraries, on secondary data. When students take five classes over a fifteen-week semester (many of them working, to boot), they haven’t the time to do a lot of “original” research. The interview, as a part of undergraduate research, had all but disappeared even forty years ago (if it ever had been stressed, which I doubt). Few students have ever been taught to construct their own surveys, conduct them, and evaluate the results. Even more infrequently have students been asked to collect documents and artifacts that might not show up in a library or database—letters, photographs, check stubs (whatever might be pertinent). The library was always a poor substitute (though it certainly did and does have its own value as repository of secondary, and even primary, information—but it should not be seen as the sole repository of information any more than should the Internet), but students were able to learn something of the “philosophy” of research through use of its possibilities.

One of the most important points to be learned from real research is that one will always find things one does not expect, things one may not even have known existed, by following the trails that research opens up. There’s no point in the researcher knowing beforehand what he or she wants to find—for the research will change that through the very process. The same is not true for data mining—at least, not nearly so often (though the experienced researcher, one who already understands the meaning of the term, can use data mining in this way). There, the initial assumption, all too often, changes quickly into a conclusion, and the search becomes simply one for bits of information that seem (at least) to support that conclusion. When this happens, the process of learning and discovery disappears.

What can be done?

First of all, teachers might consider abandoning the traditional research paper completely. After all, judgment of it is based on results, not on process—and it is the process of research that needs to be emphasized. In place of it, a series of assignments, each centering on development of a specific research skill, could be put in place. One might be an annotated bibliography. Through this, the students may learn to consider the whole of an article or book they are “mining” rather than just picking out quotes. Another might be an interview. Through this, students might learn to test the statements that people make by having to look into the assertions of the interviewee (a necessary part of the assignment) and verifying them. A third could be development of a small survey, along with after-the-fact analysis of both design and results. A fourth might require examination of certain historical documents or items alone, followed by a look at later commentary once the initial examination had been written up and preliminary conclusions drawn. These, together (or others like them), would certainly provide students a better understanding of just what research is—and of how to do it—than the traditional research paper does in the age of Internet databases.

8 comments:

Anonymous said...

You assert that the use of online databases causes shoddy research, but nowhere do I see any attempt to explain how or why. What connection is there between the delivery method and the quality of somebody's research? How is research done online any different from research done in the library? I see a lot of lamentation about the state of student research today, but precious little evidence to connect that to online databases.

AaronBarlow said...

The answers to your questions are there in the diary, if you read carefully, but I will expand on them:

First of all, the "quality" of the result isn't the point of any undergraduate research assignment. It is in utilization of what you call the "delivery method" that the learning takes place. The simplist way to answer this particular question of yours is with one word: plagiarism. If there is no connection between the "delivery method" and the "quality" of research, then plagiarism is fine--as long as the "research" presented is good. Undergraduate research projects are meant to teach the "how" and "why" of research, not to attain results. The "delivery method," in other words, is where the "quality" of the results resides.

Databases make it possible to "dip in" to an article or book without ever considering the point of the whole. Yes, indexes do somewhat the same, but without the same blinders--even looking at an index requires seeing something else of the book. In research, context is everything--it is through examination of context that the real learning takes place. "Data mining" provides no context and, therefore, no learning--or precious little. Also, "data mining" provides little challenge to the points or assumptions of the researcher. Most student searches are tailored (unconsciously) to results that confirm the student's hypothesis.

If you have to ask how research online is different than research in a library, you haven't likely done any real research in a library. The very fact of being unable to complete the research at one work station makes a difference, and brings to the researcher an understanding of the differences in the types of sources used. Furthermore, in a library, related information is often immediately available without one havint to engage in a new search. The nature of L of C groupings, for example, brings a wide range of divergent works and opinions on a single topic together where they can be simply browsed or diligently studied.

Both libraries and databases expose the researcher to a body of knowledge selected previously or somehow ranked. In libraries, this has been done by people with skill and training that is not necessarily present in the compilation of databases (where the ranking can even be automatic). Though there is quite a bit of faulty data in a library, public search-engine databases (at least) contain much more.

"Precious little evidence"? A skilled researcher would find plenty. David Horowitz relies on recent graduates as his researchers. All one needs to do is examine his recent books, all clearly dependent on online databases, to see that the notion of research that many graduate with these days is faulty.

Anonymous said...

"Databases make it possible to "dip in" to an article or book without ever considering the point of the whole."

What you seem to be saying is not that databases CAUSE these bad research practices (doing keyword searches then not even bothering to read the rest of the article), but that databases MAKE POSSIBLE certain bad research practices. In other words they are an enabler, rather than a root cause.

I would argue that students doing bad research with online databases would be doing similarly bad research in a library. Their research might be bad in a different way, but the quality and degree of effort would be no greater.

My college career has been exclusively during the era of the internet and online databases. The only time I go down to the library to look at a hard copy is when the article I want isn't in the online database. I would never consider citing an article without reading the rest of it.

The fault lies not in these databases, but in the students using lousy research practices and our methods of teaching them.

AaronBarlow said...

Thank you, you are making my point: instruction in research needs to change in response to changing research possibilities.

When you say you would never consider citing an article you haven't read the whole of, you are setting yourself apart from a great percentage of today's college students--who do exactly that in many of their papers.

If you read my diary carefully, you will see that I never blame databases, but poor instruction and usage--just as you do.

Anonymous said...

Some of my questions about your points would be more clear if I had a better understanding of what you mean by "database". Are you referring specifically to internet search engines, or are you including in "database" library research databases from companies such as ProQuest and EBSCO? With some proper instruction, students can perform fairly rigorous research in many library databases using keywords, but even more so if they learn how to use controlled vocabularly terms. As far as primary source data some library databases can provide that as well. If you are affiliated with Penn State you may want to talk with your local librarian about their efforts to help students achieve information literacy. I do agree that in some ways it could be better communicated by a faculty member who understands the course material rather than a librarian. However, you yourself admit that you are not familiar enough with the workings of the library databases to teach them to students on your own. Perhaps you can develop more expertise - and in doing so your attitudes towards electronic research may change.

AaronBarlow said...

Actually, I am competent to teach both ProQuest and EBSCOHost--and much more--to my students (where did you get the idea that I say I am not?). I have been using online databases for more than a decade and have been teaching usage of them for five years--ever since I returned to academia. No, my concern comes from knowledge and not, as you suggest, from ignorance.

It's not "information literacy" in terms of computer usage that concerns me, but "information literacy" in terms of the information itself. Generally, what we are teaching students is how to use the technology, not why it is useful to do so (and when it is not) or what to do with the information retrieved.

When I speak of primary data, I am not speaking of anything a database can provide. Sure, there are facsimiles of original material online and elsewhere, but the experience of them is quite different from the experience of the original documents. Part of what I see as missing is an understanding of that difference.

Perhaps you haven't experienced enough research outside of databases....

Rudy said...

I tried to post this last week, but it appears to not have been approved... I try again.

As a librarian, I worry about these issues quite a bit based on what I see coming to me at the reference desk. But I (and all librarians) am in a particular bind: On the reference desk, I can only react to the questions brought by the student. And when providing information literacy instruction, I am showing how to think in order to use the tools to extract information.

My job is to make information accessible to the patrons, and to assist them in obtaining that information. The job of teaching students to convert information into knowledge falls on disciplinary faculty. Research tools are my job, you should be comfortable relegating them to me. But don't be mistaken that I will be teaching how to find articles and books and evaluate websites and create proper citations and avoid plagiarism AND teach the research process all in a single 50-minute session!

I think many librarians would be open to teaching the full research process (including developing an appropriate research topic as well as the cyclical nature of the search process and also including the assimilation of information and knowledge-growing). In full semester length credit courses.

In short, while I hear and strongly sympathize with your concerns, I just want to make sure that you aren't blaming the librarian for faculty forgetfulness about the research process and their role is teaching it.

AaronBarlow said...

Rudy, you delineate the situation vis-a-vis classroom teachers and librarians quite well. Thanks.

I don't mean to disparage what librarians do, but (in part) to encourage those of us in the classroom not to simply push things off on you. It's not your job to teach the meanings of research (unless they are doing it in the context of full semester courses, as you say) but to show students how to gain access to information as fully and immediately as possible.

Your reminder is important: the real onus remains on the classroom teacher. They library can do no more than what we (or our students) ask of them, given the context.