Return to One Department's
Guidelines.
Within the realm of evaluation, no matter which way we go to revise evaluation criteria to accommodate computer-related work, we will be threatening traditional practices and so will encounter resistance from those who do not understand or perceive the value of computer-related academic work. Certainly, if we retain the traditional categories of Research, Teaching, and Service, some computer-related activities may be readily evaluated as directly equivalent to traditional activities. For instance, it is easy to see how publishing in an online, peer-reviewed periodical is equivalent to publishing in a peer-reviewed print journal; making course readings available through a Web page or FTP site is equivalent to distributing dittos and photocopies; having students create Web-based projects is equivalent to having them write traditional papers; attending committee meetings at a MOO is equivalent to face-to-face meetings.
However, there are also ways in which apparently analogous online and traditional activities are not equivalent. Publishing online can and often does encourage kinds of collaboration that are unusual or difficult in print publishing. In creating this CoverWeb, for example, several authors are creating interwoven hypertexts. The particular connections among the pieces are, in part, the result of e-mail exchanges among the authors via a HyperNews forum. This combination of individual and collaborative authorship would be difficult to duplicate in a traditional print context. In coordinating the CoverWeb, Janice Walker, Janet Cross and I have, via e-mail and MOO meetings, repeatedly brainstormed, discussed, argued, and reached conclusions about how the pieces should fit together and be linked. The individual CoverWeb pieces and the introduction that Janice, Janet and I co-author might be judged simply in terms of a published text (under the category of Publication). However, the valuable and evaluatable results of these publications go beyond the text itself: the process of creating the text has forged personal, professional, and institutional connections among the participants that will have effects and implications beyond this project. This kind of collaborative work might be regarded as an elaborate example of so-called "Professional Development" (which does not fall neatly into the traditional categories). It is also a situation in which each of the participants has represented his or her home institution to the larger profession (and so is a kind of Service). And, as Cindy Nahrwold argues in her contribution to this CoverWeb, the text-record of the collaboration (in the form of e-mail transcripts and MOO logs) provides a means by which we can show and explore the process of writing: we can use the record of our work to show our students how the collaborative writing process works. Thus, online collaboration can have direct implications for our Teaching.
Furthermore, an online publication can lead to follow-up discussion more quickly and easily than can a print publication: while an exchange of letters and response pieces, or even face-to-face conference sessions may follow a print publication, an online publication can immediately lead to both asynchronous and synchronous discussion, debate, and response pieces of any length, all of which may be logged, archived, hypertextually linked, and cited in subsequent texts. Similarly, using Web-based course materials or having students create Web-based projects makes these materials more publicly accessible: Web-based materials immediately become available for public use and discussion; they become a variety of publication, whose value will partly be determined by the kind of response they receive; "Web-publication" also constitutes a kind of service to the larger community, in that it makes available materials that others outside the institution may use; and a Web-site for a course serves to represent the institution to the rest of the world. There is a qualitative difference in the kind of impact that an online publication might have on the academic community (as well as the community outside the academy) and in how that impact might be measured: the value of an article published online might be measured in terms of the scale and quality of the online debate that the article fosters -- which may have no correlation with the length of the article or the kind of research that went into writing it; the value of a Web-based course or Web-based student projects might be measured in terms of how many people from how many different institutions use and respond to them, and in terms of the kinds of comments the authors receive.
These ways of talking about an online publication -- in terms of the processes of creation and response, and in terms of professional and community effect -- don't not seem to fit readily with traditional categories and ways of thinking about evaluating academic work for tenure and promotion purposes. And yet, as Cindy Nahrwold discusses in her contribution to this CoverWeb, one of the significant features of online academic work seems to be the ways in which that work foregrounds process: the process of composition and creation, the process of reception and response. Traditional categories and criteria for evaluating academic work (especially in the category of Publication) are structured in terms of products. It may be that the real worth of online activities cannot be evaluated using traditional criteria. This notion will need more study.
Even if some kinds of online work can be adequately evaluated using analogous traditional categories, many activities do not readily fit into just one of the traditional categories, or else do not clearly fit into any of them. If someone authors a multimedia package for use in a general education literature course and, although the package is not published, it comes to be used by several faculty members teaching a few hundred students each year, does this constitute Publishing, Teaching, or Service? The kinds of work the author did in creating the package are equivalent to the kinds of research involved in composing a complicated print publication, such as a book or a series of articles (though authoring a multimedia package probably involved activities that are not usually part of conventional research and publishing: collaborating with technical support people, learning to use and compose in new software packages, seeking rights to reproduce text, images, and sound recordings electronically, working on layout and continuity, composing storyboards, etc.). The multimedia package itself may constitute innovative teaching. And producing the package for use by fellow faculty constitutes service to the institution. To treat the multimedia package as only Publication or Teaching or Service is to deny some important feature of its value -- a value that, I think, is most clear if one examines both the product and the process by which it was created.
How should moderating an online discussion be evaluated? It is more complex than chairing a conference session, and can have a much broader impact on the profession than organizing an entire conference. Nor is it the same as editing a periodical. Authoring software that may be applied to research and/or teaching in literature and composition is akin to authoring multimedia: is it Publication, Teaching, or Service? Janet Cross and Kristian Fuglevik discuss their experiences running and working in an academic MOO. Their work clearly has value in terms of academic performance, and their Research/Publishing, Teaching, and Service activities are largely based on their MOO activities. However, to those who do not know what Janet and Kristian actually do, their MOO-work seems to be no more than Service. These authors present a clear argument for both the failure of the argument for equivalencies between online and traditional activities and the failure of traditional categories to encompass online activities.
Thus, computer-related work cannot consistently and easily be made to fit into traditional categories for evaluating academic work. Computer-related work thus poses a threat to traditional modes of evaluation. So, it would seem that we must somehow revise the categories of evaluation to accommodate computer-related work. But that, too, poses a threat to traditional ways of doing things.
Return to One Department's Guidelines.
Go to
E-mail: seth@bradley.bradley.edu
Last revised February 22, 1997