"So, although the burden of argument falls on the candidate, the candidate will receive active support from department members in making a case for the particular value of his or her computer-related work--where "particular value" means `the value of a computer-related activity within one or more of the traditional categories of research, teaching, and service'" (Cross)

I'll probably give our illustrious links editor a run for his money (!), but I'm trying (as best my toes will allow) to make this piece "speak" not only in the text, but through the links and errors and sound files and wiggling toes and pop-up telnet connections to MOOs (and bot scripts if I can figure it out)--to speak to thestrangeness of this new writing environment.

Seth says something in his piece about how difficult it is to know how to value this type of work simply because we do not know what it is ourselves. We're not sure how it fits, how it should look, sound, feel, taste, etc. And I'm not even sure I want it to "fit" at this point in time. (I'm sure I'll change my mind when it's time for me to go before a tenure-and-promotion board, though!) Well, I'll post ya'll when it's up, so you can tell me what you think (Walker).

"I am wont to look now for ways of pressing incremental change--increments that will lead people to see the logical need for larger change. So, for now, we make a small change in our evaluation guidelines--with the understanding that larger changes will, of necessity, follow--if we make the right small changes. We'll see if it works out that way."

I see a world of possibility open up. It *is* hard to see where we are going with all this, and therefore, those incremental steps are crucial. Perhaps much of this has to do with trust. Trust on our part that our admins will stand behind us as we take steps and mis-steps needed for growth and change. Trust on the part of our institutions that we can substantiate our claims (Cross).

Without the freedom to err, we're stuck in the same old (linear) modalities. If we are forced to use the language of the dominant culture, i.e., the print culture, in our hypertextual writings in order for it to be valued, what will we really accomplish? Ultimately, perhaps, we need to redefine "scholarship" in terms that include experimentation (and failure). (Walker).

You express some discomfort (I am putting it mildly) with using established categories of academic activity as a basis for evaluating computer related activities, especially those like MOO-work and collaborative writing that are largely based on play and process. On the one hand, I agree: there is simply a bad fit between the traditional categories and the new activities, and I'd like to see us tear the whole thing down and build it again along more useful lines.

At the same time, as you point out, tenure is under attack as it is; and as I point out in my hypertext, the academy will resist, tooth and nail, I think, any massive revision of existing categories of evaluation.

There is, I think, another path. In the "Jester" MOO-session log, Eric Crump offers an alternative defintion of MOO activity: that is, he proposes a way of coopting official language and thinking so that it shows the value of a `non-official' (read: nontraditional) way of doing things:

Eric says to Zecular, "that may *sound* trivial, but put it in the language admins understand: improved service to students, which in turn contributes to enhanced retention and student success (see, ya just gotta say the same thing in their language :)

It seems to me that, rather than resisting creating definitions of activities--even provisional definitions--using the traditional language for the purposes of some form of evaluation, we should use the language to our advantage: we should control and redirect the rhetoric of the evaluative context. I believe (from my very few years experience in the tenure lien) in a Menshevik/gradualist approach, rather than a Bolshevik/throw-the-bomb approach: we should start to bend the rhetoric as a way of moving towards a strong revision of the whole text of evaluation. Afterall, many who aren't really sure what we are doing will admit the value of what we are doing; and if we use the traditional language to argue that collaborative, heavily process-driven work like this CoverWeb is a high-quality publication, with all the trappings of peer-review, etc., some will buy it. We must remain true to our vision, but we have to look like we are working from the tradition in some respect or there will be no hearing. Use their language to say what WE mean; by the time they figure out that we meant something a bit different from what they thought, well, we barbarians won't merely be camped at the gates; we will already be sitting in their livingrooms, sipping their sherry and paging through their coffee table books.

There are those in my department who still do not think that conference presentations should carry much weight in tenure evaluations; for most, though, that older standard is unimaginable. It changed gradually--as a new generation of faculty, with a broader concept of the academy as an inter-campus entity, came into the professoriate and took conference attendance as a mark of academic life. I believe that the same will happen with MOOs, CoverWebs, and other process-oriented online activities. Only by persistent, gentle pressure does the starfish open the clam . . . (Katz).

We have created a MOO *list for the same purpose. If possible, we would like to see if people actually utilize the resources (MOOmailing lists in this case) on the MOO, since part of the whole project is to SHOW what MOO can do.

It is very clear from the discussion so far that the brunt of our work will be to theorize the jesterly nature of quite serious work. No suprise there. However, I am surprised there is still such a widespread call for quanitative studies and student outcomes. I'm not sure these kinds of studies are what is needed at this point in time. Given the problemmatic nature of these kinds of studies, I think at most, all we can suggest is there is a need for continued scholarly work (of all kinds) to be done onMOO. There seems to be a grievious perception that no rigorous work has been done. I hope to counter that notion by pointing to excellent work in a variety of fields. As Stainless (a non-academic who works for Hewlett Packard) suggests, one of the best ways to promote MOO and work done on MOO is to get middle mgt and admins onMOO (Cross and Fuglevik).

Taking off my stodgy curmudgeon mask, I absolutely agree with you: I am annoyed, even angered sometimes, that there is no formal structure for evaluating the computer-related work I do. Lurking on listservs is not `just like keeping up with the literature'; participating in a listserv is not `just like being a member of a roundtable session at a conference' or `making a phone call to a colleague' (comparisons that colleagues have made). Attending a MOO discussion is not just a roundtable or a conference session or a seminar or a lecture. Doing the kinds of collaborative writing that takes place on some listservs (listservs where proposals, position papers, reviews, articles, or other kinds of texts are being generated, discussed and revised by the participants) is different in character from a similar activity done solely in print; and, as you so deftly point out, this CoverWeb is another example: a situation where we are sharing ideas, drafts, responses--to content, form, style. All of these activities blur the distinctions between writing and conversation, between presentation and response, between author and audience, between product and process. An example I use somewhere in my current CoverWeb draft comes to mind: if I develop, test, and revise a multimedia package that is used solely on campus in general education literature classes, it is not likely to be treated as a publication under current standards--even though I may have done more research and revision than I would have done to create a traditional article.

So how should we evaluate these dynamic, polyvocal academic activities? What should the post-modern economy of the academy look like? How do we assess the value of an activity in itself? By its results (cf. running (an activity) is valuable because of its positive effects on physical health)? But that's just a different way of evaluating the product. What constitutes a "good" dynamic, polyvocal academic activity? What constitutes a "rich" as opposed to a "poor" example? By what criteria do we judge such an activity as being good? We generally know when we've participated in a good one; but how do we know? Can we introspect carefully and arrive at criteria? Will it just take more experience--more people doing it, seeing what it is, talking about what makes it work well, experimenting with new forms--like the REINVWs and Kairos CoverWebs?

You've given this whole subject a lot of thought, Cindy--and I guess the one thing I'm left looking for in your CoverWeb piece is your thoughts on what post-modern evaluation criteria might look like--in your dream of a system of value for academic activity. How do we avoid merely treating processes as products (e.g. "Oh, that's a very prestigious venue; so whatever you participated in there must be valuable"; or, "You particpated in creating a Web-based mutlimedia pastiche/performance work with Prestigious Scholars A, B, C, and Q? Well, that'll rack up the tenure points!"). How do you think we might evaluate processes and dialogues, without turning them into traditional objects (Katz)?

It is too easy to dismiss work simply because the evaluator does not understand the work, stands outside the material conditions which brought the work into play. It is quite easy to sit in an audience during any performance and point to only that which might be known, such as an actor's box office value.

The work/play, however, goes on. Perhaps part of *how* we need to begin to value has to do with that which is not *rational* but rather passionate. Commitment to art, commitment to education comes at a very steep price for many. As Janice notes, some changes may *not* be possible from *within.* We may not be able after all to *prove* a tangible, verifiable value to the work we play. Not as in product, not as in quantifiable, rational terms. Perhaps the only value we can ascribe to the play of our work comes in terms of the actual play, the praxis, the enactment. Will the academy listen and respond? I believe many will. After all, how do we value education? Many of us believe, and tell our students, "It's not the grade that matters, but what you learned in the process." It's not in the test scores after all, but the act of pushing the Sisyphean rock back up that nasty old hill (Cross).

You said that "we may not be able after all to *prove* a tangible, verifiable value to the work we play." Exactly. Such words reveal a modernist outlook that, to me, will value product over process.

And yes, I too believe that the academy will listen and respond -- especially as more and more of us get tenure and sit on hiring and P & T committees (Nahrwold).

I've read over your comments several times and have been thinking a lot about what you said -- especially your asking for my "thoughts on what post-modern evaluation criteria might look like," your asking "How do you think we might evaluate processes and dialogues, without turning them into traditional objects?"

Hmmm -- my immediate response is that I don't think there can _be_ such a thing as postmodern evaluation criteria. In my mind, the concept of criteria = "rules," a very modernist notion, one implying that there is one _right_ way to "play the game." And of course, there's not. I think Lyotard, in _Just Gaming_, says it much better than I, and so I'll let him speak here:

"I think that dialectics is all the prescriptive authorizes. I mean by this that dialectics cannot present itself as producing a model that would be a model that would be valid once and for all for the constitution of the social body. On the contrary, dialectics allows the judge to judge case by case. But if he can, and indeed must (he has no choice), judge case by case, it is precisely because each situation is singular, something Aristotle is very sensitive to. This singularity comes from the fact that we are in matters of opinion and not in matters of truth. ... there are no criteria, nothing but opinions" (27)

Another important point that Lyotard makes is this: given that a "law, as is always the case for both Aristotle and the Sophists, is nothing but a custom[,] [i]t can be turned. It suffices to anticipate it. Anticipation alone allows the reversal of the expected relation" (79); [w]hat allows us to decide is not that which has been attained, but that which remains to be attained; it is ahead of us, like an Idea" (83). Lyotard's invoking this Foucaultian notion of reversal has significant implications for the valuation of electronic scholarship. Instead of basing decisions upon models and modernist precedent ("that which has been attained" -- for present purposes, the model of the single-authored, linear, print-bound work as the time-honored prototype from which all academic writers supposedly should work), our discpline should stop limiting itself and look ahead to other possibilities -- such as prototypical electronic scholarhip.

Now I realize that what I've just written may not sound like an "answer," and indeed, from a modernist viewpoint, it's not. I think the closest we can come to a plan for postmodern evaluation of electronic scholarship is to say, "We need to judge case by case." Although such a statement may frustrate those who are looking for clear-cut criteria, I think we gain more by staying general. Judging case by case means that those bringing their work to the evaluation table can make the case for the value of the processes they've gone through. I think that if we come up with evaluative criteria, those criteria might very easily be based upon modernist notions, which will probably be based upon print notions and therefore upon products.

What do you think? Would such an argument improve my conclusion? What else can I say to get across the point that postmodernism does NOT mean "anything goes" (which I'm afraid far too many people think) (Nahrwold)?

Reading your responses to my response, I was struck by a way to `do post-modern evaluation,' or at least something closer to that position than the rigid, oversimplified criteria we currently have and frequently complain about. Janet says,

After all, how do we value education? Many of us believe,and tell our students, "It's not the grade that matters, but what you learned in the process." It's not in the test scores after all, but the act of pushing the Sisyphean rock back up that nasty old hill.

The grade is a value placed on a product; but to evaluate "what you learned in the process" requires a discursive explanation. Discursive evaluation is the most common institutional alternative to grades--I've used it in composition courses, where I had a number of assignments on which I gave no grades, but rather gave lengthy comments--description, discussion, evaluation, suggestions for further development, revision, refinement. In one freshman writing class, I went so far as to use William Coles' Seeing Through Writing, a series of open-ended writing assignments on the literal and metaphoric relations between "seeing," "thinking," and "writing." Coles structures the course so that no grades are awarded until the end--students work on and revise 12 papers; they may hand in as many revisions as they wish; and in the end, they select their 3 best papers for a portfolio on which their final grade is based. It took the students about 6 weeks to get over their tremendous nervousness at not getting grades; at that point, they were used to writing as a non-conclusive process; they actively revised papers because they were interested in them. To the end, some students resented the whole thing: they were so conditioned to getting grades that they couldn't get over it; and even the ones who became most comfortable with just doing the process were nervous when grade time came around (BTW: as a final exam, I had them write an essay in which they made an argument, based on their portfolio, for the grade they should receive. I only had to disagree with a couple of students--for the most part, their self-assessment was very fair).

I think that dispensing with established criteria and, instead, requiring candidates for tenure to make, in a discursive, narrative statement, the case for why their work is substantive, rich, interesing, and professionally valuable--to present their statement of the case, as it were, to a jury of their peers--might at least give us a compromise position between a modernist and a postmodernist position. While it would not make the options for play as broad as those that Lyotard seems to imply, they would make the field on which the games are played much broader than the rigid realm in which we are typically confined.

To carry Lyotard's language a little further than he perhaps intended: we will always be constrained, to some degree, by precedent. As in a court of law, though each case is judged by its merits, those merits are established against a background of previous cases and decisions. It will be incumbent on the candidate to make a case based on the sorts of values that have already been established; that is, activity X is good and rich academically because of the ways in which it draws together and interconnects students, teachers, colleagues, ideas, and ways of handling ideas--but arguments will need to be based on the particular strengths and weaknesses of prior ways of drawing together, engaging, and interweaving students, teachers, colleagues, ideas, and ways of handling ideas. That is, the candidate will have to say, activity X replicates the strengths or virtues of some prior activity, while compensating for its weaknesses. We cannot understand the new but by relating it to the known.

I guess that all I'm saying, really, is what Cindy has already said:

I think the closest we can come to a plan for postmodern evaluation of electronic scholarship is to say, "We need to judge case by case." Although such a statement may frustrate those who are looking for clear-cut criteria, I think we gain more by staying general. Judging case by case means that those bringing their work to the evaluation table can make the case for the value of the processes they've gone through.

Discursive evaluation something like what I've described broadens and enriches the possibilities for play by giving the players a larger field with more options to draw on. I see two large problems with using discursive evaluation, however: there will be large institutional resistance to using the `mushy' language that such criteria will be stated in--and in a litigious age, that's a BIG problem (even though, as we all know, all language is more or less mushy); and the other problem is that it takes a lot more time to produce, read, and deliberate over a purely discursive evaluative process than a more rigidly defined process. People groan a lot when you start talking about `adequately evaluating outcomes of student learning' and using `exit interviews and other discursive instruments' to determine what students have learned in their four years (and for their umpteen thousand dollars).

Be that as it may, I think that the change will have to come; and Cindy, quoting Janet, sums up the practical truth:

And yes, I too believe that the academy will listen and respond -- especially as more and more of us get tenure and sit on hiring and P & T committees.

We SHOULD get credit for tenure and promotion for what we're doing; we teach writing as process, and this is our chance to practice what we preach.

I agree with every point.

BUT

Tenure is already under attack in the academy. Trying to change the rules of tenure and promotion to include dialogue will make tenure's position in the academy, well, more tenuous. Of course, NOT valuing important scholarship (which IS what I see us doing) puts tenure in a, well, in an untenable position!

In order to grant credit for online work in non-traditional formats, most tenure and promotion committees will want to define strict guidelines for what constitutes "scholarly" work online. Are we REALLY ready to define what we're doing?? What happens when we finally DO define it? Are we then going to stick within those definitions rather than continue to explore (Walker).

I really like what you say about discursive evaluation -- as well as what you point out as being the drawbacks of it. True, those "raised" in a modernist, quantitative environment will probably squirm at the "mushiness" of qualitative language. And true, it _does_ take more time to "produce, read, and deliberate over a purely discursive evaluative process than a more rigidly defined process." But the effort, I think, will be worth it -- just as we've found that the extra time and effort taken to place students in writing classes based on holistic essay assessment is worth it (Nahrwold).

I'm not so sure about this though. I've found in classes where I put a good chunk of the teaching load on the students (assign students to groups to lead discussion on works we're reading or ideas/themes/principles we're studying) that I have more time both to introduce material when it is most useful for me to introduce or explain, AND I have more time to provide discursive analysis and evaluation of what the students are doing--both those who lead class and those who are participating otherwise.

Hmmm. I dabble with the idea of taking on an Intro Lit section with 50 students and seeing if it really works--I've done it effectively with 30. Maybe more IS, in fact, more (Katz).

I'm of two minds when it comes to your response. On the one hand, I'm in agreement with you that institutional structures WILL need to change in order to accomodate discursive evaluation, which does take more time than "mathematical" evaluation (e.g., "you got an 89 on that last test on comma rules"). I'd say that such structures are based on the mindsets of those holding the institutional power reins, whose educational writing experiences probably didn't include discursive evaluation but lectures on grammatical and mechanical aspects of writing, followed by quizzes and tests. They're also based, as you pointed out, on the economic "bottom line." I think it's important for us to understand institutional points of view so that we can better respond to their concerns.

On the other hand, at the same time that I say "we need to understand their points of view," I'm wondering if your "idea of taking on an Intro to Lit course with 50 students and seeing if it really works" is the way to go. What if it doesn't work? Do you think that you'll then be able to go to, let's say, your dean and say, "Sorry, but we need to move the numbers back down"? Too many times I've had to fight hard to keep the numbers down in my writing classes (especially developmental writing courses); those memories are fresh in my mind and color MY mindset, I know.

As I read over my contradictory (?) response here and think of what you said about putting some of the teaching responsibility on the students, I'm reminded of what's often said about a teacher not having to read, comment upon, evaluate every piece of writing a student does. So in this case, too, "more IS, in fact, more" (Nahrwold).

And just as our students should take more responsibility for their education, so also should more of our colleagues run student-centered classroom. But this last clause opens up the whole power "can of worms." Old habits die hard, the cliche goes -- for both students and faculty (Nahrwold).

If we were to come up with strict definitions of what we do and then stick with them, I'd say we're doing little more than reproducing what those enmeshed in modernist, print-bound notions of publication have used over the years to determine tenure. What we're doing in terms of electronic publication is changing so quickly, changing every day, I'd hazard to guess, that I don't think it would be wise to attempt to define what we do, for such definitions would quickly be obsolete. Consequently, guidelines based upon those definitions would also be obsolete. So I'd say that we need to continue making the case for guidelines that are, of necessity, provisional -- subject to continual change and updating. Dialogue, then, seems a necessary part of the tenure and promotion process -- as those of us engaged in electronic publishing "educate" those sitting on P & T committees _not_ familiar with such forums.

You also said, "Trying to change the rules of tenure and promotion to include dialogue will make tenure's position in the academy, well, more tenuous." Why (Nahrwold)?

Many people already view online sites as little more than "chat rooms"--cocktail parties online. Not only do many people in the academy view obvious sites like MOOs and MUDs and hypermail forums as informal conversation and not polished scholarship, but many people even view "published" hypertexts this way. Of course, part of the reason for this is our insistence (okay, MY insistence) that what we're doing IS conversation, and that all (print as well as online) scholarship, ultimately, is conversation. Of course, as was pointed out in a recent MOO conversation, the difference between this online conversation and the traditional (print) one is a matter of speed.

The "cocktail party" view of online scholarship makes it appear--well, somehow, unscholarly. And, since tenure is already under attack, attempting to add "cocktail party banter" to the list of what is valued in the academy for tenure and promotion could add more fuel to the fires that are already attempting to consume our tradition of tenure.(Walker).

Please don't forget that what we do online, whether on MOO, email, or e-pubs has much to do with what we BRING to the conversation. Berthoff brought Ramus, Pierce and others together in a print (originally speech) medium. She brought her scholarly knowledge of their theories together to enact a print conversation. Winterowd does the same. We are not working online from a vacuum. We have spent many years at our studies, researching and writing.

I would also suggest that the attack on tenure has far less to do with our scholarly work than good old money and politics.

And one word about the view of MOOs as games. I think we are actually dealing with many people who don't even have the basic knowledge that MUDs started as games. Speed has its many attractions. We grabbed the games and made them our own. Now we can get down to the business of informing those who need to know about why they necessarily *must* value the work we play. That includes tenure. They won't know unless we tell them (Cross).

Let me add my own insistence here -- an insistence that the work we do can be, SHOULD be, FUN. There seems to be this assumption in academia that scholars shouldn't have fun (read "love what they're doing"), and that if they do, well, then it doesn't count as "real work." If such an assumption is a valid one, I'm surprised that ANYONE chooses an academic career!

We in the academy SHOULD love what we're doing (me being a bit prescriptive here, I know). Writing -- and teaching writing! -- can be fun, as can be learning. I want the students in my classes to have fun at the very same time that they're learning (I hope!) from me (Nahrwold).

And I totally disagree with those who look on fun as somehow less serious than other kinds of "work." I hope that I have made this clear--referencing the view of ethnographic research as somehow less "serious" and "scholarly" than quantitative studies, and the view of popular culture as somehow not worthy of "serious" work. I totally disagree with those who purport that "serious scholarship" cannot include wiggling toes, and that a room full of students sitting at computer terminals and laughing can't also be a room where serious LEARNING is taking place.... :) (Walker).