Chapter Eleven. Grading and Evaluating
Grading, in its various forms, is a constant. You grade in your office, in the library, and at home. You grade in cars, trains, and planes; at soccer tournaments, professional meetings, and on the picnic table out back. There is always written work for you to read, evaluate, critique, and react to. The work you read and grade takes many forms and is produced at many different levels of intellectual sophistication. Examples follow.
Homework assignments. Common in elementary language classes. Simple but endless.
Quizzes. Primarily in elementary language classes.
Full-period examinations. Most courses involved at least one major examination, and most such exams included one or more essays.
Final examinations. Up to three hours in length, they often included a combination of shorter, more fact-oriented questions and longer essays.
Short papers. In some courses, two or three shorter papers, each of three to five pages, worked better than a single longer paper.
Term papers, or more extended papers, were common in various courses, among them advanced undergraduate classes. In some cases, it was possible to have undergraduates carry out original research; that would involve careful supervision by the instructor. If the papers were ten pages long, and one had forty students–a common number in classics courses–the instructor had some 400 pages of papers to read. Longer papers, often twenty pages or so, were common in graduate classes where enrollments were smaller.
Graduate written examinations. Candidates for Master’s degrees took a four-hour translation exam and a four-hour essay exam on literature. PhD candidates in the languages took four four-hour exams (Greek translation, Latin translation, Greek literature, and Latin literature) and a two-hour examination on a special field such as epic poetry. For candidates in classical archaeology there was a similar series of examinations, but with different content. All graduate examinations were read by at least two members of the faculty.
Honors thesis. Undergraduate students who wished to graduate with Honors wrote a thesis of some thirty to sixty pages. The work was supervised by a committee of three, of whom one, the director of the thesis, worked closely with the student throughout.
Master’s thesis. A major research paper of original research, written under the supervision of a committee of three, of whom one, the thesis director, read every draft of every chapter [1].
Doctoral Dissertations. PhD candidates wrote a dissertation, a substantial, essentially book-length, piece of original research. A committee of five faculty supervised the work of the student: a director and four readers. In general, the fourth and fifth readers did not read the whole dissertation in each of its drafts, but worked closely with the student on those sections in which they were particularly interested. The director and second reader generally read every draft of every chapter.
Drafts of articles, chapters, and books. Faculty and graduate students who were writing an article, chapter, or book would commonly ask others in the department to read and comment on drafts of their work. We did this critiquing to help one another and to help our students as they began publishing their work.
Published works of younger members of the department who were being considered for tenure. A committee of three would read and assess all of the published work of a colleague who was a candidate for tenure. See further on this process in Chapter Twelve, “Research and Tenure.”
Published works of associate professors in the department who were candidates for promotion to full professor. A committee of three read and evaluated the work of such candidates.
Published works of faculty in the department whose research was being reviewed as part of the process of post-tenure review. A committee of three read the published works of tenured faculty. Post-tenure reviews were not introduced at UNC until the 1990s; from then on, each tenured faculty member was to be reviewed every fifth year.
Published works of faculty at other institutions who were being considered for promotion to associate professor with tenure or to full professor. Just as at UNC, the teaching and research of faculty eligible for promotion to full professor was, at most colleges and universities, reviewed very carefully. As part of such reviews, the institutions involved asked four faculty members at other institutions to read and write assessments of the published works of the faculty concerned. Our faculty were often asked to serve as such outside referees for faculty at other institutions. This could involve quite a lot of work if the candidate had published a significant amount. There was ordinarily no pay for such work.
Drafts of articles submitted for publication. Many professional journals asked two or more experts to read and judge the quality of articles that were submitted to them for publication [2].
Drafts of books submitted for publication. As with articles, publishers regularly asked two or more experts to read and judge the quality of book-length manuscripts that were submitted to them for publication [3].
In sum, if we imagine a hypothetical month, it was entirely possible to find yourself reading, more or less at the same time, quizzes, tests, and paper proposals from the students in courses you were teaching; a first draft of part of a Master’s thesis; a chapter or two of a dissertation; the work of a junior member of the department; the work of a senior member of some other department; and articles or chapters that scholars in other institutions had submitted to journals or publishers for publication.
Final exams and many full-period exams were usually written out by the students in booklets of twenty-four pages with light blue covers, which were known, universally if not very imaginatively, as “blue books.” During the final exam period at the end of each semester, one often saw professors and teaching assistants walking along with a stack, sometimes small but often large, of blue books. The books, and the pages in them, were about seven by nine inches, and the amount of text a student could cram onto one page varied greatly, since some students wrote in large looping letters on alternate lines, while others wrote in tiny letters, covering every square inch. I liked using blue books because you, the reader, could turn back the cover before you started reading and thus have no idea whose exam it was. This helped guarantee impartiality in grading.
Since ours was a language department, the more significant exams regularly included written essays that took the students twenty or thirty minutes to complete. As I read them, I would be asking myself a number of questions. Does the student have control of the facts? Has the student organized those facts in a way that is coherent and logical and that results in a convincing response? Does the student communicate well? That is, am I able to follow the line of thought without struggling to do so? Does the student provide solid support for whatever conclusions or general statements she makes? [4]
Once I began reading an essay, I would soon have a tentative grade in mind. “This looks like a B response,” I might think after a page or two, or “Hm. This is excellent, probably an A.” Then, as I continued reading, I would adjust, according to what the student gave me. An essay that started off well–a tentative A–might become repetitive or veer off course, become confused or just wrong. It could end up, disappointingly, as a C or a D. Some essays, on the other hand, improved markedly as they went along. Others, probably most, continued about as they had begun.
Reading research papers–in which we can include the longer term papers, theses of all sorts, and drafts intended for eventual publication–was a very different process from reading essays that formed part of an examination. Some things, granted, were the same. You looked for a clear and complete presentation of the available evidence, organized in such a way that a convincing conclusion emerged, and this was more or less what you looked for in an examination essay. But in some respects the process was quite different. Because we were trying to train students to develop sound methods of analysis that would lead to reliable results, we emphasized the need for honesty. We wanted our students to deal with the evidence in a responsible way, without distorting or misrepresenting it. To this end, it was necessary to check many, or in some cases all, of a student’s references: did the student give the correct citations? Did the source–whether ancient or modern–say what the student claimed it did?
To check references, one needed a good library. I usually read theses and substantial papers in Davis Library (or, earlier, in Wilson Library), where I could quickly find texts of ancient authors and most secondary materials, or in the library of the classics department [5]. Also, while on exams one expected the student to deal with many or most of the important points or types of evidence, but not necessarily all of them, on research papers one expected students to gather not just some but all of the relevant evidence, and it was necessary to read papers critically with that in mind. Had the writer overlooked any potentially useful sources? Would the conclusions have been different if other sources had been deployed in the argument? If we readers were to answer those questions, we ourselves had to know, at least in general terms, what evidence was available, and what previous scholars had done with the evidence. And that meant in turn that we had to read widely and keep up with new discoveries, so that we would be familiar with the essential materials in our field.
Much more could be said about reading different types of exams and papers in various specific courses, but for the most part such information would be pertinent only to my own courses; it might or might not reflect the work of other instructors. We can, however, make a couple of general observations. First, as I remarked above, reading and assessing the work of others–students above all–occupied much of one’s time, day in and day out. Deadlines could mean that one would need to do a great deal of grading in a short time. The university registrar, for example, needed to know which students had passed your courses in time for commencement exercises. (Students and parents were interested in this, too.)
Second, as I think back over the forty years when I was teaching and grading students’ work, I am struck by the amount of freedom we instructors had, or seemed to have. No directive from the administration told us the criteria to use in grading. In classics, we faculty seldom discussed in a formal way the standards we thought suitable for grades of A, B, C, and so on. We were not required, although we might be encouraged, to assign term papers in every course. On the other hand, there was constant informal discussion of these matters. At lunch and over coffee we discussed grades, courses, students, and recent scholarly work in our field. We talked with one another about grade inflation. In department meetings we discussed what made a graduate course a “seminar,” and what percentage of grades in graduate courses should be H(igh passes). These exchanges of information helped to ensure a uniformity of expectations and grading standards across the many courses we offered.
Once one had read and assessed a given piece of work, or during the very process of reading and assessing, one had to articulate one’s response and communicate it to the student, the author, or whatever interested committee had asked you to read the work. In undergraduate classes, this was usually a simple process: marking things as wrong and writing in the correct answer; writing marginal comments on essays and papers; writing a paragraph at the end of a research paper in which you summarized the strengths and shortcomings of the paper and suggested ways to improve it. In graduate courses, we entered written comments on the paper itself, and we also, at the end of the course, prepared a “report on work in course,” which summarized the student’s work, both day-to-day and on the research paper. Copies of the report went to the student and to the director of graduate studies, who was tasked with making sure that our students were making steady and useful progress toward the degree. When the work being reviewed was more complex than a paper written for a course, we readers would need to provide detailed comments and discussion. Thus, when one was directing a thesis or dissertation, one often met with the student to talk about progress, problems, and possible ways forward. All of this is predictable, given the nature of the written work. But some of our assessments needed to be more elaborate.
In those cases where you were asked to assess the published work of another scholar, whether you were on a committee reviewing the work of a UNC colleague or were evaluating the work of a scholar who was up for promotion at another institution, you were generally expected to provide a detailed written account of the scholar’s work. This would mean a paragraph on each of the scholar’s published articles, outlining the content and commenting on the quality of each article, a rather longer review of any books he or she had written, and a statement on the nature and extent of the scholar’s overall contribution to his or her field. A comprehensive report on the work of a scholar often came to several pages and could take some days to write. It needed to be accurate, complete, and fair.
When present or former students sought admission to a graduate program or an internship, or (if they were more advanced) applied for a job, or submitted a grant application, or for whatever reason needed support, they might ask you to write a letter of recommendation. Such letters were almost always a part of a student’s application for any kind of opportunity, and they were a constant cause of work for faculty. In the early years of our half-century, they tended to be rather simple: a well-known professor would simply vouch for the ability and training of his student, and that would be that. But as time went on, letters of recommendation became more detailed, providing specific evidence of the student’s strengths. They also suffered from a form of grade inflation, with a gradual shift from a frank and objective assessment of a student’s abilities to a kind of encomiastic hymn of praise. It became difficult to include negative comments at all, because such a comment might be seized upon as justification for excluding the student from the competition. In the case of PhD candidates seeking tenure-track jobs, this was due at least in part to the fact that, from some time in the 1970s on, there were many more candidates than available jobs, and search committees were eager to narrow the field of candidates by whatever means they could. A letter containing some hint of a problem might be just what they needed: seizing on the negative comment, they could rule out the candidate, regardless of the candidate’s strengths. This, of course, meant that writing letters of recommendation became an increasingly delicate matter: you wanted to be honest; you felt a responsibility to describe a student’s strengths and weaknesses alike; yet you were aware that doing so might jeopardize the student’s chances. The result was very careful letter-writing.
Ah! How simple it would have been if we had been able to write letters of recommendation such as those the Romans wrote. Here, in its entirety, is a letter in which Cicero recommends a certain Castronius Paetus to Marcus Brutus:
Cicero to Brutus, (Greetings). Lucius Castronius Paetus, by far the leading man in the town of Luca, is an honorable and dignified man, always ready to help his friends. He is, simply, a fine man and blessed with both excellent qualities and good fortune, if that is of any relevance here. Moreover, he has been very supportive of me, so much so that there is no other member of the senatorial order whom he assists as much. I therefore commend him to you: he is both my friend and a worthy recipient of your friendship. However you may be able to assist him, you will no doubt find it useful, and I will certainly be grateful. Good-bye [6].
Cicero’s recommendation is based on personal connections and the exchange of favors. It is enough if Paetus is his friend and helps him; that should qualify Paetus to be the friend of Brutus as well, and to receive favors from, and do favors for, Brutus. Cicero speaks in generalities. There are no vivid anecdotes illustrating Paetus’s good qualities, and no account of his specific activities or offices or even his services for Cicero [7]. Had we faculty in the latter half of the twentieth century been able to write such letters, we would have had a good deal more free time. But, in general, vague letters like Cicero’s were not acceptable, and ordinarily one worked hard to convey a precise sense of the subject’s abilities, accomplishments, and promise [8].
In twentieth-century America, letters of recommendation were not limited to current or recent students. As young people were hired and began to move up from assistant professor to associate professor, or as they gained experience in whatever profession they had entered, they would often need letters of recommendation, because search and review committees always required such letters. The process could continue for years and even decades. One of my former undergraduate students contacted me to ask for a letter some twenty years after he had graduated. Luckily, I remembered him and still had my records of his work, so that I could provide details. Former graduate students regularly needed letters for ten or more years after they left Chapel Hill. I myself asked my dissertation director for a supporting letter when I applied for a grant about twenty years after I received my degree. The net result of the system was an ever-increasing number of requests for letters. In the 1970s and early 1980s, we kept carbon copies of everything we wrote. The next time the person needed a letter, we could pull out our copy, update it with comments on the student’s recent work, and have the secretary type the revised version. Once computers arrived, the copies became electronic, and revising and updating was much simpler; but still one had to go over every text carefully, making sure it was complete, up-to-date, and, importantly, addressed to the right institution.
[1] Through most of our period, the Master’s thesis in classics was usually quite substantial, with theses of seventy or eighty pages being common, and a few running to 120 pages. There were occasional efforts to limit the length.
[2] In some cases, the editor and one other scholar would read the article, but during the last two or three decades of the twentieth century the most common procedure for the leading journals seems to have been to have two scholars other than the editor read and evaluate submissions and make recommendations to the editor. In general, the reviewers did not know the identity of the author, nor did the author know who had refereed the paper. Just when this system of anonymous refereeing came to be standard is not clear to me; it seems to be typical of American, not European journals; it seems to have begun with medical and scientific journals; and the system as we know it may have begun in the 1930s and 1940s. For further information, a good place to start is chapter one, “Peer Review,” in Kathleen Fitzpatrick, Planned Obsolescence: Publishing, Technology, and the Future of the Academy (New York 2011).
[3] In general, one was not paid for reading articles and recommending publication or rejection, or for reading the scholarly productions of a faculty member up for promotion. The one consistent exception was when a publisher asked you to read a book proposal, or a complete draft of a book, in order to judge whether it should be published or not. In such cases, the reader was usually offered an honorarium. In the 1990s, a typical honorarium for reading a book was (in my experience, at least) about $250, or double that amount in the publisher’s books.
[4] Other instructors had somewhat different systems. Some assigned a certain value to each point the student made. Some had mental lists of things they wanted to see in an essay, and deducted points if something was missing. There were probably as many different systems as there were instructors. However we assessed the material, though, we were all trying to discover what the student knew and what he or she could do with that knowledge.
[5] Davis Library opened in 1984. By the end of the century, it was possible to check many references online, and that made reading theses easier.
[6] Cicero, Epistulae ad familiares 13.13, written probably in 46 BC. We do not know anything else for sure about Castronius Paetus.
[7] Similar letters were written by officials to the emperors in the centuries that followed. Richard P. Saller, Personal Patronage under the Early Empire (Cambridge 1982), 106, remarks upon the “banality of the adjectives and the complete absence of…a critical evaluation” in such letters.
[8] This was not quite always the case. As an example of the opposite, I can cite one letter of recommendation for a student applying to our graduate program. The letter was written by a distinguished professor elsewhere, and it stated, approximately but in its entirety, “This is an excellent student. If you need further information, please let me know.”