Outcome Coding Scheme for Reference Questions

Developed by John V. Richardson, Jr.

Grade	Definitional Description of Reference Question Outcome	Evaluation
5.0	Referred to single source, complete and correct answer	Excellent
4.0	Referred to several sources, one of which gave complete and correct answer	Very good
3.0	Referred to single source, none of which leads directly to answer, but one which serves as a preliminary source	Good
2.0	Referred to several sources, none of which leads directly to answer, but one of which serves as a preliminary source	Satisfactory
1.0	no direct answer; referred to [external] specific source or person or institution	Fair/poor
0	no answer; no referral (I don't know)	Failure
-1.0	Referred to single inappropriate source	Unsatisfactory
-2.0	Referred to several inappropriate sources, none of which answers question correctly	Most unsatisfactory

SOURCE: Juris Dilevko and Elizabeth Dolan, "Government Documents Reference Service in Canada: Implications for Electronic Access" at dsp-psd.communication.gc.ca/Rapports/ Dilevko_Dolan/dilevko-e.html:

"Richardson's definitional descriptions were retained, but his evaluation levels were reworked and simplified into four categories. Richardson's categories of "excellent" and "very good" were collapsed into the category of "complete answer"; his categories of "good" and "satisfactory" were collapsed into the category of "partially complete answer"; his category of "fair/poor" was retained intact, but was renamed "referral"; and finally, Richardson's bottom three categories of "failure," "unsatisfactory," and "most unsatisfactory" were categorized as "no/incorrect answer." Table 3 summarizes the modifications."