PART 1 - Exam Analysis

Part 1 prints test statistics for individual questions and for the test overall. If an item has been skipped, it is omitted from the print out, but subsequent question numbers correspond to their original numbers on the test. Information listed for each question includes: the percent frequency of each response (including the frequency of omitted and multi-marked responses), the correct answer, the item difficulty (percent right responses), item variance and the item-test correlation.

Item difficulty - The item difficulty can range from zero (no one got the item correct) to 1.0 (everyone gave the correct response). To maximize test variance, items should be approximately equal in difficulty level and of medium difficulty. However, in tests where guessing is likely to occur, the optimum item difficulty is somewhat higher than 0.5. An item that everyone answers correctly or an item that everyone misses is not contributing to the variability of the test scores. It is not discriminating among the students in the class.

Item variance - Item variance is a measure of the variability of the item responses. It is computed from the formula; Variance = P * Q, where P is the proportion of individuals getting an item correct, and Q is the reciprocal (1-P), or the proportion of respondents who miss the question. The item variance is maximized when P equals 0.5.

Item-Test Correlation - The item-test correlation provides a measure of the relation between the dichotomous variable (correct or incorrect response to the specific test question) and the continuous variable, the total test score. This statistic is also known as the Point Biserial Correlation. It's values range from -1 to +1, with larger values indicating that the individuals who answered that particular question correctly achieved higher overall scores on the test. The formula used to compute the item-test correlation is:

R = (Mp - Mq) * sqrt (P*Q) / stdev

where:
R = item-test correlation
Mp = Mean score for those who answer the item correctly
Mq = Mean score for those who answer the item incorrectly
P = Proportion who answer the item correctly
Q = Proportion who answer the item incorrectly
sqrt (P*Q) = square root of the product P * Q
stdev = standard deviation of the overall test scores

Total test statistics printed include:

The number of students taking the test, the mean score, standard deviation, Kuder Richardson 20 reliability score, the standard error of measurement, the minimum, maximum, range and median.

Standard error of measurement - The standard error of measurement "can be viewed as the standard deviation of the discrepancies between a typical examinee's true score and the observed scores over an infinite number of repeated testings". (Crocker and Algina, (1986), p. 150) It is the standard deviation of error scores. Each examinee has a personal distribution of possible observed scores around his/her true score, and these distributions each have a standard deviation. The average of these individual error standard deviations is called the standard error of measurement. It is computed from the following formula:

If the reliability is perfect (a value of 1.0), then the standard error is zero, and if the reliability is zero, then the standard error is at its maximum value, that is, it is equivalent to the standard deviation of the test. The standard error of measurement can be used to create a confidence interval around an examinee's observed score.

PART 2 - Summary of Student Responses

Part 2 lists the correct response for each question, followed by students' error responses. Up to 40 questions are printed on a page. The students are listed in descending order by total score, and for each student the print out lists: the number of correct responses, number of incorrect responses, rank in the test, and a list of incorrect responses (letters A - E). If the student's response was correct, nothing is printed. Other symbols printed are: "*" indicating multiple responses to a question and "." indicating no response by the student. By looking down a column under a particular question number, you can readily see how many incorrect answers were given for that question, and which incorrect response was selected most frequently.

PART 3 - Student Results

Part 3 lists test results for each student with students sorted alphabetically. The following information is printed for each student: percent score, rank, number of questions answered correctly, incorrectly, unanswered, and the number of multiple answers, the Z-score and the T-score. The Z-score measures how many standard deviations above or below the mean an individual's score falls. That is, it indicates the distance between the individual’s score and the group mean in standard deviation units. Another standard score, the T-score, is a transformation of the Z-score in which the distribution mean is transformed to 50 and the standard deviation to 10. A number of test summary statistics are listed at the bottom of the page. These include the number of students taking the test, the mean score, standard deviation, average percent score, the minimum, maximum, range and median score. See the description for Part 1 for further information about these statistics.

PART 4 - Histogram

Part 4 prints a histogram of the raw test scores. One asterisk, "*", represents one student, and the chart shows the number of students receiving each specified score in the test.

PART 5 - Student Responses

Part 5 produces an alphabetic listing of each student's responses and overall score. The student's score is printed in several forms: raw score, percent score, Z-score and T-score. The student's responses to each question are listed, and each incorrect response is flagged with a hyphen, "-".  The test answer key is printed at the top of each page.

PART 6 - Posting Sheet

Part 6 prints two lists containing student identification information and grades. The first list, Part 6-A, is sorted alphabetically and contains the student's name, the student's computer user name, the student's special code number (if assigned), raw score, percent score, Z-score and T-score. You can use this list to verify a student's special code number, or to look up a student's grade, given the student's name, ID number, or special code number.
The second part of the output, Part 6-B can be used as a grade posting sheet. It contains only the student's special code number and grade (raw score, percent score, Z-score, and T-score), and is sorted by special code number.

PART 7 - Spreadsheet File of Students' Grades

A spreadsheet file, a .csv file which contains test score information, can be sent to you as an email attachment. To use this option, you MUST code the number of the test in the GRADE or EDUCATION grid on the answer key. This .csv file can be read directly into MS Excel or WebCT.

The column headings in the file are:

User ID- The student's computer user ID, both old & new format

Test n- The student's raw score on test number "n"

Pct Test n- The student's percent score on test number "n"

Name- The student's name

*NOTE: The "n" in the column titles (Test n & Pct Test n) will be a number from 1 to 16, corresponding to whatever number you have coded on the test answer key.

Next: Statistical References Previous: Procedures