1 Don Bacon & Elizabeth Anderson November 14, 2003 ([email protected])Using Computer Automation to Assess and Facilitate Improvement in Writing Mechanics Don Bacon & Elizabeth Anderson November 14, 2003
2 Overview Description of problem AssumptionsInstrument development and analysis Interventions and related results Conclusions and questions
3 The Problem Of 18 skills, employers ranked written communication skills second in need for improvement (McDaniel & White, Mkt Ed Rev, Fall,1993). Business faculty may lack time to grade all papers for grammar, punctuation, etc. Business faculty (including part-timers) may be unsure of their own writing skills.
4 Assumptions Writing mechanics provide a foundation for the total product of the writing process. Non-mastery of mechanics injects interference, or noise, into the attempt to communicate, so mastery is desirable. Students who are unskilled in (or indifferent to) the importance of correct mechanics will also be unskilled in or indifferent to higher-level skills of writing.
5 The Instrument Executive Summary APeter’s Specialty Salads will be1 a quick-service salad restaurant2 located near the3 campus and will target4 it’s students,5 faculty, and surrounding community.6 Peter’s mission is to7 accommodate a fast-paced,8 on the go society9 better then its competitors10 by meeting there needs11 for a healthy alternative to fast food.12 …. up to 50 “segments”
6 Data Entry Students enter on Scantron formStudent ID number Form ID letter 1 or 2 for each of 50 segments If correct as is, enter 1; if incorrect, enter 2 Scantron forms are scanned into a computer file Students need about 15 minutes to complete one form
7 Instrument PretestingIn pretests, item-to-total correlations were examined for each of 50 segments. Ambiguous segments were edited or excluded from analysis in later runs. Reliability of .64 to .71 was achieved.
8 Analysis SPSS program reads data, computes scores, and prints a custom one-page report for each student. The report includes: Overall score Performance by topic (e.g., use of then/than)
9 Output Writing Assessment Report for: Doe, John A.Form taken: A Course: B1050 *** Your Score *** Your overall score was Good! Each of the following areas were assessed for possible improvement: Comma: Needs work That/which: OK Semicolon: Needs work Then/than: Needs work Colon: Needs work There/their/they're: OK
10 Interventions Assess, offer feedback, offer guidance (2-page handout), then retest. Assess, offer feedback, offer guidance, then retest with 5% of grade contingent on retest performance.
11 Results 1) Does ability to edit for mechanics improve as students go through four years of college? No statistically significant difference in basic editing skills related to year in school (R2 = 2.6%, n = 120). 2) Does this ability improve with feedback and retesting? (p < .01) SD = .7
12 Conclusions and QuestionsWriting mechanics skills can be improved in an automated process if students are held accountable through grades. Instruments could be customized for different disciplines Did students “improve” just because they were being more careful? Will faculty buy in across courses?