1 Driving Substantive Teaching & Learning Improvement Using the Five Sloan-C Pillars of QualityPresentation By: Jon Mott & Larry Seawright Co-Contributors: Stephanie Allen & Whitney Ransom BRIGHAM YOUNG UNIVERSITY “Making a Significant Difference,” Educause 2008 • Orlando, Florida

2 Does Technology Make a Difference?“Making a Significant Difference,” Educause 2008 • Orlando, Florida

3 Does Instructional Design Make a Difference?“Making a Significant Difference,” Educause 2008 • Orlando, Florida

4 How Can We Tell?* *How do we convince others?“Making a Significant Difference,” Educause 2008 • Orlando, Florida

5 The Traditional ApproachComparative Media Studies Long History of “NSD” But what are we really measuring? “Making a Significant Difference,” Educause 2008 • Orlando, Florida

6 Sir John Daniel* Comparing test scores is a “futile tradition.”Comparing instructional media is a “pointless endeavour.” Teaching and learning systems are complex realities. The usual result is ‘no significant difference’. We should examine “new learning technologies on their own terms.” “with the futile tradition of comparing the test performances of students using new learning technologies with those who study in more conventional ways. This is a pointless endeavour because any teaching and learning system, old or new, is a complex reality. Comparing the impact of changes to small parts of the system is unlikely to reveal much effect and indeed, ‘no significant difference’ is the usual result of such research.” Instead we should examine “new learning technologies on their own terms.” *Preface, Distance Education & Distributed Learning (2002, Published by IAP). “Making a Significant Difference,” Educause 2008 • Orlando, Florida

7 Instructional Technology OftenLarry Cuban* Study of K-20 in Silicon Valley: Teachers use technology to “maintain existing practices.” Technology is general “peripheral to the daily routines.” Teaching & learning activity is largely unchanged. “Lecturing still absorbs half to two-thirds” of teaching time. Instructional Technology Often Automates the Past! *Oversold & Underused: Computers in the Classroom (2001, Harvard University Press). “Making a Significant Difference,” Educause 2008 • Orlando, Florida

8 An Alternative ApproachDesign-Based Research Goal-Driven Results-Focused Iterative New Technology = New Possibilities “Making a Significant Difference,” Educause 2008 • Orlando, Florida

9 Goal-Driven Design & EvaluationPROBLEM GOAL Strategy Tactics SOLUTION RESULTS “Making a Significant Difference,” Educause 2008 • Orlando, Florida

10 A Framework of Goals QUALITY Learning Effectiveness Access(Flexibility) Faculty Satisfaction Cost Effectiveness Student Satisfaction QUALITY * Janet C. Moore Sloan Consortium Quality Framework and the Five Pillars, The Sloan Consortium (http://www.aln.org/publications/books/qualityframework.pdf). “Making a Significant Difference,” Educause 2008 • Orlando, Florida

11 CASE 1: Learning EffectivenessPROBLEM: Introductory Chemistry Courses Limited Lab Time & Space Cost & Safety Concerns Student “Cook-Booking” “Making a Significant Difference,” Educause 2008 • Orlando, Florida

12 CASE 1: Learning EffectivenessSOLUTION: Introductory Chemistry Courses STRATEGY: Safe, Virtual Environment TACTIC: Build 3D Virtual Lab w/ Maya & Adobe Director Semi-Realistic Environment Multiple Labs Instructor Interface “Making a Significant Difference,” Educause 2008 • Orlando, Florida

13 CASE 1: Learning EffectivenessRESULTS: Introductory Chemistry Courses Students think “more like chemists,” i.e. they think more critically and are better at solving chemistry problems Performance comparable or better than students with significant “wet” lab time “Making a Significant Difference,” Educause 2008 • Orlando, Florida

14 CASE 2: Cost EffectivenessPROBLEM: Introductory Accounting Significant Faculty Time (Limited Time for Upper Division Courses) Repetitive Instruction Redundant Remediation students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

15 CASE 2: Cost EffectivenessSOLUTION: Introductory Accounting STRATEGY: Hybrid Course TACTIC: Self-Paced CD-Based Course w/ Video & Presentation Slides Substantial “Drill-Down” Materials Supplementary “Applied Accounting” Lectures students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

16 CASE 2: Cost EffectivenessRESULTS: Introductory Accounting Costs & Time Reduced Student Satisfaction Improved Heightened Interest in Accounting Major students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

17 CASE 3: Access PROBLEM: Audiology & SpeechLimited Access to Audiometer & Patients for Practice In-Class Demonstrations Ineffective students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

18 CASE 3: Access SOLUTION: Audiology & SpeechSTRATEGY: Create Virtual Tools & Environment TACTIC: Build Virtual Audiometer w/ Maya & Adobe Director Virtual Patients w/ Customizable Profiles Self-Paced Practice Mode Instructor Assignments students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

19 CASE 3: Access RESULTS: Audiology & SpeechSubstantially Enhanced Access Students Acquire Skills Faster students / year “Making a Significant Difference,” Educause 2008 • Orlando, Florida

20 CASE 4: Student SatisfactionPROBLEM: Histology Tedious Methods for Studying Samples Limited Access to Slides & Microscopes No Contextual Support “Making a Significant Difference,” Educause 2008 • Orlando, Florida

21 CASE 4: Student SatisfactionSOLUTION: Histology STRATEGY: Self-Paced, Flexible Study Tools TACTIC: CD-Based & Online Histology Tutorial Thousands of Digitized Samples Contextual Information & Help Drill & Practice Tools “Making a Significant Difference,” Educause 2008 • Orlando, Florida

22 CASE 4: Student SatisfactionRESULTS: Histology Student Satisfaction ↑ Student Reliance on Tool “Making a Significant Difference,” Educause 2008 • Orlando, Florida

23 CASE 5: Faculty SatisfactionPROBLEM: Classroom Interaction Poor Feedback From Students During Class Low Participation in Discussion “Making a Significant Difference,” Educause 2008 • Orlando, Florida

24 CASE 5: Faculty SatisfactionSOLUTION: Classroom Interaction STRATEGY: In-Class Response System TACTIC: Standardized on iClicker Multiplatform Easy to Use Integrated w/ Blackboard “Making a Significant Difference,” Educause 2008 • Orlando, Florida

25 CASE 5: Faculty SatisfactionRESULTS: Classroom Interaction Real-Time Formative Assessment Better Class Discussion Faculty Very Satisfied “Making a Significant Difference,” Educause 2008 • Orlando, Florida

26 CONCLUSION Technology HAS a Significant ImpactThe Impact is Demonstrable Starting w/ Clear Goals is the Key Articulate Problem Design Solution Aimed at Problem Develop Solution Implement Solution Evaluate: Problem Solved? Surprise! This looks like ADDIE! One More “I” – ITERATE!!! “Making a Significant Difference,” Educause 2008 • Orlando, Florida

27 Goal-Driven Design & EvaluationPROBLEM GOAL Strategy Tactics SOLUTION RESULTS “Making a Significant Difference,” Educause 2008 • Orlando, Florida

28 Discussion “Making a Significant Difference,” Educause 2008 • Orlando, Florida

29