1 Helping Give Away Psychological Science to Improve Teaching Assessment and Statistics: What You Can Use, What You Can Add CC-BY-SA by Eric Youngstrom, August 2016 https://creativecommons.org/licenses/by-sa/3.0/us/ Eric Youngstrom, PhD & Mian-Li Ong, MA University of North Carolina at Chapel Hill

2 Goals Focus on Wikiversity Goal: Next action: Mashing upROC Party = Better way of building evidence Wikipedia = Making the evidence available to the widest audience Focus on Wikiversity Goal: Getting people involved, organized to work effectively (Joe’s question… who will write it? Next action: Visit site Sign up!

3 ROC Underlying data (same as t-test!)

4 Ready to ROC!

5 ROC Party ROC as more clinically relevant techniqueROC Party as a collaborative teaching model flipped classroom distance learning & ask for “ROC Primer care package” Google: Wikiversity + ROC

6 Resources attached to PrimerArticle (key deliverable for academic! Often the only…) Goal = teaching, not new data findings Redundant analyses (to be able to compare & contrast) Syntax in SPSS & R Open access to data used in examples Excel sheet with formulae not in SPSS We can do better!

7 ROC Party Rationale Dissemination: Teaching:Not just for bipolar disorder If I did anxiety, ADHD… so what? If Tom Ollendick, Gerry Gioia did same… people would notice! If their team learned technique, that would be a multiplier Teaching: Experiential learning Problem based learning Could do small group, high structure, report out (flipped class)

8 Core ingredients Sample code, data, annotated figures and outputIn primer Online (DropBox OSF.IO Wikiversity) Agenda/syllabus From twinkle in eye to aims, methods, results, tables and figures done in 6 meetings Vet the data ahead of time (planning canvas) Want to ensure success experience Increases chances of conference presentation, publication Provides incentive for participation

9 ROC Party I – Summer 2014 Invitation only6 teams started projects (3 at UNC, in room, 3 long distance) 6 meetings “Classes” -- GoogleHangouts/Skype/Phone only/WebEx Asynchronous shairing: , DropBox, OSF.IO Learnings: Seeing each other’s analyses provided a lot of exposure to different scenarios Code sharing (how did you make that?) Fun! 5 are published, one cancelled due to conceptual issues

10 ROC Party II Extended network of collaborators (Children’s National, OSU, NUS in Singapore, University of Lancaster in UK) 5 meetings instead of 6 Continued hiccups with technology Again, lot of fun, lot learned Started to save process materials (WebEx recording; annotated output; responses to peer reviews (Venkatraman test) Team structure: ROC Star: Experienced ROCer, can run or troubleshoot analyses, knows how to write up Results Wingman: Understudy, may duplicate analyses – if external data, this often is a grad student or postdoc familiar with the data Big Dawg: Content expert (will make sure Intro, Discussion are current and accurate) 6 manuscripts published, 3 more still in process

11 Wikipedia: “Best of the Free” AssessmentsWrite pages for free use tools that have good score psychometrics across samples Link to copies of measures Solves Awareness and Access issues Supported by grants from SCCAP, APS, SSCP, APA CODAPAR & D12 Project Site:

12 Reaching People #1: Google #5: Yahoo #6: Baidu #7: Wikipedia GYMR#150,000+: ABCT.org, AACAP.org, SCCAP.org #150,000ish: EffectiveChildTherapy (#1,000,000+: Youngstrom.anything)

13 Wikiversity: EBA, Teaching and Clinical ApplicationEvidence-Based Assessment (EBA) overview and concepts Disorder-specific assessment toolkits Teaching Vignettes Research: Toolkit for EBA statistics Ready to ROC

14 Helping Give Away Psychological Science (HGAPS)New student organization to Teach people editing skills Critical thinking and evaluating evidence Helping increase the information about Psych Science available online Like HGAPS on Facebook at  https://www.facebook.com/HGAPS/ Follow us on Twitter (audience participation!)

15 Wikiversity Resources to Teach ROC (Google: Wikiversity ROC)

16 Organizing Small Groups

17 ROC Planning Canvas (from Business School – get for triage & consulting)

18 Wikipedia home page

19 #1 -- Talk page

20 Talk page Functions similar to the “comments” function in MS WordAdds ability to talk backchannel (within Wikipedia) by clicking user name Keeps dialog organized on threads (better than a subject line in ) Keeps discussion tightly linked to the content

21 #2 – View history

22 View history Functions similar to the “track changes” function in MS Word EVERYTHING done on Wikipedia is permanently archived & easily accessible Don’t need to worry about losing good information Disagreements? They get hashed out on Talk page (and there is an arbitration process if needed) Better transparency than DSM, RDoC Keeps discussion tightly linked to the content

23 View history This is similar to “Compare Documents” in MS WordBetter transparency than DSM, RDoC Keeps discussion tightly linked to the content

24 #3 – Account information

25 #3 – Account informationSimilar to setting up a home page on Facebook or ResearchGate Add links to your other pages (practice, university, whatever you want) When people “backchannel” you on Wikipedia, it will be on your Talk page (but you can get alerts)

26 #4 – Watchlist

27 #4 – Watchlist Get notifications when edits are made to page(You don’t need to keep checking pages) Can watch people’s pages as well as content pages

28 Those four tools let us make effective changes to WikipediaTalk and View History to advocate and negotiate change Account as a way of building presence, credibility (and linking back to real world work & life) Watchlist as way of monitoring the things that you care about (without it taking lots of extra steps)

29 Keeping it organized and building small teamsUsing WikiEducation “classroom” to build list Use their teaching materials to orient people who want to learn about Wikipedia Build small teams assigned to high priority pages Keeps automatic track of our progress

30 Our “Class” for the coming year 

31 Dashboard provides real-time feedback about progress!

32 Dashboard provides real-time feedback about progress! 10-30-2016

33 EBA Assessment Template – Wikiversity

34 What you can use Any of the content on WikiversityAny of the teaching techniques or principles Working on a Meta-Analysis party! IRT Party? MLM Party? Could plug module into class ROC in 5-6 meetings

35 What you can add Email me suggestions ([email protected]) Add commentsMake edits Add examples Write new pages (HGAPS will consult)

36 What you can do now Like HGAPS on Facebook at https://www.facebook.com/HGAPS/ Follow us on Twitter

37 Thank you!