1 A Mixed Methods Evaluation of Wise Guys: A final analysisJudith W. Herrman, PhD, RN, ANEF, FAAN Christopher C. Moore, BA, LSSGB Katherine M. Haigh, MSN, APRN, CNP Brian Rahmer, PhD Mellissa Gordon, PhD Barbara Habermann, PhD, RN, FAAN
2 National Institutes of Health under grant number U54-GM104941.Work supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM
3 Significance Decreased teen birth and pregnancy rates, but significant disparities continue Few efforts focus on young males Men who are of special concern: living in poverty ethnic/racial minorities Gay, bisexual, trans out of school in juvenile justice, foster care emerging adults (early 20’s) 42% of young males are sexually active-tend to be sexually active earlier, more sexual partners, and initiators 14% of males have caused a pregnancy: 22% African American, 19% Hispanic, 10% non-Hisp. whites; 6% are fathers- 9% by age 20
4 Why there are fewer male-oriented programs…Lack of social and political advocacy for male reproductive services Limited contraceptive options for males Lack of routine sexual healthcare for males Healthcare focus on women’s health and positive birth outcomes Advocates for teens, in general, are women
5 Wise Guys Male-focused-11-18 Promising/emerging programDeveloped by the Family Life Council, Greensboro, North Carolina Several robust evaluations since 2001 minute classes over weeks Middle school, high school, The Next Level, and Spanish versions
6 Wise Guys Knowledge about sexual activityAttitudes toward women and relationships Safe sexual behaviors Themes: Acceptance of responsibilities of sexual behavior and parenthood Respect for women Communication skills Positive personal growth and self-esteem The importance of values-based decision-making Resistance skills Identification of support systems
7 Evaluation of the Wise Guys ProgramStudy 2 Study 1 N=159 Assessed Attitudes—Thoughts on Teen Parenting Survey Assessed additional attitudes toward sex, relationships, and communication with parents Assessed Knowledge—general and STIs Assessed Behaviors Pre/Post test design Found positive changes in behaviors and communication with parents (Herrman, Moore, & Rahmer, 2016) N=70 Assessed Attitudes—Thoughts on Teen Parenting Survey Pre/Post test design Some positive changes in attitudes (Herrman, Moore, & Sims, 2013)
8 Study Issues Assessing authentic and meaningful findingsIssues-surveying teens Neutral answers Leave blanks Lack of validity Fidelity to program (Wise Guys allows for free discussion) Large number of unusable data Lack of consents No pre or post test Educator did not implement tool Non-completion of program Anecdotal successes not captured in evaluation Assessing authentic and meaningful findings Community-based Participatory Research
9 Why new methods with teens? Issues with….Authenticity Modes of expression Validity and Reliability Developmental Appropriateness Engagement Technologically Driven Youth
10 Specific Aims: Study 3 Develop a valid, reliable, and teen-friendly web- based survey instrument, based on teen perceptions and informed by teen focus groups and a Youth Advisory Group, to measure changes in knowledge, attitudes, and behaviors pre/post the Wise Guys intervention Evaluate the impact of the Wise Guys program using a mixed methods design including the study-developed web-based survey instrument, focus groups, and video journaling content analysis
11 Research Components Created Youth Advisory GroupFocus Group 1: Major themes of Wise Guys, led to-- Focus Group 2: Generate survey items, led to-- Development of survey, led to-- Focus Group 3: Content validity indexing of items Adapted the survey for REDCap and delivered via mobile devices, led to-- Pilot Survey as pre/post-test led to– Wise Guys Implementation and use of Survey Video Journaling Project Focus Group 4: Video Journaling Project Focus Group 5: Member Check of emergent themes
12 Major Constructs/Themes-Emerged from the focus groups-informed survey1. Communication and Relationships 2. Sexual Health and Decision-making 3. Respect for Self and Others 4. Responsibilities, Values, and Goals 5. Thoughts on Self 6. General thoughts about Wise Guys
13 Video Journaling Project MethodsTwenty young men who completed the Wise Guys program- assent and permission Training on the VJP and given Ipods with video capability. The participants were charged with taking videos of their daily life while considering how Wise Guys and the lessons learned impacted their daily lives through the lenses of directive questions. This self-reflective exercise was designed to address: Personal goals and objectives Recent accomplishments Obstacles Social support and environmental factors
14 Video Journaling Check ins Every 1-3 days over three weeksText messages by Research Assistant Questions to guide their videos Encouragement to be creative Reminders to video
15 Emerging and Prominent ThemesCommunication and relationships “I like learning about healthy relationships.” “I could relate to how a female is feeling.” Sexual health and decision-making “I make better choices.” “Wise Guys teaches you about sexual health, how to prevent pregnancy and what’s the best way to do it, like puberty and stuff like that…all about your sexual health and stuff.” Respect for self and others “I approach situations with more respect…for the people and their experiences.” “I respect women more.”
16 Emerging and Prominent ThemesResponsibilities, Values, and Goals “We all have our own values. I have goals. Every day I wake up and work on them.” “it teaches you to take responsibility for yourself in everything you do.” Thoughts on Self “Wise Guys changed my life, is that everything I learned made me become a man.” “Wise guys helped me with my anger, I still have trouble with my anger.”
17 Quantitative Data N = 115 Mean age = 16; 65% AfricanAmerican; 57% sexually active, 62% live in single parent families Pre/post test design Significant improvements in 4 constructs: Communication and Relationships Respect for Self and Others Responsibilities, Values, and Goals Thoughts on Self Data failed to detect significant differences in sexual health and decision-making—already knowledgeable/veterans of the program
18 Major findings of the VJPValidated prominent themes from focus groups. These were used in developing tool: Communication and Relationships, Sexual Health and Decision-making, Respect for Self and Others, Responsibilities, Values, and Goals, Thoughts on Self, and General thoughts about Wise Guys Validated quantitative findings. More acceptable means of communicating for some young men; others noted a preference for writing answers or “just talking.” Provided ecological momentary assessment—perceptions in real-time, genuine and authentic glimpses into the minds/thoughts of young men
19 Triangulation of Data ConclusionsMixed methods and VJP provided a medium for accessing rich data from young men. This is especially true with regard to the: Sensitive, personal nature of the data Active engagement in daily technology and social media Varying levels of verbal expressiveness of teens. Video Journaling may provide an alternative means of evaluation Video Journaling may be an effective method to provide voice to marginalized populations