1 QUANTITATIVE ANALYSISGraduate Thesis & Dissertation Conference Saturday, February 4, 2017 QUANTITATIVE ANALYSIS Jen Sweet Associate Director; Office for Teaching, Learning, and Assessment Shannon Milligan Assessment Coordinator; Faculty Center for Ignatian Pedagogy Shannon

2 SESSION AGENDA Part I: Types of DataPart II: Types of Quantitative Data Analyses Part III: Tools for Data Analyses Shannon

3 SESSION OBJECTIVES Participants will be able to:Differentiate between different types of data and identify which analyses are appropriate for each type Identify tools appropriate for analyzing quantitative data Shannon

4 Types of Data Jen

5 Types of Data Nominal/Categorical Ordinal Interval Ratio/Scale Jen

6 Nominal/Categorical DataNames Data (or arranges data into categories). No numbers associated with this type of data No Concept of Degree or Order No category is “higher” or “better” than another Analysis It is not appropriate to perform any arithmetic operations on nominal data (such as calculating or comparing means). Frequencies and Percentages of the number of cases that fall into each category may be the most appropriate type of analysis for nominal data. Jen

7 Examples of Nominal Data AnalysisExample: Race/Ethnicity Table: Graph: Chart: Race/Ethnicity Frequency Percentage Hispanic or Latino 37 34.0% American Indian or Alaskan Native 0% Asian 13 11.9% Black or African American 20 18.3% Native Hawaiian or Other Pacific Islander 1 0.9% Caucasian (Non-Hispanic) 36 33.0% Race/Ethnicity Unknown/Prefer not to Report 2 1.8% Jen

8 Ordinal Data Ordinal data specifies an order to the information. However, the distance between each data point is not fixed or known Analysis It is not appropriate to perform any arithmetic operations on ordinal data (such as calculating or comparing means). Frequencies and Percentages of the number of cases that fall into each category may be the most appropriate type of analysis for nominal data. Many people calculate means anyhow Important to know how violation of assumptions for conducting arithmetical operations affects interpretation of results E.g. 4 is not double the score of 2; 3.5 is not halfway between 3 and 4 Jen

9 Examples of Ordinal DataExample: Likert scales (agreement scale) Table Graph Strongly Disagree Disagree Agree Strongly Agree Frequency 14 33 57 40 Percentage 9.7% 22.9% 39.6% 27.8% Jen

10 Interval Data Interval data specifies an order to information with equal, fixed, and measurable distances between data points. (No absolute zero) Analysis – Interval data meets the assumptions necessary to conduct certain arithmetic operations addition and subtraction violates assumptions to perform multiplication or division With careful interpretation, use of any arithmetic operation may be justifiable. without a meaningful (absolute) zero, a 4 not necessarily double a score of 2. Possible Analyses (with careful interpretation): measures of central tendency measures of distribution spread measures of relationship mean comparisons Jen

11 Examples of Interval DataExample: Scores on a Test Table Graph Average Test Scores Domain Test Items 100-level Courses Capstone Theory 1, 4, 9, 11,15, 20, 25, 29 64.52 66.73 History 2, 7, 12, 15, 22, 28, 30 73.26 68.54 Socio-Cultural 3, 5, 8, 10, 13, 14, 18, 24, 27 59.63 78.36 Globalization 6, 16, 17,19, 21, 23, 26, 27 58.29 78.31 Jen

12 Ratio/Scale Data Ratio data specifies an order and fixed interval between data points. Ratio data also has a meaningful (absolute) zero. zero that indicates a complete lack of whatever is being measured Possible Analyses: measures of central tendency measures of distribution spread measures of relationship mean comparisons Same as for interval data Jen

13 Examples of Ratio/Scale DataWeight, height, time, sometimes temperature Counts (ex. number of people who attended a given activity) Jen

14 Distinguishing Between Interval and Ratio DataIs 0 absolute? Examples of non-absolute zeros Selection of zero is somewhat arbitrary Longitude: 0 = Royal Observatory (Greenwich, England) prior to 1884, included El Hierro, Rome, Copenhagen, Jerusalem, Saint Petersburg, Paris, Philadelphia, and Washington D.C. Altitude: 0 = Sea Level Jen

15 Illustration of Interval – Sea LevelDenver (above 0) Denver Altitude: +5, 280 feet Sea Level (0) New Orleans (below 0) New Orleans Altitude: -6.5 feet Jen

16 Bottom Line: Interval and RatioBoth types of data can be analyzed using the same techniques The difference is in the interpretation of results A zero on a test doesn’t necessarily mean that the student knows nothing about the content (Interval) Zero people in a room means that there isn’t anyone there (hopefully) (Ratio) A person who scores a 100 on a test isn’t necessarily twice as smart as someone who gets a 50 (Interval) An NFL linebacker probably does weigh 3 times as much as Shannon (Ratio) Jen

17 Types of Quantitative Data AnalysesShannon

18 Common Types of Quantitative Data AnalysisMeasures of Central Tendency Measures of Distribution (Spread) Measures of Relationship Measures of Comparison Shannon

19 MEASURES OF CENTRAL TENDENCYKey question = what is the middle? Three Primary Measures: Mean-the arithmetic average Median-the middle; 50% of data points are above and 50% are below Mode-the most commonly occurring result Example Data: Mean: 20.3 Median: 5.5 Mode: 7 Individual Result 1 2 150 3 4 18 5 6 7 8 9 10 Shannon

20 ADVANTAGES AND DISADVANTAGESMean Median Mode Advantages Most widely used measure of central tendency Broadly recognized measure This measure is not sensitive to outliers Can give you better information about the distribution of your results Does not assume your results are normally distributed Can use with categorical data Disadvantages Sensitive to outliers in data Example = Annual Salaries In 2013, mean household income in U.S. = $87,200 median household income = $46,7001 This measure is not as well-recognized by all audiences May be more difficult to interpret, especially when there are multiple modes General audiences will probably be least familiar with this measure Shannon Federal Government reported 4% growth in household incomes between 2010 and 2013…however, median household incomes FELL 5% (gains were all at the top…federal government was reporting based on means) 1Board of Governors Federal Reserve System (2014). Federal Reserve Bulletin. https://www.federalreserve.gov/pubs/bulletin/2014/pdf/scf14.pdf

21 Measures of Distribution (Spread)Most commonly used is the standard deviation What is it? A relative measure of how far individual data points are from the mean of the data set Why is it important? To give a sense of how spread out the data are overall-are most cases close to the mean? To give a sense of whether an observation is an outlier To determine whether the observation is likely due to chance Shannon

22 Measures of Distribution (Spread)Mean of data set = 20.3 Standard Deviation = 43.5 43.5 is very large, which means the data are quite spread out Shannon 20.3 63.8 107.3 150.8

23 Measures of RelationshipCorrelation: tells us whether and to what extent two variables are related This relationship can be: Positive: variables are related and increase together Negative: variables are related but one decreases as the other increases Non-existent (0) Size of correlation indicates strength of relationship (e.g. totally positively correlated = +1, totally negatively correlated = -1) Advantage: Good for insight/planning and directions for future study Shannon

24 Measures of RelationshipDisadvantage: correlation is often conflated with causation Correlation says that a relationship exists (or doesn’t), not why it exists Does not account for all possible variables Example: there is a strong positive correlation between temperature and ice cream consumption Do high temperatures cause increased ice cream consumption? Does higher ice cream consumption cause an increase in temperature? Shannon

25 Measures of ComparisonExamples: Pre- Post- Data; Primary Questions Is there a difference? Is the difference significant? More Sophisticated Analyses: what was the cause of the significant result ad hoc analyses Shannon

26 Analyses for Comparison/PredictionGeneral Linear Model (GLM) T-test Comparison of two quantities (ex. pre- post- score averages) ANOVA Comparison of results for two groups (ex. pre- post- score averages for males versus females) Multiple Regression Comparison of results for two groups; two or more independent variables (ex. Pre- post- score averages by gender and ethnicity) Multivariate Comparison of two or more dependent variables; one or more independent variables (ex. Pre- post- score averages and internship ratings by gender and ethnicity) Shannon

27 Analysis Decision GuideIn a nutshell… with nominal data, it may be best to stick to frequencies and percentages! Adapted from Nominal data Ordinal data Interval/ratio data Group differences Chi-Square T-test, ANOVA, MANOVA Relationships Correlation Prediction Linear Regression, Multiple Regression Shannon

28 Tools for Quantitative Data Analyses

29 Common Data Analysis ToolsSPSS/SAS Excel R Shannon

30 SPSS/SAS Disadvantages Advantages Widely-usedUser-friendly “plug and chug” Does all calculations for you Disadvantages Requires some training A lot of options; need to know how to select appropriate options for the analysis you would like to run Need to be able to read and appropriately interpret output Potential problem = too easy to run analyses without understanding them May be expensive (DePaul no longer has free access outside computer labs) Limited data visualization capabilities Shannon

31 Excel Advantages Disadvantages Limited data-analysis capabilitiesWidely-used and readily available For most no additional training will be required to use Excel Easy to use with minimal training Integrated ability to visualize data Create graphs, charts, etc. Disadvantages Limited data-analysis capabilities Good for frequencies, percentages, distributions, means, but not capable of other statistical analyses. Shannon

32 R Disadvantage Advantages Free Very FlexibleNo pre-sets; can be programmed Can accommodate more complex statistical modeling/analyses Robust data visualization capabilities Disadvantage Requires programming skills (though you can find on Google) You need to know what you are doing or feel comfortable teaching yourself Shannon

33 Questions? Shannon/Jen

34 Contact Information Shannon Milligan Assessment Coordinator Faculty Center for Ignatian Pedagogy Jen Sweet Associate Director Office for Teaching, Learning, and Assessment Shannon/Jen