1 Designing and Conducting Field ResearchShort Course #2 APSA 2011 Instructors: Diana Kapiszewski, UC Irvine Naomi Levy, Santa Clara University Building on a course initially developed and taught by Melani Cammett (Brown University), Marc Morjé Howard (Georgetown University), Evan S. Lieberman (Princeton University), Julia F. Lynch (University of Pennsylvania), Lauren Morris MacLean (Indiana University), Benjamin L. Read (UC Santa Cruz), Scott Straus (University of Wisconsin, Madison) & Sara Watson (Ohio State University)

2 Agenda Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) ~ Break (4:10-4:20) ~ Part III – Interviewing (4:20-5:15) Part IV – Organizing and Analyzing Data & Assessing Progress (5:15-6:00) ~ Break (6:00-6:10) ~ Workshop (6:10-7:00)

3 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

4 Fieldwork in Political Science: What Is It?Working definition for this course: Leaving one’s home institution to collect data that significantly informs one’s research Fieldwork is not delimited to one’s time in the field Fieldwork is a very iterative process

5 Fieldwork in Political Science: Grounds for CautionCostly in terms of time money Roadblocks and challenges of all types Most graduate programs provide little training Read, read, read on your own!

6 Fieldwork in Political Science: What Does it Look Like?The pre-dissertation scoping trip The “surgical strike” short trip A series of short trips spread out in time The brief shadow-case foray Full-blown, “traditional” 1-2 year stay The follow-up trip Learning what’s out there: More and more data available on the web. But much that isn’t. Taiwan: 1) tried to use existing datasets from data archives. But questions were highly inadequate. 2) Went to Taipei official’s office, saw report; visited researcher who had written report; on his introduction, went to central government ministry; they told me about a survey they had commissioned from another academic; visited that academic and got the data Networks: Recent trip to Beijing, got internal data about local elections. Bolster readers’ confidence in your results by showing converging findings through different data sources This is a general principle in all research, but especially important when validity is in question Examples: Interviews in addition to survey data Documents or internal archives in addition to interviews Using high-validity method as a check E.g. interviews by local interviewers in addition to the ones you do yourself

7 Fieldwork in Political Science: What Does it Look Like (cont’d.)?Selecting the topic Reading existing literature Defining the research question Assessing feasibility Completed proposal Follow-up data-gathering Loosely Structured: Open-Ended Research Highly Structured: Narrowly Focused Research

8 Varieties of Fieldwork: Trade-offsLong stays: Pros Experience another culture In-depth research Contacts/network Ideas More relaxed & fun Cons May be far away Lack of urgency “Too much” data Shorter trips: Pros Efficiency Forces you to take stock Identify comparisons/ contrasts quicker Less time away Cons Can be more costly Scheduling difficulties Not enough time Personal: being apart from friends and family can be rough Financial (short-term trips expensive)…. Renting for short periods is much more costly Price of flights add up Other tradeoff to consider with multi-case research: If you’re doing multi-case research, another dilemma that arises is how to split your time between the different cities/countries/etc. - Should you Example: Spanish unemployment, effects of union competition. I took it for granted because everyone had a certain narrative. If I had gone back and forth more frequently, might have caught on more quickly that the logic didn’t make sense

9 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

10 Design-Driven Fieldwork and Fieldwork-Driven DesignField work must fit your research design Your design must accommodate field realities Competing Imperatives Your ideal research design Practicality

11 Design-Driven Fieldwork and Fieldwork-Driven DesignDesign “do-able” research Think about your own limitations Find a way Think positively Revise Think in terms of variables

12 Fieldwork and the Research DesignNomothetic Thinking in terms of variables Idiographic Thinking in terms of cases

13 Data Matrix Variable 1 Variable 2 Variable 3 Case 1 Case 2 Case 3

14 Case Selection and SamplingSelection decisions arise at many stages Macro-level Country cases Meso-level Regions or Towns Time periods Sectors, etc. Micro-level Individuals for interviews Documents for content analysis

15 Small-N Use case selection to provide causal leverageUse a variable-centered approach Hold rival explanatory variables constant Allow your primary explanation to vary Example: Dan Posner (2004) Macro-level: Zambia & Malawi Meso-level: Town selection held rival variables constant

16 Posner (2004) Meso-level Selection

17 Large-N Where possible, use random samplingThis increases the generalizability of your findings Need a list of the universe of cases Consider cluster sampling Sometimes not possible or desirable Special cases you want to include Important variation you want to capture

18 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

19 The Months and Weeks Before… AdministrationFunding Make a budget Apply for lots of money Timing Sources Dealing with your Institutional Review Board This does not have to be difficult! Follow the directions, adhere to deadlines Find out about exemptions and consent

20 The Months and Weeks Before… Intellectual PrepDig into your topic READ!! Maximize creativity in terms of data sources! Background research on your country/ies News Running lists

21 The Months and Weeks Before… More Intellectual PrepBegin to write documents Ways to present your research One-page description Interview questions Contact s Collect documents you’ll use in the field Brush up on methods Think about your foreign language skills

22 The Months and Weeks Before… Reach Out!Develop your network of scholars and contact them! Contact grad students, professors Network at conferences Get on listserves Contact NGOs, think tanks in-country journalists You are not a bother!

23 The Months and Weeks Before… And Reach Out Some More!How about a host institution (research affiliation) Plenty of benefits But be aware of profile and reputation

24 Converting your Research Design into a “To Get” ListVariable Indicators Data needed to measure Where/ How to get data When to get/time needed DV IV 1 IV 2 IV 3

25 Sidenote on MeasurementHow do you know variation when you see it? Conceptual Definition Operational Definition Much of the measurement process happens on the fly in the field Keep detailed notes on your measurement decisions

26 Converting your Research Design into a “To Get” ListVariable Indicators Data needed to measure Where/ how to get data When to get/time needed DV IV 1 IV 2 IV 3

27 Example of “To Get” ListVariable Indicators Data needed to measure Where/ how to get data When to get/time needed Legal infra-structure governing inf. work # laws regulating inf. sector # const. rts. re: IWs 1. Lists of nat’l., state, local laws 2. Const. rights 1. Gov’t. ministries/offices/ on-line 2. Const. Early (1-2 mos.) Early Use of legal system by IWs IWs’ court cases Protest based on law/const rights Record of cases filed with courts History of protest 1. Courts’ records/ on-line 2. News-papers Early (2-3 mos.) Later

28 About the “To Get” List You can head to the field before you have your entire “to get” list filled out! Not everything you will need “to get” in the field will be on your “to get” list To get list Helps make your project manageable Is a measure of your progress Is the link between all the millions of tasks you carry out in the field, and your larger project

29 Converting your Research Design into a “To Get” ListVariable Indicators Data needed to measure Where/ how to get data When to get/time needed DV IV 1 IV 2 IV 3

30 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection(3:25-4:10) 1. Forms of Data Collection 2. Data Collection: Choices, Challenges, Assistants * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

31 Non-Interactive Data CollectionObservations Following the local media Newspapers, magazines, radio and TV Books & articles Obtaining Documents & Existing Datasets Government agencies & Ministries NGOs Archival Work

32 Interactive Data CollectionEthnography Participant Observation Large-scale surveys Interviews Oral Histories Focus Groups

33 Choices in Data CollectionStrengths and Weaknesses Non-interactive Documents don’t “react” Documents and archives can be biased Interactive Respondents can provide first-hand account Respondents might be inaccurate Triangulate!

34 Triangulation

35 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) 1. Forms of Data Collection 2. Data Collection: Choices, Challenges, and Assistants * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

36 Data Collection Begins with QuestionsInitial data collection task = brainstorming questions Items on to-get list  questions Those questions point you towards sources, or places to ask your questions The more you know, the more and better questions you’ll have!

37 Sequencing and PrioritizingFeeling overwhelmed? Distinguish necessary from desirable data Centrality to core hypotheses or key variables Other factors to consider in ordering data collection Overlapping strategies Harder-to-get vs. easier-to-get data “Low-risk” vs. “high-risk” contacts Temporally-tied events Depth vs. breadth

38 Anticipate Data Collection ChallengesRoadblocks to accessing elites, ordinary citizens, archives, datasets Affiliate Network – find a “connection” Be professional Reciprocate Think empathetically Consider the investment: time, effort, patience, $ Under validity: Actual validity Perceived validity (“They couldn’t have been telling you the truth!”) Triangulation (among interviewees, categories of respondents, types of sources)

39 Methods for Capturing DataDocuments Take notes Reproduce Interviews & Focus Groups Field notes after the fact Jotted notes during Audio/Video tape Combo How do decide? How much data do you want to capture? How are you going to use the data?

40 Methods for Capturing Data (cont’d.)Reproducing/Recording Pros You aren’t just relying on your notes and memory Capture verbatim quotes Richer data Reproducing/Recording Cons Too much data Costly Equipment Storage Transportation Transcription Recording can make people nervous

41 Hiring Research Assistants (I)Does It Make Sense? Pros: ‘Giving back’ to countries Can be a great source of information Building long-term relationships Cons Training = time consuming May cause problems or quit or fail to follow instructions May introduce bias

42 Hiring Research Assistants (II)Recruitment How to find RAs Interview What are you offering? Contract How much and how to include RAs in your research

43 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

44 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) 1. Preparing and Writing Questions 2. Conducting the Interview/Follow-up Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

45 Interviewing General GuidelinesDo your homework Interview “those who study” before “those who do” Varieties of interviews Structured? Semi-structured? Informal chat? Sampling redux Scheduling interviews Where to conduct the interview

46 Interviewing Writing Interview Protocols (I)Language Grand tour vs. open-ended vs. specific qstns. What’s the goal of the interview? Will your questions produce useful data? Theoretically motivated/in colloquial terms What are you asking?! Asking challenging questions Weeding questions

47 Interviewing Writing Interview Protocols (II)Transitional language Varieties of responses Keep questions simple and direct Get local input on your questions Pretest Crucial questions: how are you going to analyze and use the data?

48 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) 1. Preparing and Writing Questions 2. Conducting the Interview/Follow-up Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)

49 Conducting the InterviewIntroducing yourself and your research Interacting with your respondent Conversation vs. Interview DOs and DON’Ts Prioritize your questions Know your protocol

50 Conducting the InterviewProbes follow-up questions that are used to deepen a response to a question Types Basic signals Detail-oriented Questions who, what, where, when, how Elaboration Probes “Tell me more” Clarification Probes “What do you mean?”

51 Conducting the InterviewUsing Silence One of the most useful “probes” Gives the subject the space to talk Wait longer than is comfortable for you Builds constructive tension Taking notes A form a body language What to write Wrapping up Be thankful, be very very thankful!

52 After the Interview Write up your notes!!!Worth being VERY disciplined about this! Do it before you conduct another interview Include a description of the person & location Note the most important new information Remember: writing notes is a form of data reduction If you record, dealing with your recordings Transcribe? “Listen through”? Put aside? Send a thank you note! Getting better

53 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection(3:25-4:10) **Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing and Analyzing Data & Assessing Progress (5:15-6:00) 1. Organizing and Analyzing Data 2. Assessing Progress and Wrapping Up **Break (6:00-6:10) Workshop (6:10-7:00)

54 Data Organization: Questions to ConsiderYou’re Project Manager of your study – organize and systematize from the start! Why organize? Helps you to see the progress you’ve made Eases re-entry What needs to be organized? data contacts thoughts

55 Organizing Your Data Sources of Data Interviews ArchivesWritten Primary and Secondary Sources Datasets Observations Misc. Documents Forms of Data Notes, Tapes, Transcripts Notes, Copies Electronic Files, Printed Copies Notes, Tapes

56 Establish a System for Filing Physical and Electronic Documents

57 Organizing Your Contacts (I)Develop a contact management system What matters most is that it works for you. General suggestions Separate from your general address book. Lots of fields Set up to automate thank-you letters Easy retrieval for follow-up, thank you notes, future projects Make it the beginning of a database . (Refer to handout)

58 Organizing Your Contacts (II) Chart for Potential RespondentsStatus Respondent and contact info. Position/ Comments Suggestion of whom Progress ( s; phone calls, etc.) X Fernando Prieto … Sec: Claudia Head of Central Bank… was important… Gustavo Sainz (said talk to him esp. about...) 02Jan2005 – spoke with secretary… Status: X = need to find X = found, need to get going X = going X = set and/or done X = don’t do anything for now X = give up/declined

59 Organizing Your Contacts (III) Interview Clean-upIntrvw. code Intervw. done Names given added to contact list Type up notes / things said off record / thoughts Send thank you note Still need/ respondent promised/ need to follow-up EC-01 22 sept. 2005 22 Sept. 2005 22 Sept. 2005 Promised document on…

60 Digesting Your Data Develop a strategy for keeping track of your thoughts Make a habit of writing Organize your thoughts Begin your analysis while in the field Periodically read through your documents and notes Write tentative memos Prepare your data for analysis Think about your coding strategy Import data into software programs

61 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) **Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing and Analyzing Data & Assessing Progress (5:15-6:00) 1. Organizing and Analyzing Data 2. Assessing Progress and Wrapping Up **Break (6:00-6:10) Workshop (6:10-7:00)

62 Assessing Your Progress (I)Challenge: simultaneously thinking micro (trees) and thinking macro (forest) Addressing overall progress entails getting some perspective on the forest Where are you on the “to get” list? How much data have you collected? How many of the people you consider crucial have you spoken with?

63 Assessing Your Progress (II)Research projects often seek to build toward a specific argument Have evidence to support your hypotheses? If new argument, data to support that? Do you have evidence to address “alternative hypotheses”? Get help to correct for “field goggles” -- to push larger questions to the forefront Keep in touch!

64 Assessing Your Progress IIIDifferent vehicles for telling your evolving story Periodic written reports or memos to advisors Write conference papers. Present your work in-country informally Upsides Helps you do some early analysis Gets you feedback on your analysis and story Helps you see where you really are!

65 Making Adjustments It’s common for people to (want to) make some adjustments Carefully consider any change to your project Be patient and set some reasonable deadlines. Carefully diagnose the problem. Talk with others about your potential “fix” Make sure your “fix” addresses the problem.

66 What seems to be the problem? Why is this happening? Possible fixes Should I change my topic? I can’t answer the question Unfalsifiable question; Access issues Triangulate; Come back later; Do without Only if multiple, central hypotheses are untestable I’m surprised by the answer Your initial guesses were wrong; You’re not getting the full story Make it a good surprise Only if nothing of value to be gained from pursuing question in its current framing I’m testing the wrong hypotheses Theories were inapplicable; You made a good guess, but guessed wrong Come up with new hypotheses and ways to test them Generally not a reason to change topics I’m asking the wrong question You’re bored; Your question got dated; It really was never the right question Remind yourself why you asked this question; Take a historical view; Link to different theories Radical change usually not advisable

67 Tweaking Your Project Data Matrix Variable 1 Variable 2 Variable 3Case 1 Case 2 Case 3 Case 4

68 Tweaking Your Project Micro-level Matrix -- Individuals Association# of Grants/yr Sources Person 1 Org 1 12 Government Person 2 10 International Person 3 13 Person 4 Org 2 2 Private Person 5 1 Person 6 3

69 Tweaking Your Project Meso-level Matrix Funding Sources ConnectivityOrganization 1 Well-funded Gov’t & Int’l Organization 2 Struggling Private

70 Knowing When to Wrap Things UpDevelop some criteria to help you determine when you have “enough” Check your “to get” list What information is only available abroad? Consider how much paper to cart home (or between countries) Return trips are an option Research sites won’t disappear You will be able to hit the ground running

71 Preparing for “Life After Field Research” While in the FieldConsider reintegration strategies Request office space from your home dept Join a dissertation-writing group To TA or not to TA? Try to plan out your first month back Unpack your boxes ASAP Deal with unfinished field business Data to transcribe, code, enter, or clean up Reconnect with your advisors Plan to go easy on yourself!

72 Where We Are in the CoursePart I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Workshop (6:10-7:00)