Research Design and Field Methods

1
Research Design and Field Methods

• Short Course - APSA 2007
• Instructors
• Diana Kapiszewski, UC Berkeley
• Benjamin L. Read, University of Iowa
• Sara E. Watson, University of British Columbia

Building on a course initially developed and
taught by Melani Cammett (Brown Univ.), Marc
Morjé Howard (Georgetown Univ.), Evan S.
Lieberman (Princeton Univ.), Julia F. Lynch
(Univ. of Pennsylvania), and Lauren Morris
MacLean (Indiana Univ.)
2
Agenda for the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

3
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

4
Field Research in Political Science What Is It?

• Working definition for this course
• Field research refers here broadly to research
  that requires leaving ones home institution in
  efforts to generate new data rather than relying
  on pre-existing or pre-packaged sources.
• Field research need not mean going to other
  countries.
• Diverse approaches and methods
• Participant observation (ethnographic/anthropologi
  cal)
• Archives, libraries, government ministries, NGOs
  and other non-governmental sources
• Survey research and other quantitative
  data-gathering
• Interviews (elites, non-elites, experts)
• Experiments
• Combinations of the above

5
Field Research in Political Science Why Do It?
(1)

• Most important reasons
• Its simply the only way to get many forms of new
  data
• (And often to learn about sources of existing
  data)
• It may be especially conducive to establishing
  validity, for instance of measurements and causal
  processes
• Complements other research methods
• Provides opportunities for reality checks
• Builds networks that can be invaluable for
  current, future research projects
• Provides opportunities for productive
  serendipity, inspiration, fresh questions and
  ideas

6
Field Research in Political Science Why Do It?
(2)

• Less tangible reasons
• Helps establish your credibility
• Seen by some as a rite of passage
• Can be greatly enjoyable, fulfilling

7
Field Research in Political Science Grounds for
Caution

• Costly, both in time and sometimes money
• Encountering road blocks is likely
• Can be challenging to get what you want
• Easy to be distracted
• Things can just go wrong
• Little training is provided in most graduate
  programs
• Many techniques are not well institutionalized

8
Make Sure That You Ought to be Doing Field
Research

• Is it necessary for your project?
• Will you enjoy collecting primary data in the
  field? Are you well suited to the tasks you have
  planned?
• Cold-calling respondents
• Combing the archives
• Unexpected situations
• How much time do you want to spend away from home
  base?
• Many different models of fieldwork (as Sara will
  discuss).

9
The Usefulness of Multiple Methods and Measures

• Bolster readers confidence in your results by
  showing converging findings through different
  data sources
• This is a general principle in all research, but
  especially important when validity is in question
• Examples
• Interviews in addition to survey data
• Documents or internal archives in addition to
  interviews
• Using high-validity method as a check
• E.g. interviews by local interviewers in addition
  to the ones you do yourself

10
Example 1 Lily L. Tsai, research for 2004
dissertation, published as Accountability without
Democracy (2007)

• Research question What explains variation in
  village government performance?
• Started with a set of village case studies
• During this process, networked and exchanged
  ideas with local researchers
• Obtained ministry-level support for her research
  (!)
• Designed a survey questionnaire
• Trained a team of six student RAs
• Conducted a village-level quantitative survey of
  300 villages
• Note This didnt happen all in one trip, but in
  a series of visits

11
Example 2 James C. Scott, research for Weapons
of the Weak (1985)

• Research question How are class relations
  understood and class conflicts played out at the
  micro-level?
• 14 months of fieldwork in Malaysian village,
  Sedaka
• In part, he used standard ethnographic techniques
  such as household surveys, lineage charts
• Supplemented by background research on both the
  village and the broader agrarian setting
• But for the most part extensively talking to
  villagers, listening to what they said, taking
  copious notes.

12
Example 3 Marc M. Howard, dissertation
research, published as The Weakness of Civil
Society in Post-Communist Europe (2003)

• Research question Why do post-communist
  societies have especially underdeveloped
  associational sectors?
• Several months of fieldwork each in Russia and
  the former East Germany
• Conducted in-depth interviews with ordinary
  citizens
• Supplemented these with elite and expert
  interviews
• Commissioned surveys that were carried out by
  polling companies in the two countries
• (Later) Supplemented data he gathered himself
  with data from World Values Surveys

13
Starting Places Where Are You in Your Project?

• Selecting the topic
• Reading existing literature
• Defining the research question
• Assessing feasibility of data-gathering
• Completed prospectus or proposal with
• a research question or puzzle
• a set of hypotheses to test
• potential observable implications of those
  hypotheses
• Follow-up data-gathering
• Already have collected some data need more or
  need a different kind.

14
Where Are You in Your Project?Implications

• Selecting the topic
• Reading existing literature
• Defining the research question
• Assessing feasibility
• Completed proposal
• Follow-up data-gathering

Loosely Structured Open-Ended Research
Highly Structured Narrowly Focused Research
15
Bens Sample Research Project
Topic State-Sponsored Grassroots Organizations
in East Asia Existing Literature Corporatism
Soviet and other authoritarian cases Social
networks Patron-client ties Development
literature on synergy etc. Research Question
How do ordinary people look upon and interact
with state-sponsored neighborhood organizations
in China? Hypotheses (simplified with some
observable implications) - Alienation
(expressed hostility resistance no useful
purposes seen) - Community (close ties with
neighbors frequent interaction prefer the RCs)
- Clientelism (material payoffs dependency
atomization among clients)
16
Bens Sample Research Project
Topic State-Sponsored Grassroots Organizations
in East Asia Existing Literature Corporatism
Soviet and other authoritarian cases Social
networks Patron-client ties Development
literature on synergy etc. Research Question
How do ordinary people look upon and interact
with state-sponsored neighborhood organizations
in China? Hypotheses (simplified with some
observable implications) - Alienation
(expressed hostility resistance no useful
purposes seen) - Community (close ties with
neighbors frequent interaction prefer the RCs)
- Clientelism (material payoffs dependency
atomization among clients)
1) Exploratory study 2) Primary fieldwork in
China 3) Follow-up fieldwork in South Korea,
Taiwan
17
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

18
The Months and Weeks Before Your Journey

• You as Project Manager
• Organization and systematization
• Fieldwork begins long before you head into the
  field
• Exhaust home resources before heading out

19
The Months and Weeks BeforeAdministration and
Logistics (I)

• Funding
• Youll likely need external funding
• Fieldwork is more expensive than you think its
  going to be
• Make a budget
• Apply for lots of money
• Timing
• Sources
• Applying for more funding from the field?
• Find out your options!

20
The Months and Weeks BeforeAdministration and
Logistics (II)

• Dealing with your Institutional Review Board
• This does not have to be difficult!
• Origins
• Follow the directions
• Exemptions
• Consent

21
The Months and Weeks BeforeAdministration and
Logistics (III)

• Logistics
• Registration status at your home institution
• Dealing with your belongings in the U.S.
• Money/Banking
• Insurance
• Visa
• Plane tickets
• Finding a place to live in the field
• Health
• Documents and other things to take
• Getting yourself ramped up technologically

22
The Months and Weeks BeforeIntellectual Prep

• Dig into your topic
• Maximize creativity in terms of data sources!
• Inter-library loan
• Electronic databases and archives
• What materials to take with you
• Background research on your country/ies
• News
• General country knowledge
• Create a timeline of important events

23
The Months and Weeks BeforeMore Intellectual
Prep

• Begin to write documents youll use in the field.
• Ways to present your research
• One-page description
• Interview questions
• Brush up on anything you might need
  methodologically
• Think about your foreign language skills if this
  is relevant

24
The Months and Weeks BeforeReach Out!

• Begin/continue to develop your network of U.S.
  scholars and contact them!
• Get in touch with people who know your topic to
  get their feedback
• Contact grad students, professors, NGOs, think
  tanks
• Network at conferences!
• Get on listserves
• U.S.-based NGOs and think-tanks
• You are not a bother!

25
The Months and Weeks BeforeReach Out Some More!

• Begin/continue to develop a network of in-country
  contacts and contact them!
• Network at APSA, LASA, and other conferences
• Work your U.S. contacts with experience in the
  field
• Look at in-country university web sites
• E-mail the in-country contacts you have
• Network using U.S.-based and local NGOs and
  think-tanks
• E-mail in-country journalists

26
The Months and Weeks BeforeAnd Reach Out Some
More!

• Identify and connect with a host institution
  (research affiliation)
• Upsides
• How to find a research affiliation
• What to consider when choosing an institution
  with which to affiliate
• Get letter early

27
The Months and Weeks BeforeOrganize and
Systematize

• How will you organize your data?
• How will you keep track of all of your
  contacts/potential interviewees?
• How will you keep track of your progress -- what
  you did/what youre doing/what you have yet to
  do?
• How will you keep track of your thoughts?
• Word docs?
• Whatevers EASY!

28
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

29
Varieties of Field Research

• Field research can be done in various stages, and
  trips can be of varying duration.
• The pre-dissertation summer probe
• Full-blown, traditional 1-2 year stay
• Series of short trips spread out in time
• The brief shadow-case foray
• The follow-up trip
• What variety of field research you adopt depends
  on
• Type of data you need to answer your question
• Time you have to dedicate
• Funding
• Personal life

30
Varieties of Field Research Trade-offs to
Consider

• Long stays
• Pros chance to experience another culture, do
  in-depth research
• Cons far from family and friends
• Surgical strike trips
• Pros Less time away from home
• Cons More costly
• If things go wrong you may not have enough time
  to do mop-up

31
How to Organize Multi-Case Research?

• Should you spend long chunk of time in each
  country/city or should you do multiple
  back-and-forth trips?
• At stake thinking about counter-factuals
• Your goal using comparisons to make causal
  inference
• If you stay too long in one place with no check,
  its easy to turn into a historian rather than
  the political scientist you want to be!
• Example Spain, in light of Portugal

32
Converting Your Research DesignInto a To Get
List

• Part of the process of Project Management
• The To Get list
• Helps make your project manageable
• Is your measure of progress
• Is the link between field research and broader
  project
• Start making your To Get list by brainstorming
• What do you want to know?
• Be as specific as you can be (at this point)

33
Example Initial To Get List

• Look literature books, articles, local
  dissertations, masters' theses -- related to
  topic
• Newspaper articles (if relevant) in order to
  construct basic timeline of events
• Relevant archives and ministry libraries -- go
  through their holdings
• Sources of different types of data
• Relevant NGOs and other local organizations
  involved with your topic

34
From the To Get List to the To Do List

• Over time your To Get list will become more
  detailed, and will include more specific types of
  data.
• For each data item you add to your To Get list
  start thinking about how youre going to get that
  data.
• This involves moving from your To Get List to
  your To Do list

35
How do you go from the To Get to the To Do
List?

• Example of To Get item
• Get better picture of the relationship between
• unions and sister-parties between 1980 and 2005.
• Sounds straightforward, right?
• Perhaps, but do spend some time thinking about
  how to get this item.
• Is interviewing union leaders is the best way
  to go?

36
The Associated To Do List

• Locate relevant secondary literature
• Identify and locate relevant union and party
  leaders
• Write letter requesting interview
• If no response, call
• Schedule the interview
• Prepare interview questions
• Do the interview!
• Digest interview (write up notes)
• Thank you note dont forget the secretary!

37
The Point

• Not to scare you!
• Not to suggest you need a full-blown To Get
  list or a 50-page To Do list before you leave
• Both lists will become more specific as your
  research progresses
• However
• Start yourself thinking along these lines
• For every to get item you have, think about how
  youre going to obtain that piece of information

38
Timeline

• Consider putting together a Timeline document
• Include different sections for each month youre
  in the field
• Here, you can begin to catalogue tasks, and
  to-dos (and eventually accomplishments!) 
• This can help you prioritize tasks, and will make
  accomplishing all the items on your various lists
  seem less daunting

39
When Am I Ready to Go?Setting the Date

• Field research obviously begins months before you
  enter the field
• That said, there comes a point when it is
  counter-productive to stay at home
• The temptation may be high to postpone.
• Think carefully about what more you really can
  accomplish stateside, then set a date and buy a
  plane ticket!
• One way or return ticket?

40
Planning out the First Month Personal Life

• Find a safe and pleasant place to live
• Figure out your surroundings locate the local
  grocery store, pharmacy, etc
• Logistics
• buy a cell phone
• register with the local authorities, police (!)
• set up a bank account
• Strategies for meeting people, staying sane

41
Planning Out the First MonthEasing Into Your
Research

• Make an appointment with the institute at which
  you have an affiliation, check out their library
• Make appointments with local scholars, ask them
  for references related to your topic
• Begin interviewing potential RAs
• Bottom line
• Start ramping yourself up to full-time research

42
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

43
Prioritize and Focus

• Dive right in once you arrive
• Distinguish necessary from desirable data, and
  pursue data collection accordingly
• Centrality to core hypotheses
• Possibilities of alternative measurement
• Order data collection in a way that makes sense
• E.g. Geographic location
• Printed materials before interviews
• Causal priority/chronology
• Low-risk to high-risk contacts

44
Anticipate Trade-offs

• To photocopy, scan, take digital pictures, or
  take notes?
• To tape-record, or listen and take notes? (more )
• To hire RAs or go solo? (see next section on
  working with RAs)
• To subcontract or conduct your own survey?
• Talk to higher-ups within an organization, or
  people lower on the totem pole
• Depth vs. breadth

45
Developing Strategies to Address Data Collection
Challenges

• You may encounter difficulties in gaining access
  to
• Elites, ordinary citizens, archives, datasets
• You may face validity challenges (are the data
  youre getting biased in a particular way?)
• General strategies for gaining trust and access
• Affiliate
• Network
• Reciprocate
• Consider the investment time, effort, patience,
  money

46
Develop a Contact List Management System

• Names, numbers, addresses, emails, secretaries,
  referred by fields
• Contact progress/follow-up
• Mail-merge for thank-you letters
• Easy retrieval for follow-up, thank yous, future
  projects

47
Information Management is Particularly
Challenging for Field Research Projects
Sources of Data Interviews Archives Secondary
Sources Datasets Observations Misc. Documents
Forms of Data Notes, Tapes, Transcripts Notes,
Copies Notes, Copies Electronic Files, Printed
Copies Notes, Tapes Notes, Copies
48
Establish a System for Filing Physical and
Electronic Documents
49
From Information ManagementTo Information
Analysis

• Create a (physical) filing system
• Not boxes labeled South Africa
• Simple enough to maintain
• Avoid the scorched earth strategy
• Create an electronic filing system
• Cataloging and processing can help you begin to
  analyze!
• E.g. spreadsheet of interview subjects together
  with their attributes, coding key opinions
• E.g. logging archive documents in EndNote,
  together with keywords flagging important content
• E.g. start to shape the data into cases
  (cities, sectors, events)
• E.g. marking crucial concepts/words in interview
  transcripts

50
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop

51
You as Professional Researcher

• Self-presentation and image
• Hold your head up high!
• Uncomfortable situations
• Varieties
• Think in advance about how you will respond,
  balancing data-gathering needs vs. personal
  concerns
• Topic for workshop?

52
Working CollaborativelyUpsides

• Greater trust and rapport with participants may
  lead to greater validity of data
• May gain access to new data or contacts
• Can be incredibly refreshing and stimulating
• Provides a reality check
• Lays groundwork for future projects/ extensions
• May be able to help you after you return home

53
Working CollaborativelyConcerns

• Consider how your insertion in any data gathering
  context can shape your data
• Consider information issues
• Local communities of respondents or researchers
  are not necessarily harmonious!
• Balance!

54
InterviewingGeneral Guidelines (I)

• Do your homework on issues and respondents
• Interview those who study before those who do
• Varieties of interviews
• Structured? Semi-structured? Informal chat?
• Think about what you want to know and how youre
  going to use the data

55
InterviewingGeneral Guidelines (II)

• Scheduling interviews
• Dont wait too long to get started
• Dont schedule your interviews too close together
  
• Location
• Office vs. home vs. café
• Considerations
• Send a thank you note!
• Keeping track of interviews
• ORGANIZE!

56
Interviewing Constructing Outlines/Questionnaires
(I)

• Language
• Open-ended or closed questions?
• Whats the goal of the interview? Will your
  questions produce useful data?
• Theoretically motivated/in colloquial terms
• What are you asking?!
• Asking challenging questions
• Weeding questions

57
InterviewingConstructing Outlines/Questionnaires
(II)

• Transitional language
• Could you expand on that?
• Varieties of responses
• Keep questions simple and direct
• Get local input on your questions
• Pretest
• Crucial questions how are you going to analyze
  and use the data?

58
InterviewingConducting the Interview (I)

• Introducing yourself and your research
• Interacting with your respondent
• Be interested, interesting, engaged, engaging!
• LISTEN!
• Redirection
• Prioritize your questions and know your
  questionnaire
• Respondents lack of knowledge

59
InterviewingConducting the Interview (II)

• Wrapping up
• Be thankful, be very very thankful!
• Capturing the interview
• Record? Take notes? Do both?
• If you record, dealing with your tapes
• Transcribe? Listen through? Put aside?
• Write up your notes!!!
• Worth being VERY disciplined about this!
• Consider embellishing
• Getting better!

60
Research Assistants (I)

• Considerations
• Consider your project and data collection
  techniques
• Consider time and money
• Pros
• Building great relationships
• Knowledge!
• Cultural broker
• Companionship
• Cons
• May quit
• May cause problems in community
• May be quicker to do a task yourself than train
  someone
• May not follow your instructions

61
Research Assistants (II)

• Recruiting
• Begin recruitment early
• Selection criteria
• Contract?
• Think about how, and how much to include your RAs
  in the project
• Upsides of integrating your RAs
• Take them to interviews with you?

62
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and
  Wrapping Up (Sara)
• 4. Discussion
• Part III Workshop

63
Digesting Your Data

• Take time to analyze your findings as you go
  along
• Develop a strategy for keeping track of your
  thoughts whatever is EASY for you.
• Note recurrent or aberrant themes in the margins
• Keep a notebook
• Record your reactions to new pieces of
  information as a whole
• 1. Ask why relevant?
• 2. How does it fit?
• 3. What should I do next?

64
Using Qualitative Data in your Work Keeping It
in Qualitative Form

• Using qualitative data for illustration
• List of examples, varieties
• Short or long quotations
• Coding and Quantifying Qualitative Data
• 14 out of 19 neighborhood leaders attested that
  their peers use money to purchase votes.
• 23 of entrepreneurs in Sector A had a
  background as government officials, compared to
  87 in sector B.
• Typologies, two-dimensional tables
• Timelines

65
Early Analysis of Quantitative Data

• Consider having quantitative data entered in the
  field
• Run preliminary analysis on a random sub-set
• Use the results of that preliminary run to
• Draw some initial inferences that you can discuss
  with your advisors
• Orient subsequent research and data collection

66
Assess Your Progress

• Develop strategies to evaluate the state of your
  project
• Identify where are you on the to get list
• Do you have enough evidence to address
  alternative hypotheses present in the
  literature?
• Assess how are you building toward a specific
  argument
• What were your initial hypotheses? Do any of them
  seem to hold?
• If not, what are the contours of your new
  argument?

67
Assess Your ProgressKeep an Eye on the Big
Picture

• When you return from the field, you will be
  expected to write an introductory chapter, which
  will include
• Your Question/Puzzle
• Your Argument
• Plus, the theoretical implications of the
  argument
• As you are assessing your progress, ask yourself
  what your argument has to tell us about Big
  Literatures in political science.

68
Assess Your ProgressSolicit Input from Others

• Get help to correct for field goggles
• Keep in touch with your advisors even if theyre
  not keeping in touch with you!!!
• Check in with your more junior network
• Advisors and colleagues can help you keep an eye
  on the forest. not just the trees

69
Making Adjustments

• Dont worry unless you have to
• Its common for people to make some adjustments
• Two Common Responses
• Holy Cow! Nothing is what I thought!
• X, Y, and Z are waaay more interesting than my
  topic.
• Does this warrant a dramatic change in your
  project? (See handout)
• If you do need to switch tracks, its better to
  know sooner rather than later
• Set a deadline

70
Know When to Quit

• Develop some criteria to help you determine when
  you have enough
• Check your to get list
• Identify what information is only available
  abroad
• Keep in mind that you often dont need as much as
  you think!
• Carefully consider how much paper to cart home
  (or between countries)
• Shipping costs can be pricey
• 50 lb. suitcase limits harder to carry it all
  home
• Remember, a lot of material is available in the
  USA

71
Prepare for Post-Field Blues While in the Field

• Consider social and intellectual reintegration
  strategies
• Request office space from your home dept
• Join a dissertation-writing support group
• To TA or not to TA?
• Helpful Plan out your first month back
• Unpack your boxes ASAP
• Unfinished follow-up and thank you notes
• Data to transcribe, code, enter, or clean up
• Drop in to reconnect with your advisors - dont
  hide!

72
The Return Trip

• Spain, China, and Brazil are not going to
  disappear when you leave!
• You already know SO much more than when you first
  arrived, so when you go back you can hit the
  ground running

73
Where We Are in the Course

• Part I Preparing for Field Research
• 1. Field Research and Your Project (Ben)
• 2. Preparing for Field Research (I) (Diana)
• 3. Preparing for Field Research (II) (Sara)
• 4. Discussion
• Recess
• Part II Working in the Field
• 1. Obtaining and Managing Data (Ben)
• 2. Interacting in the Field (Diana)
• 3. Analyzing, Assessing, Adjusting, and Wrapping
  Up (Sara)
• 4. Discussion
• Part III Workshop