Check Yes or No: Creating a WellDesigned Survey Research Process

1
Check Yes or No Creating a Well-Designed
Survey Research Process

• October 20, 2006
• 900 AM - 1030 AM  

2
Case Research Community
3
Objectives

• Review of Human Subject Research Definitions and
  Policies
• Review of IRB role
• Types of survey research
• How survey research is done
• Survey research challenges
• Things to Consider Helpful Hints
• Survey Research References
• Survey Research Databases

4
Definition of Research

• A systematic investigation, including research
  development, testing and evaluation, designed to
  develop or contribute to generalizable knowledge.
  
• 45 CFR 46.202

5
Definition of a Human Subject

• A living individual about whom an investigator
  conducting research obtains
• data through intervention or interaction with the
  individual, -or-
• identifiable private information.
• 45 CFR 46.202

6
The Belmont Report
The National Commission for the Protection of
Human Subjects of Biomedical and Behavioral
Research. April 18, 1979

• Basic Ethical Principles
• Respect for Persons
• Individual autonomy
• Protection of individuals
• with reduced autonomy
• Beneficence
• Maximize benefits
• minimize harms
• Justice
• Equitable selection of subjects
• distribution costs and benefits

7
Review of IRB Role

• The IRB is an administrative body established to
  protect the rights and welfare of human research
  subjects recruited to participate in research
  activities conducted under the
• auspices of the institution with
• which it is affiliated.
• IRB Guidebook OHRP
• Department of Health Human Services

8
Types of Survey Research

• In Person
• This also includes focus groups and group
  discussions.
• Can take place anywhere.
• Telephone
• The most popular means of
• data collection.
• Mail
• Slow return and subject to errors
• Could be costly
• Internet/Web-based
• Growing in popularity
• Useful, but not as reliable as one would think

9
When did survey and public opinion research start
. . . and why?

• The first known survey done in the United States
• was the U.S. Census of 1790. While some
• organized survey research was done
• between then and the early 1900's, the
• real growth of the industry came after WW II.
  Surveys and You, CASRO
• website
• Survey research is the most commonly
• recognized and applied social science
• research methodology for studying
• large populations.
• University of Illinois _at_ Chicago

10
How is survey and public opinion research done?

• Most survey research simply involves surveys
  among a representative sample of individuals.
• Very carefully designed questionnaires, usually
  administered in a structured manner, are used to
  guide the interview.
• They could be about attitudes, needs or
  preferences. The questions could be
• closed-end types (for example
• yes or no) or they could be
• open-ended
• (what do you think of. . .).
• Surveys and You, CASRO website

11
How is survey and public opinion research done?

• There are no right or wrong answers
• in survey research. Researchers should only
  be interested in the opinions of the people
  being questioned.
• If done carefully, sampling is more efficient
  than interviewing everybody.
• Even the U.S. Census Bureau interviews
• a small, but very carefully chosen
• sample at the end of its regular
• Census, and uses the results to adjust
• the findings of the whole Census!
• Surveys and You, CASRO
• website

12
How is survey and public opinion research done?

• Smaller samples are obviously much
• more economical for research
• purposes and, because they are small,
• can be controlled better for fewer
• errors.
• A properly chosen sample can be
• perfectly representative of the whole.
• For example, doctors take only a
• sample of blood for a blood test.
• Surveys and You, CASRO website

13
Survey Research Challenges

• Paper/Manual Data Collection
• Training
• Who will be administering interviews/surveys?
• Are they trained by RI? Do they know how to take
  no for an answer?
• CREC-Certified? (not mandatory for CIs,
• but it cant hurt)
• Recruitment
• Where and how would RI obtain
• names/contact information?
• Permission to use names, if applicable?
• Scripts, Flyers, Emails, etc.
• Follow-up How, where how often?
• Be careful not to coerce

14
Survey Research Challenges

• Data Collection
• Informed consent and/or a waiver
• How will data be collected would collection
  method be understood by subjects?
• Privacy and Confidentiality concerns
• Will subjects be able to change their
• responses? If so, how? If not, do subjects
  know that?
• Data Storage/Disposal
• Data must be stored separately from
• consent in locked drawer/office.
• PCs/Laptops/Jump drives must be
• password protected
• Destroy identifiable data/master lists ASAP

15
Survey Research Challenges

• Electronic Data Collection
• Authorization to access websites or chat rooms.
• Does the site/room consider itself a private
  place?
• Does it have research policies?
• Permission from webmaster, listserv manager or
  owner
• Recruitment
• All advertisement must comply w/IRB
• guidelines
• Authenticate subject respondents.
• i.e. Provide PINs for subjects
• Data Collection
• Data should be encrypted
• Note that encryption standards vary

16
Survey Research Challenges

• Server Administration
• Recommend a professionally administered survey
  server
• Server should have firewalls, limited access, and
  frequent security audits
• Data Storage/Disposal
• Store personal identifying information, email
• and IP addresses separate from data
• Encrypt and backup data
• Store in safe/secure location
• Properly destroy data ASAP
• If safeguards cannot be assured, inform
• subjects that complete confidentiality
• CANNOT be guaranteed

17
Survey Research Challenges

• Internet Research Issues (NCPP Not reliable)
• No clear evidence that samples are accurate
• Only 1out of 3 American adults have internet
  access
• No clear evidence that methodology worked
• How would you know your study asked for the right
  information?
• Most electronic studies permit
• participating more than once
• (i.e. American Idol)
• Ballot stuffing

18
Survey Research Challenges

• Internet Research Issues (NCPP Not reliable)
• Does a web-based organization
• have a good track record?
• Professional research company with a good
  reputation?
• Does a web-based organization
• comply w/NCPP, CASRO, or AAPOR
• codes standards of conduct?
• No different than Paper/manual standards
• Is a web-based organization willing
• to disclose questions methodology?
• Can you explain it withstand scrutiny

19
Rights of Survey Participants

• Every Belmont Report principle
• applies
• Informed consent
• Subjects appropriately informed about surveys
  intentions and how their personal
• information and survey responses
• will be used and protected
• Participation is voluntary
• Protect subject identifiers

20
Rights of Survey Participants

• Compensation Is it pro-rated?
• Special care interacting
• w/vulnerable populations
• Is there a power differential?
• If so, how will that be addressed?
• Should you be RI?
• Verify researcher credentials

21
Researcher Responsibilities

• Understand privacy confidentiality
• How will you protect subjects data?
• Be prepared to justify results
• To IRB, Integrity Boards, Publishers
• Research trumps marketing
• Regulatory agencies understand
• research standards are higher
• Be clear about what will
• be done w/subject info
• Informed consent is a process!

22
Researcher Responsibilities

• How will final results be disseminated?
• Survey sponsor
• When how was data collected?
• Sampling method
• Population sampled
• Consistent wording and questions
• and understandable language
• Percentage of responses
• Was there follow up?
• Incentives for surveyors /or participants

23
Things to Consider

• Privacy Confidentiality
• Privacy - having control over the extent, timing,
  and circumstances of sharing oneself (physically,
  behaviorally, or intellectually) with others.
• Confidentiality - methods used
• to ensure that information
• obtained by researchers about
• their subjects is not improperly
• divulged.

24
Things to Consider

• Using Secondary Data
• Secondary data are data that already exists prior
  to the initiation of research.
• Secondary data sets are considered identifiable
  private information if they contain individual
  identifiers.
• Data about employer, job title, age
• and gender may not individually
• identify a subject, but, when
• combined, could in certain cases
• identify a specific individual.

25
Things to Consider

• Using Secondary Data
• If the information does not include individual
  identifiers and can not be linked to a living
  individual (anonymous) by the investigators
  conducting the secondary data analysis (RI or
  student), or is considered public or is given
  with the expectation that it will be made public
• and that it will be linked to the individual
• (i.e., biography, news story), then it would
• not be considered research involving
• identifiable private information and
• not require an IRB application as it is
• not human subject research.
• The IRB determines if an application is required.

26
Things to Consider

• Standardized Questions Usable Responses
• Questions have to be asked in exactly the same
  way of everybody--even though certain questions
  may be more interesting, or seem more relevant,
  to some people than to others.
• Changes in questions could cause a
• situation where a specific question
• may not seem meaningful, or is
• difficult to answer, to a specific person.
• Surveys and You, CASRO website

27
Things to Consider

• Know your research and methodology Have a clear
  goal.
• Carefully develop the questions. In what order
  will they be asked?
• Pretest questionnaires and procedures to identify
  problems prior to the survey CANNOT KEEP DATA!
• Train interviewers carefully on interviewing
  techniques and the subject matter of the survey.
  Maximize
• cooperation or response rates within the limits
• of ethical treatment of human subjects
• Know the potential ethical issues associated
• with your research
• Carefully develop and honor
• confidentiality given to respondents.
• Standards Best Practices,
• AAPOR 2002
• National Council on Public Polls

28
Helpful Hints

• Plan ahead
• What is your goal?
• Do you plan to publish?
• Dissertation, Thesis or Classroom?
• Use correct current
• version of the IRB forms
• Answer all application
• questions fully completely

29
Helpful Hints

• Attach all applicable
• documents including
• online documents
• Contact the IRB if you have
• any questions
• Be prepared to receive
• feedback from the IRB

30
Survey Research References

• American Association of Public Opinion Research
• www.aapor.org
• Council of American Survey
• Research Organizations
• www.casro.org/
• National Council of Public Polls
• www.ncpp.org
• Great information on how to explain work to the
  press/public

31
Survey Research References

• Survey Research Laboratory
• University of Illinois at Chicago
• http//social-sciences.uchicago.edu/survey-lab/
• ESOMAR
• The World Association of
• Research Professionals
• www.esomar.org
• U.S. Census Bureau
• www.census.gov

32
Survey Research Databases

• The Roper Center for Public Opinion
  Researchhttp//www.ropercenter.uconn.edu
• OhioLINK databasehttp//www.ohiolink.edu/resource
  s.cgi
• Community Research and
• Development Information
• Servicehttp//www.cordis.lu/en/home.
• html

33
Survey Research Databases

• Fedstatshttp//www.fedstats.gov
• The Washington Research Library
  Consortiumhttp//www.wrlc.org
• Case Linkhttp//msass.cwru.edu/library
• /ref/journal/resdbs.asp

34
Contact Information

• Case Behavioral/Social Science IRB
• Administrative Office
• Office of Research Compliance, Sears Building,
  6th Floor
• Location Code 7230
• cwru-irb_at_cwru.edu
• IRB Director Isabel Sánchez
• ias5_at_cwru.edu
• 368-6993
• IRB Assistant Maureen Dore-Arshenovitz
• mxd4_at_po.cwru.edu
• 368-6925