Emergency Room Survey

1
Emergency Room Survey

• Survey Planning, Administration and Cost

2
SIDUC AN INTEGRATED SYSTEM OF DRUG USE SURVEYS
Key Informants
FORENSIC MEDICINE
Detention Centers
HOUSEHOLD SURVEYS
JUVENILE OFFENDERS
EMERGENCY ROOMS
STUDENT SURVEYS
TREATMENT CENTERS
3
SIDUC STANDARDIZATION
4
EMERGENCY ROOMS

• Population of reference
• Patients of Emergency Rooms.
• Analyzed population
• a) all patients b) typical weeks. Judgmental
  sample.
• Method of data collection
• Interview laboratory analysis.

5
FORENSIC MEDICINE

• Population of reference
• Death by accident, homicide, suicide.
• Analyzed Population
• a) all , b) Typical weeks.
• Method of data collection
• Autopsy and laboratory analysis.

6
Steps to be Taken

• A funding source or multiple sources must be
  found (or no survey)
• Decisions must then be made about who will direct
  the research undertaking
• What types of expert and staff support will be
  needed, and
• What is a realistic timetable
• for the overall endeavor
• for the major components

7
Session Outline

• Core Survey Activities
• Personnel
• Budget
• Schedule

8
Core Activities

• Establishment of Advisory Committee
• Develop Proposal
• Seek Permission to conduct
• Finalize Methodology
• Training
• Survey Implementation
• Data Collection, Handling, Analysis, Report
  Writing

9
Advisory Committee

• Stakeholders identification/representation
• Ministry of Health
• Public Hospital Authority
• ER Director
• Police
• Ministry of Transport
• Rep from private hospital
• COB Research Unit
• N.Bgt Can avoid problems at the end
• Can help to sell the survey

10
Communications Regarding Survey

• Ministerial Responsibility (CMO)
• Institutional Responsibility
• Public Hospital Authority (PMH, RMH)
• Institutional Review Boards/Ethics Comm.
• How to contact the chosen hospitals
• Administrator
• ER Manager

11
Contact With Selected Institutions

• Step 1
• An introductory letter to the Administrator,
  informing him/her of the study and its purposes.
  It will also
• Ask to inform the ER Manager of decision
• Step 2
• Contact the ER manager via telephone to confirm
  that everything is in order just prior. At this
  point

12
Project Personnel

• Lead Investigator(s)
• Core Staff
• Site managers
• Data Managers
• Data Collection Staff
• Doctors
• Nurses
• Medical Records Clerks
• Consultant(s)

13
Personnel Selection Process Options

• Advisory Committee to provide oversight of the
  enterprise and the selection of the lead
  investigators
• AD can help to engage people or organizations
  needed to fund, conduct, or make use of the
  research
• DIS too many cooks in the kitchen and may
  introduce political considerations into the
  process
• REC committee should deal with the broad policy
  and financial issues and leave the scientific
  decisions to the scientists
• Recruitment of one or more individuals to run the
  project
• Individual researchers may take the initiative to
  conduct such a survey

14
Lead Investigator

• Ideally will be a part of the activity from
  beginning to end
• Will provide the planning and integration needed
  to be sure that the end product matches the needs
  and objectives
• Should be trained social scientists with some
  experience in survey research techniques

15
Core Staff(Site Managers)

• These key individuals will remain with the study
  for its duration
• Participate in various activities and make sure
  that they are carried out according to plan.
• Most likely will supervise various components of
  the study, under the general direction of the
  lead investigator(s)
• Should be well educated and preferably have some
  experience with research activities.

16
Data Collection Staff

• Decide whether the data will be collected by
  individuals from outside the hospitals or not
• ER Staff If the patients will trust the MDs or
  Nurse to protect their confidentiality
• Outside Staff If the staffing situation presents
  a challenge, then staff members must be hired,
  trained, supervised and usually compensated
• trained health social workers, trained field
  interviewers from a survey research organization,
  university students, etc

17
Consultants

• Overall Planning
• To direct technical assistance at various stages,
  depending of course on the areas of expertise of
  the lead investigators
• If the lead investigators are new to this area of
  research, consider a short-term consultation with
  an experienced expert in the initial planning
  stage, again at the analysis planning stage, and
  perhaps at the interpretation stage
• If brought in from abroad, seek the assistance
  and support of one of the international
  organizations that deal with the control of drug
  abuse (UNODC, OAS)
• Sample Planning
• Consultation with a sampling statistician also is
  likely to be very helpful, again at an early
  point in the planning effort
• Statistical Analysis
• If neither a general consultant from the
  substance abuse field nor statistician is able to
  help on data analyses, seek an expert on
  statistical analyses.
• The job is usually not to actually conduct the
  analyses but to advise on the choice of analyses
  and appropriate computer programs for conducting
  them.

18
Budget Planning

1. Personnel Costs
2. Non-Personnel Costs

19
Factors that May Impact Personnel Budget

• Labor costs in the country
• Whether it is necessary to use data collection
  staff from outside the hospitals
• The method of data entry
• The need for technical assistance
• The size of the sample needed and the size of the
  country across which that sample will be spread
• influence both staff and travel costs
• Amount of the personnel assigned to plan and
  conduct the ER survey who are already on the
  payroll of a participating agency.

20
Sample Budget Outline
PERSONNEL COSTS Lead investigators Field staff (if applicable) for __ weeks Supervisors/Site Manager Facilitators/MDs, Nurse, Med Rec. Core support staff Secretarial/Clerical Subcontractor Costs For data entry (if applicable) For any other services being purchased Consultants Fringe benefits NON-SALARY COSTS Office Rental (if applicable) Furniture (if applicable) Equipment (as needed) Phones, Fax, Copy machine, Computers Supplies Telephone service Advertising/Recruiting Costs Printing Questionnaires Manuals Brochures, instruction sheets, etc. Final report Shipping and postage Questionnaires to hospitals Questionnaires back from hospitals Other Travel Costs For investigators For field staff (if applicable) Overhead charges (if any)
21
Estimated Budget for Bahamas Emergency Room Drug
Survey
22
Estimated Budget for Bahamas Emergency Room Drug
Survey (Contd)
23
Survey Scheduling
24
Scheduling Tips

• Considerable time and expense can be saved by
  undertaking several streams of activity
  simultaneously
• Anticipating which efforts need to be completed
  before the next steps can proceed
• Schedule should not be too abbreviated as there
  may be some unexpected developments
• Effort should be made to make realistic estimates
  of the time necessary to complete each line of
  activity
• Field staff should not be promised work until the
  investigators actually expect to proceed with the
  data collection (increased costs)

25
Survey Scheduling

• Hospital Recruitment
• Data Collection
• Analysis and reporting
• Dissemination and utilization activities

26
Scheduling Hospital Recruitment

• If participation is decided by central edict
  (Ministry of Health), the process may be fairly
  rapid.
• If involved in the planning, this assures their
  willingness to cooperate
• If the individual hospitals or hospital
  authorities have the authority to decline
  cooperation, the process of securing cooperation
  can be a substantial and time-consuming one.
• Investigator(s) may have to write to each
  hospital administrator inviting participation
• Conduct a follow-up call (or possibly a series of
  calls) to urge the hospitals participation and
  answer questions
• Communicate with higher authorities, if their
  approval is also required.

27
Scheduling Data Collection

• Once the main data collection is proceeding, the
  responsible investigators should be monitoring
  carefully the quality of the data being
  collected, to be sure that those collecting it in
  the field are following instructions, and to
  identify problems early that might be rectified.
• Plans can also be underway for how the data is to
  be collected and forwarded and how the returned
  data are to be coded and/or edited
• Also, the investigators can begin planning the
  analyses that they would like to conduct at the
  completion of data collection and data cleaning.

28
Scheduling Analysis and Reporting

• Often not accorded the attention that they
  deserve because not enough time and resources
  were set aside for them at the outset of the
  study.
• Be sure to leave a significant interval for the
  analysis, interpretation, and writing of results.
• At this late stage, normally only the lead
  investigator(s) and an analyst or two are still
  needed on the study staff, perhaps in addition to
  a secretary.

29
Scheduling Dissemination and Utilization
Activities

• Once the report is completed, arrangements should
  be made to get it into the hands of people likely
  to be influenced by its results.
• The lead investigators may wish to meet with
  certain groups, or make presentations to
  particular audiences to whom the work has
  relevance.
• Policy and programmatic level

30
Flow Chart of Activities for the Conduct of a
School Survey
 INSTRUMENT DEVELOPMENT AND REFINEMENT Develop questionnaire ? Pretest questionnaire ? Pilot test questionnaire field procedures ? Make final revisions to questionnaire field procedures PERSONNEL Select lead investigators ? Hire core staff    FIELD PREPARATIONS Develop field procedures ? Hire and train field staff (if applicable) ? Develop instruction manual for survey leaders  SAMPLE HOSPITAL RECRUITMENT Design sampling plan ? Secure data for selecting hospital sample ? Draw sample of hospitals ? Recruit the sampled hospitals ? Schedule the data collection in each hospital
31
Flow Chart of Activities for the Conduct of a
E.R. Survey
CONDUCTING THE MAIN SURVEY Conduct full data collection ? Code/clean/edit questionnaires ? Enter data ? Build computer file ? Conduct data analyses ? Write reports ? Disseminate/utilize results
32
Ethical Considerations

• Data must remain completely confidential or
  completely anonymous
• no identifying information on a patients
  questionnaire
• If specific identifying information for the
  individual is contained on the questionnaires
  de-identify at the earliest possible stage
• Store the personally identifying information
  separately from the answers to the rest of the
  questionnaire with some type of link system
• Avoid publicly identifying individual hospitals
  if possible
• Permission to participate at all levels
• Commitment to use results

33
Emergency Room Survey

• QUESTIONNAIRE DEVELOPMENT

34
Factors That Impact Final Content

• Amount of time the medical staff can afford to
  make available in an AE setting
• Quality, completeness
• Treatment Vs Prevention
• SIDUC Rules
• Possibility of including other topics due to cost
  sharing with other agencies
• Ability of respondents to complete the
  questionnaire
• Complexity should not exceed respondents
  capabilities

35
Levels of Priority
Highly Recommended Necessary to achieve objectives. Questions likely to be important to almost any epidemiology study of substance use
Recommended Those that should be given very serious consideration Risk and Protective Factors Other sources Injury Surv. Questions from Stakeholders
Optional May not measure concepts of vital interest in every study
36
Sequence and Priority of Elements In The
Questionnaire

• Introduction
• Background and demographic characteristics
• Age, Gender, Employment, Occupation
• Use of alcohol
• Use of illicit substances
• Use of controlled substances

37
List of Drugs
Alcohol Cocaine HCl. Coca Paste
Marijuana Tranquilizers Methamphetamines
Solvents and Inhalants Sedatives Flunitrazepam (Rohypnol, etc)
Hallucinogens Amphetamines Crack Cocaine
Heroine Anticholinergics Ecstasy
Opium/Morphine Anti-depressants Other
38
Defining Drugs For The Respondent

• Names and descriptions must be reviewed to see if
  they are appropriate in the cultural setting in
  which they will be asked.
• The main point is that the names used accurately
  communicate to respondents which substance(s)
  should be included in what they report, and which
  substances should not, and under what
  circumstances.
• To determine a list of appropriate slang or
  street names for various drugs, the investigators
  may want to speak with treatment professionals
  and known drug users in the age group under
  study.
• If a longer question stem is required to get the
  definition clarified for your respondents, then
  that may be justified. (See SIDUC Manual)
• For legally prescribed drugs, it is important
  that the respondents understand what occasions of
  use they should and should not report in
  answering the questions.

39
Definition of Drugs
Formal Name Street Name
Cocaine Coke, crack, white lady, blow, etc.
Marijuana Weed, herb, grass, refer, pot, ganja, etc.
40
Example of A Stem Change
Concern General question Vs specific questions that will ensure that all questions are answered by all respondents Concern General question Vs specific questions that will ensure that all questions are answered by all respondents
Original (OAS) Change
Have you taken any drugs within the 6 hours preceding your injury Have you taken any of the following within the 6 hours preceding your injury Alcohol Marajuana etc
41
Layout In The Event of Lab Confirmation

• Blind Vs Not Blind
• If Not, questions and Lab results on same form
• No means to connect necessary
• IF blind, questions and lab results are separated
• Unique identifiers are needed to re-unite
• NB identifiers can be added after AE but before
  transfer to lab

42
Other Useful Suggestions

• Skip patterns are more difficult for respondents
  to follow correctly than a simple, uninterrupted
  series of questions therefore, it is advised
  that they be minimized.
• Pre-test by getting a limited number of like
  respondents to complete the questionnaire.
• Determine average time to complete.
• individually interview them about whether the
  instructions were clear and whether there were
  any questions or answers that they had difficulty
  understanding or using.
• Determine if they understood each class of drug

43
Other Useful Suggestions Contd

• Pilot testing to
• see how the actual administration procedures in
  the ER will go,
• see how long it takes the patients to answer the
  questions and
• identify remaining problems in the content and
  clarity of the questionnaire.

44
Emergency Room Survey

• Data Collection Procedure

45
Decisions Prior to Collection

• WHAT What data will be collected
• WHO Who will be responsible for data
  collection
• WHEN When to collect data to ensure a normal
  sample
• WHERE What hospitals are to be included
• HOW How to conduct all aspects of survey
  administration

46
What data will be collected

• Questionnaires
• SIDUC
• Bahamas
• Lab confirmation data
• Site reports on
• refusals
• ineligibles

47
Who will be responsible for data collection

• Conducting the interview of each section of the
  questionnaire
• How to select survey leaders
• AE Data Flow
• Lab sample flow
• routinely collected information
• Ensuring that a continuous supply of
  questionnaires are available
• Ensuring a continuous supply of lab sampling
  equipment are available (?)
• Ensuring that completed forms and lab samples are
  routed properly

48
Selection of Survey Leader

• ER Staff
• Pros
• already in the ER
• know the ER and are familiar with ER routines
• Patients may feel more comfortable with MD or
  Nurse
• least expensive
• Cons
• Competing interests
• ER Services Vs survey completion

• Research Assistant
• Pros
• More consistency across hospitals
• Cons
• More expensive
• doubts regarding the ability to convince patients
  to participate
• E.g., Social Workers in Haiti

49
When to collect data

• SIDUC - Sample
• Vs
• Institutionalized routinely collected as part
  of diagnosis

50
Timing of Data Collection

• It is important to choose a period which should
  not be preceded by any holiday, ensuring that the
  patients refer to a normal week or month when
  answering the questionnaire
• If more than one hospital is included, it is a
  must to administer the data collection at the
  same time in all hospitals
• In the event of inter-country comparisons
  (SIDUC), the time of data collection must be as
  similar as possible since the use of alcohol and
  other drugs vary at certain identifiable periods.
• e.g. December (Christmas) ?
• January (Recuperation) ?
• Holidays ?
• Lent ?

51
Where will data collection take place

• Geographic location
• Islands
• Hospitals
• Public
• Princess Margaret (Nassau)
• Rand Memorial (Freeport)
• Private
• Doctors (Nassau)

52
Activities During Administration

• Instructions to the Survey Data Collection Staff
• Instructions to the patients
• Stress anonymity and confidentiality

53
Who Should Complete Survey

• Inclusion Criteria
• Exclusion Criteria
• Persons lt 12 years of age
• Gynae patients
• Language barriers (Non-English speaking)
• Unconscious patients (?)
• If hospitalized and no identifiers to follow-up

54
Patients who Refuse

• Concern is due to the possible association
  between refusal and drug use
• It is important that the number of refusals be
  documented
• Could compare refusals from low and high
  prevalence areas
• It is important that refusals are treated in the
  same way in all participating hospitals and
  countries.

55
Survey AdministrationInstructions to Patients

• Survey Instructions can be written on the front
  page of the questionnaire and should include
  information on
• The purpose of the study (If not incorporated
  into the system)
• The selection of patients (All selected during
  study period)
• The study is anonymous and/or confidential
• Steps to ensure anonymity
• Participation is important but if not willing,
  that is their choice

56
To Ensure Anonymity

• Following SIDUC Method
• Questionnaires should not contain any identifiers
• Name
• hospital record , etc.

57
Survey AdministrationReport

• A report should be completed (periodically) by
  the survey leader
• Total number of refusals
• If the study only targets a specific group, the
  number of those excluded

58
Checklist For Data Collection

• Chose a survey leader trusted by the patients
• Instructions to the survey leader
• Describe how to treat patients not belonging to
  the target population
• Describe how to treat refusals
• Careful planning of the contacts with selected
  institutions
• Safe transportation of material (lab component)
• Detailed planning of the survey administration,
  including
• stress anonymity and confidentiality
• instructions to the survey leader
• instructions to the patients
• Shift report

59
Overview of Methodological Issues

• Interpretation Of Data

60
Overview of Session

• Representativeness the extent to which a sample
  mirrors the population of interest
• Reliability the extent to which repeated
  measurements used under the same conditions
  produce the same result
• Validity the extent to which answers are
  accurate representations of the underlying
  reality that they are intended to measure

61
Factors that Influence Representativeness

• The sampling method
• The size of the sample
• Response rate The number of eligible non-gynae
  emergency room patients that agree to participate
  in the survey

62
The Method of Sampling Target Population

• The target population in the Emergency Room
  survey is, by definition, persons who chose to
  come to the emergency room and excluding other
  individuals who, for whatever reason, chose not
  to attend.
• Beware that all injured persons do not come in
  due to a number of reasons (choice,
  inaccessibility, etc.)
• Less-severely injured persons who do not come in
  may be the result of reasons related to substance
  use
• Large differences in emergency room attendance
  (access, availability) between countries may make
  it difficult/impossible to make meaningful
  international comparisons

63
Method of Achieving Representativeness

• Random sampling is fundamental to obtaining a
  sample that is representative of the population
• OR
• Sample the entire target group

64
Sample Size and Representativeness

• the number of sampled patients must be of
  sufficient size
• considerations must be given to the extent of the
  analysis of drug habits in different subgroups
• always important to sample enough patients to be
  able to analyse data separately for males and
  females
• Response rate must be sufficiently high to enable
  representative data to be obtained

65
Categories of Response Rates

• Institutional Cooperation
• Public and Private Hospitals
• Patient Cooperation For Interview
• Proportion Participating in Lab Analysis

66
Issues Impacting Institutional Cooperation

• An increased number of proposed surveys has made
  hospitals in some countries somewhat reluctant to
  allow patients to participate directly in such
  surveys
• highlights the necessity of approaching
  institutions with emergency rooms formally and in
  a way that makes them feel that they are part of
  an important study

67
Issues Impacting Patient Participation

• Participation should always be voluntary
• Suspicions regarding True objectives has made
  patients somewhat reluctant to take time to
  participate in such drug surveys
• highlights the necessity of approaching patients
  in a way that makes them feel that they are part
  of an important study
• All questionnaires should be treated
  confidentially
• The use of questionnaires without names or other
  kinds of identification
• Guaranties of confidential treatment of
  questionnaires and data
• Promises not to report data for individual
  patients

68
Decisions Regarding Non-Participation

• If hospitals with a large percentage of overall
  ER visits refuse to participate it is essential
  to do a careful analysis of the reasons
• If systematic errors is suspected, interpretation
  might be difficult and international comparisons
  may be jeopardised.
• E.g.
• refusing patients come from areas where drug
  and/or alcohol consumption is known to be high

69
Reliability

• The extent to which repeated measurements used
  under the same conditions produce the same result
• Or
• Whether an indicator is consistent across time
  and observers

70
Methods of Measuring Survey Reliability

• To conduct repeated studies using same
  methodology
• By using data from different questions within a
  questionnaire
• Laboratory Confirmation

71

• Factors Influencing Study Reliability
• Completeness - proportion of total information
  known
• All respondents answer questions
• All targeted respondents take survey
• Sources of Error
• Instrument
• Person
• MD, Nurse, Med. Rec Clerk
• Study Subject or Respondent
• May occur at
• subject selection
• data collection (Subjective)
• Kappa Statistic
• analysis and interpretation

72
Sources of Error Instrument and Procedures

• Poor selection of questions and variables (SIDUC)
• Ambiguous questions (additional questions)
• Non-response
• Poor selection of study subjects (friendliest)
• Failure to validate measurements, recordings
• lab

73

• Sources of Error
• Person
• Interviewer
• Inter-observer variation
• the way medical staff explain and/or ask
  questions
• Opinions on extent drug/alcohol use contributed
  to injury
• Subject
• Inter-subject variation
• Different Interpretation of questions (what drugs
  were thought of without prompting)
• willingness, survey environment

74
To Achieve Reliable Data

• Thorough knowledge of study population
• Unbiased selection of study subjects
• Standardized, calibrated and consistent
  instrument(s)
• Clear, unambiguous questions
• Minimal personal discomfort or discomfiture
• Interviewers, recorders thoroughly and equally
  well trained
• Need to validate

75
Validity

• The extent to which answers are accurate
  representations of the underlying reality that
  they are intended to measure.
• In the context of emergency room surveys, the
  degree to which the questionnaire measures the
  aspects of patients drug consumption that it was
  intended to measure.

76
Strategies to Enhance Validity

• Studies must guarantee anonymity and
  confidentiality
• One is to use a data collection leader trusted by
  the patients
• Stress anonymity during the introduction
• no names or other identification marks should be
  on the questionnaire (SIDUC)
• The patients must also have enough time to answer
  the questionnaire
• important that the questionnaire is not too long
• They must understand the questions
• They must be willing to answer the questions
  honestly

77
Strategies to Assess Validity

• Patients willingness to co-operate
• Patients comprehension
• Missing data rates
• Logical consistency
• Reported willingness to answer honestly
• The cultural context in which a survey is
  conducted

78
Strategies to Assess Validity Cultural Context

• Questions must be culturally or locally
  appropriate
• Use the appropriate street-names or nicknames
  used for different drugs
• Willingness to admit drug use may be influenced
  by the attitudes towards drugs in a given society
• perceived risk of substance use, disapproval of
  different kinds of substance use and the
  availability of different drugs differ between
  countries.
• Low availability Negative attitude less
  willing to admit
• High availability Positive attitudes more
  willing to admit
• Survey Tradition
• Persons in countries where surveys are less
  common may feel less comfortable answering
  questions about sensitive behaviours.

79
Checklist For Survey Methodology

• Representativeness
• Define the target population
• Assess the importance of non-students in the same
  age groups as the target population
• Decide a proper time for the data collection (if
  international comparisons are planned)
• Assess the importance of non-participating
  schools/classes
• Assess the importance of non-participating
  students
• Reliability
• Assess reliability (whenever possible by using
  data in the questionnaire)

80
Checklist For Survey Methodology Contd

• Validity
• Anonymous and confidential data collection
• Measure and report
• Number of eliminated questionnaires
• Survey leader information (from the classroom
  report)
• Time to answer the questionnaire
• Proportion of unanswered questions
• Logical consistency
• Possible use of a willingness question
• Possible reported use of a dummy drug
• Construct validity

81
The End
82
Really

• The End