Title: Constructing a CrossSite Evaluation of Ethnic Minority HIV Mental Health Services The Mental Health
1Constructing a Cross-Site Evaluation of Ethnic
Minority HIV Mental Health ServicesThe Mental
Health HIV Services Collaborative (MHHSC) Program
- Michael Costa, Abt
- Barbara J. Silver, CMHS
- Maria Madison, Abt
- Tandiwe Njobe, Abt
- Gabriela Garcia, Abt
2Presentation Goals
- Describe the MHHSC Program
- Components
- Context
- Relevance
- Convey
- The process of this collaborative.
utilization-focused evaluation - Accomplishments to date (products)
- Uses of the outputs of products
3Substance Abuse Mental Health Services
Administration(SAMHSA)
- Center for Mental Health Services (CMHS)
- Center for Substance Abuse Prevention (CSAP)
- Center for Substance Abuse Treatment (CSAT)
4SAMHSA HIV/AIDS HISTORY
- Prior to 2001 CSAT funds targeted
capacity/HIV/AIDS Substance Abuse treatment
programs for African American, Hispanic/Latino,
and other racial/ethnic minorities. - 2001 CMHS funds a similar targeted/expanded
capacity program for community based
organizations (CBOs) serving African American,
Hispanic/Latino, and other racial/ethnic
minorities.
5HIV Infection among People with Severe Mental
Illness
Across all published studies, the rate of HIV
infection among psychiatric patients is 10, 25
times higher than that of the general
population. Cournos McKinnon, 1997Krakow et
al., 1998Rosenberg et al., 2001
6Sexual Risk Behavior Among People With Severe
Mental Illness
COMPARED TO GENERAL POPULATION, PATIENTS HAVE
FEWER EPISODES OF SEX WITH A PARTNER, BUT THEY
HAVE
- ? Number of partners
- ? Number of risky or anonymous partners
- ? Frequency of sex trading
- ? Rates of coerced sex
- McKinnon et.al., 1996, 1999
7Psychiatric Disorders and Risk for HIV Infection
- Elevated risk for HIV infection in psychiatric
- Patients
- Risk factors
- Alcohol and other drug use
- Unsafe sex
- Environmental circumstances (poverty,
institutionalization, etc.) - Substance use is associated with both psychiatric
symptoms and HIV risk
8MHHSC Program
- 21 Mental Health Service Sites CBOs
- at least 2 years experience in behavioral health
care services - MH Centers, Substance Abuse facilities,
- Primary Health Care /or HIV/AIDS clinics
- Abt Associates, Inc. Coordinating Center
9MHHSC Program
- Congressional requirement (CBC CHC) provide
these new HIV/AIDS-related mental health services
in both traditional and non-traditional settings. - Funding for mental health treatment services and
related case management only. - However, grantees are required to develop
comprehensive integrated individual treatment
plans and monitor primary and substance use
treatment.
10WHO ARE THE SERVICE SITES?
- New HIV/AIDS-Related Services
- New services (no prior HIV/AIDS-related MH
services) 5 sites - Expanded services 16 sites
- Service Delivery Settings
- Traditional (primarily clinic-based) 13 sites
- Non-traditional (e.g., mobile treatment, ) 1
site - Both settings 8 sites
- Target Populations
- African American -19 sites
- Hispanic/Latino 14 sites
- Haitian 1 site
- Native American 1 site
11Demographics Gender
12Demographics Race / Ethnicity
Hispanic Latino/a 30.65
13Demographics - Age
14DSM IV Diagnoses
15DSM IV Diagnoses Categories
16MHHSC Program Goals
- Expand
- Effective
- Culturally Competent
- Mental Health Services
- For PLWHIV
- In Minority Communities
17MHHSC Capacity Building
- The Coordinating Center provides technical
assistance to grantees - Local and regional trainings (e.g., ethics,
neuropsychology, cultural competence) - Expert speakers at national meetings on topics of
mental health and psychiatry, gender issues,
consumer involvement, etc. - Assistance with data collection and management
18CULTURAL COMPETENCE
- Strategies to Promote Culturally-Competence
Service Delivery - Demographically similar staffing as target
population(s) - Treatment facilities reflect cultural interests
of target group(s) (e.g., artwork/decor, waiting
room music, etc.) - Diversity/cultural competence training
- Consumer feedback
- This is being addressed in great detail by the
MHHSC Cultural Competence Subcommittee
19MHHSC Utilization-Focused Cross-site Evaluation
- Clinically/Programmatically Relevant Evaluation
- meet the needs of the clinical and other program
staff in their efforts to better serve their
clients - The MHHSC cross-site evaluation is voluntary,
except for submission of required aggregate data - critical that the cross-site evaluation be
clinically and programmatically relevant and
valuable, otherwise no site would agree to
participate
20Evaluation as a Pain in the Neck
- Mental health/social service providers
historical experience with evaluation - Intrusive reporting requirement
- Necessary to satisfy Local, State and Federal
funding requirements. - Often data are never reported back to programs
- Concerns that data will used to make them look
bad or draw inappropriate comparisons between
sites and providers
21MHHSC X-Site Evaluation Collaborative Process
- Engaging the key stakeholders
- Local site-specific evaluators
- Local site clinicians
- Program Administrators
- Consumer Advisory Boards (CABS)
22MHHSC X-Site Evaluation Collaborative Process
Evaluation Subcommittee (ESC)
23MHHSC X-Site Evaluation Collaborative Process
24X-Site Candidate Evaluation Foci
25Quantitative Methods Used Across Sites
26Qualitative Methods Used Across Sites
27MHHSC X-Site Evaluation Collaborative Process
28MHHSC X-Site Evaluation Collaborative
ProcessNext Steps
29Final Cross Site Foci
- Four main domains of interest across sites
- Mental Health
- Client Satisfaction
- Client Retention and Service Utilization
- Quality of Life Medical Health
- Cultural Competence appears in all domains.
30Common Questions of Interest
- Who is being served?
- What are the barriers to care and to services for
the target population? How do programs overcome
these barriers? - What services are being used by the target
population? - Are the services being provided in a culturally
competent manner?
31Mental Health
- Given the target population being served, what
are the - Prevailing mental health diagnoses?
- Co-occurring disorders? (e.g., substance use)
- Changes in mental health symptoms?
- Disorders that may be more prevalent with HIV
positive status? - Physiological HIV disease factors that contribute
to mental health symptoms?
32Client Satisfaction
- Questions relating to care received by clients.
- What care is being received?
- What is the frequency of care?
- What is the level of client involvement in care?
- What is the setting in which care is received?
- Is the client satisfied with the care?
33Client Retention Service Utilization
- Client Retention
- How many cases become active/inactive in a given
time period? - What is the site definition for active/ inactive
clients? - What are the site policies for case closing?
What reasons are given for case closings? - What retention strategies have been effective
with the target population?
34Client Retention Service Utilization -
continued.
- Service Utilization
- What is the type, frequency and duration of
services used? - How are services provided?
- Referrals in and out of program
- Agency linkages - in-house and external - to
other providers - Service setting traditional/ non-traditional
35Quality of Life
- How does the quality of life status and general
medical health of a client impact upon - Treatment compliance (medications, ITP)?
- Response to treatment?
- Client retention?
- Service utilization?
- How does quality of life status and general
medical health change over time with treatment?
36Methods Indicators
- CLIENT LEVEL INSTRUMENT
- confidential unique client id
- Demographics
- Social supports
- Acculturation
- Substance use and risk behavior
- Trauma
- Medical health
- Medication adherence
- Clinicians report
37Methods Indicators
- CLIENT SATISFACTION SURVEY
- anonymous
- Client characteristics
- Service utilization
- Care
- Client involvement in care
- Access/ barriers to care
- Cultural competence in care
- Overall satisfaction with care
38Methods Indicators
- FOCUS GROUPS
- with clients on site
- Focus groups will provide qualitative backdrop
to analyze quantitative data from client
satisfaction survey and client instrument - Types of services used
- Satisfaction with services
- Barriers to care
- Cultural competency
39Methods Indicators
- SITE VISIT DATA COLLECTION
- MHHSC Coordinating Center Staff
- Continuum of services
- Location of site
- Geographic setting
- Site community
- Service setting (traditional / non traditional)
- Client retention strategies
- Site activities to overcome identified barriers
to mental health care
40Utility of Cross-Site Analyses From the Sites
Perspective
- Site buy-in has been an on-going process
- Clinicians and local evaluators participate in
the evaluation design work group - They took ownership of the evaluation design
- Made decisions on utility of collecting certain
types of data across sites
41Client Focused Domain
- Client Characteristics
- Clients Presenting Diagnosis
- Barriers/Access to Care (enhance/maintain
Clients Quality of Life/Health)
42Program Context Domains
- Program Structure
- CBO VS Large System of care
- Staffing patterns (FT, PT)
- Sustainability Efforts
- Plan in place
43Client Focused Domains
44Client Focused Domains (cont.)
45Client Focused Domains (cont.)
46Program Context Domains
47Program Context Domains (cont.)
48Program Context Domains (cont.)
49Purposes/Uses of Data
- Cross-site analysis
- Attention to differences across sites, client
characteristics - Careful appropriate
- Nuanced, not reductionist
- Program context critical
- Use of qualitative and program-level data (e.g.,
differences in resources/capacity available)
50Purposes/Uses of Data (Recap)
- UTILIZATION-FOCUSED EVALUATION
- Data/analysis to be provided to sites
- User friendly feedback
- A timely manner
- For program improvement /development of better
intervention strategies - NOT REPORT CARDS
- Individual clinicians
- Individual sites