Title: Measurements
1 Measurements
- Measure of Variability, Scale Levels of
Measurements, Descriptive Statistics, Measures of
Central Tendency
2Measurements need to
- Produce valid and reliable results
- be sensitive and specific
- be able to identify clinically important changes
- have outcome measures and endpoints defined
- be easy to interpret
3Reasons for errors in measurement
- Improper function or calibration of equipment
- patients providing misleading or dishonest
answers to verbal/written questions - Improper recording/transcribing of data
- Investigators recording or making inaccurate
measurements
4Types of Errors
- Random error
- Random in occurrence, often balancing out over
course of study - mean or average of measurements still close to
true value - Large patient size reduces random error
5Types of Errors
- Systematic error
- represents bias in measurements and does not tend
to balance out over course of study. - Bias can be knowingly or unknowingly
- Good study design minimizes systematic error.
6Measurement Terms
- Validity- degree to which an instrument is
measuring what it is intended to measure. - Predictive, Criterion, Face
- Reliability- reproducibility of a test
- Sensitivity- ability to measure a small treatment
effect - Specificity- how well the test can differentiate
between the effect resulting from treatment and
random variation
7Validity terms used in association with
measurements
- Predictive validity
- the extent to which a measurement or test
actually reflects or predicts the true condition. - Criterion (construct) validity
- the degree to which a measurement or test agrees
with or obtains the same results as other proven
tests designed to measure the same. - Face validity
- the extent to which a measure appears reasonable
or sensible for measuring a desired outcome
8Reasons for False Positive Results
- Patient related
- patients werent as ill as originally believed,
and drug was more effective in mildly ill pts. - Patients were much more ill than originally
believed, and drug was more effective in severely
ill patients. - A few patients had a very large response, which
skewed the overall results. - Patients gradually improved independent of drug
treatment.
9False Positive Results
- Patient related
- More medicine was absorbed than anticipated
- Patients took excess medication.
- Patients felt pressure to report a positive
medicine effect - Concomitant non drug therapy or other drug
therapy improved results
10False Positive Results
- Study Design and Drug Related
- Blinding was broken or ineffective
- open label study can sometimes produce a larger
positive response - no placebo control to help interpret
- error occurred in dosing patients- gave more drug
than intended - inadequate wash-out period, carry over effect
- inappropriate clinical endpoints, tests or
parameters were used
11False Positive Results
- Investigator related
- influenced response by great enthusiasm
- chose inappropriate tests to measure
- Results and Data Related
- systematic error- reporting large drug effect
- high percentage of non-responders dropped out
- not all data was analyzed
12False Negative Results
- Patient Related
- were much more ill than realized
- responded less to the drug than anticipated
- study group had large number of non responders
- non-compliance-- took fewer doses
- concomitant medicines- interactions
- exposed to conditions that interfered with study
13False Negative Results
- Drug Related
- not adequately absorbed
- kinetics were different in study group than in
other patient groups - Study Design Related
- Too few of patients
- inappropriate study design
- insufficient drug dose was tested
14False Negative Results
- Study Design Related (cont.)
- Ineffective tests or parameters used
- Inadequate wash-out period in previous treatment
period - Concomitant non-drug therapy interfered
- Investigator Related
- influenced patients with skepticism displayed
- chose inappropriate tests to measure effects
15False Negative Results
- Results and Data Related
- Patients who improved dropped out leaving higher
number of non-responders - systematic error resulted in reporting of an
inappropriately small drug effect.
16Outcome Measures
- Example A study is performed to compare the
effects two antihypertensives, atenolol and
propranolol in 2 groups of patients with mild
high blood pressure. 2 types of outcomes
measurements are selected for this study
measures of efficacy and measures of safety - Measures of efficacy BP, HR, symptom relief
- Measures of safety adverse effects, blood
glucose, electrolytes, serum lipids
17Criteria Used for Outcome Measures
- Presence or Absence criteria Is sign, symptom
present or absent? - Graded or Scaled Criteria the use of grading on
a scale to measure clinical symptoms - Relative change criteria- measured changes
- Global assessment criteria- Quality of Life
- Relative effect criteria- change in time to
effect.
18Measurement Endpoints
- Endpoints are measurable points used to
statistically interpret the validity of a study. - Valid studies have appropriate endpoints.
- Endpoints should be specified prior to start of
study (should be included in study design) - Quality studies have simple, few and objective
endpoints.
19Endpoints
- Objective- based on actual or measurable findings
or events (heart rate, BP, Temp.) - Subjective- based on thoughts, feelings, emotions
(pain scale, mobility) - Morbidity- quality or condition at the present--
quality of life - Mortality- causing death or a death rate
20Endpoints Example
- In a study determining the effects of clonidine
on quality of life, the researchers determine the
number of days a patient misses work. Each
patient is also asked to complete a rating scale
to describe the degree of fatigue they
experience. - What type of endpoints are used?
- What type of criteria are used?
21Surrogate Endpoints
- These reduce the quality and validity of the
study. - Surrogate or Substitute endpoint examples
- CD4/CD8 ratios instead of survival in studies
for treatment of AIDs. - Measuring volume of acne instead of proportion of
patients cleared of acne. - Determining cardiovascular disease or
atherosclerotic disease instead of measuring
blood pressure in a study of antihypertensive
drug treatment
22Hawthorne Effect
- Refers to the influence that a process of
conducting a study may have on a subjects
behavior - Subject
- Environment
- Research design
23Reasons for Clinical Improvement in a Patients
Condition
- Natural regression to the mean (most acute and
some chronic conditions resolve on their own - Specific effects of treatment (drug or
intervention) - Non-specific effects- attributable to factors
other than specific drug/intervention effect. - Called a Placebo Effect
24Placebo Effect
- A placebo is an intervention designed to simulate
medical therapy, but not believed to be a
specific therapy for the target condition. - A placebo is used either for its psychological
effect or to eliminate observe bias. - Placebo response due to change in pt. Behavior
following admin. of a placebo - Placebo effect change in pts illness due to
the symbolic importance of a treatment. - A placebo effect doesnt require a placebo.
25Why do we see a Placebo Effect?
- Three different theories
- 1. The effect is produced by a decrease in
anxiety - 2. Expectations lead to a cognitive readjustment
of appropriate behavior. - 3. The effect is a classical conditioned
Pavlovian response.
26Placebo Effect
- Expectations lead to behavior change
- Patients and providers expectations
- Patients positive attitude toward provider and
treatment - Providers positive attitude toward therapy
- Provider interest in patient (sympathy, time,
positive attitude) - Compliant patients have better outcomes than
noncompliant patients even with a placebo. - The placebo response is stronger when stronger
drugs are used. - Crossover studies show a stronger placebo
response when given in the 2nd period of study.
27Appropriate Statistical Tests
- To determine whether appropriate statistical
tests have been used, you must know 3 things - 1. The specific research question or hypothesis
being addressed. - The number of independent and dependent variables
- The scales or levels of measurement used for the
dependent variables
28Variables in a Study
- Dependent variables
- those variables whose value depends upon or is
influenced by another variable. - It is the variable that is measured, and the one
that changes as the result of a drug action. - Independent variables
- Those variables which modify a dependent variable
(drug treatment)
29Example
- Patients given Lovastatin to lower cholesterol.
- Dependent variable- lowering of cholesterol
- Independent variable- Lovastatin
- There can be more than one independent and
dependent variable in a study.
30Dependent/Independent Variables
- Example A single blind study of 30 patients with
poison ivy dermatitis were randomized to receive
either topical hydrocortisone 1 or 2 and apply
QID. Severity of the dermatitis was evaluated
daily using a 5 point scale, where 5- severe and
0-none. - What is the independent variable? Dependent
variable? - Example A study was conducted to compare the
efficacy of procainamide and quinidine for
reducing ventricular arrhythmias. The number of
ventricular ectopic depolarizations was
determined in patients both before and during
therapy with either drug. - What is the independent variable? Dependent
variable?
31Scales of Levels of Measurement
- Nominal Level
- variables are grouped into mutually exclusive
categories. - Gender as female or male
- cured and not cured
- response and no response
- include histograms (bar graphs)
- weakest level of measurement
- referred to as dichotomous data
32Scales of Levels of Measurement
- Ordinal level
- ranked or ordered categories
- 1-2-3-4
- severe, moderate, mild, none
- always, sometimes, never
- stronger level than nominal
- not measured quantitatively, but qualitatively
- distance between groups need not be equal
33Scale Levels of Measurement
- Continuous Measurement
- Interval level exact difference between two
measurements is known and constant - has arbitrary zero point
- highest level of measurement
- quantitative data
- Examples BP (mm Hg) serum Theo levels (ug/ml),
WBC (cells/cu mm)
34Continuous Level of Measurement
- Ratio level
- exact differences between measurements is known
and constant - true zero point (Centigrade temp scale)
- can make ratio statements (21) that denote
relative size - Can be converted to an ordinal scale (but ordinal
scale cant convert to interval)
35Scale levels of Measurements
Baseline Pain Assessment 0 1 2 3
(absent) (mild) (mod) (severe) Placebo 0 2 18
14 PainawayR 0 4 12 16 (number of subjects in
each group with varying degrees of baseline pain
intensity. What scale level of measurement?
36Scale level of Measurements
Infectious Outcome Among 46 Patients Infection N
o infection Total Oxacillin 2 20 22 Placebo 0 2
4 24 Column total 2 44 46 What scale level
of measurement?
37Types of Interval/Ratio Data
- Discrete scale of data (non-continuous) when a
measurement has the interval characteristics but
can only be assigned integer values. (HR, number
of patients admitted to hospital/day) - Non discrete (continuous) scale of data each
data point falls on a continuum with an infinite
number of possible subdivisions (temp, BP, BG,
weight)
38Data Distributions
- Once data is collected, it can be organized into
a distribution, or graph of frequency of
occurrence, or chart of the number of times that
each measurement value occurs. - Bar Graphs
- Bar Chart (Histogram)
- Line Graphs
39Data Distributions
- Nominal and Ordinal level data use histograms
(Bar charts) because data classified into
distinct categories - Continuous level data are distributed in the form
of curves and line graphs (normal distributions
and non-symmetrical distributions)
40Bar Chart (Histogram)
41Continuous Level DataNormal DistributionGaussian
Curve
42Non-Normal DistributionsBi-Modal Curve
Weights of American Adults (women and men)
43Non-symmetrical distributionsNon-normal
distributions
44Continuous Distribution Examples
The distribution of GPAs of college students
1.0 2.5
4.0
45Continuous Distribution Example
Distribution of the ages of patients taking
Digoxin
20 40 60 80
46Descriptive Statistics
- Measures of Central Tendency
47Measures of Central Tendency
- Mean-
- mathematical average of a set of numbers.
- Affected by extreme data points (outliers)
- Useful for continuous level data
(interval/ratio). - Ex uric acid concentrations 8,6,5,4,3,2,2,2.
Total number of samples 8. Sum of measurements
32. 32/8 4 (mean).
48Measures of Central Tendency
- Median
- Middle number of a group of numbers in which an
equal number of responses above and below that
point exist. (called 50th percentile) - Not affected by outliers. Useful for ordinal,
interval and ratio data and non-symmetrical. - Ex Uric acid concentrations 8,6,5,4,3,2,2,2.
Since even number, median lies between 4 and 3 or
median 3.5.
49How to Recognize skewed data
- If the magnitude of the difference between the
mean and median is none or small, the data is
approaching normal (symmetrical) distribution. - If the difference between the mean and median is
large, the data usually prove to be skewed.
50Measures of Central Tendency
- Mode
- The most commonly or frequently occurring
value(s) in a data distribution. - Useful for nominal, ordinal, interval/ratio data.
- Only meaningful measure for nominal data.
- Can have more than one mode in set of data
- Ex Uric acid concentration 8,6,5,4,3,2,2,2. The
mode 2.
51Measures of Central Tendency
Scale Level of Normal Non-Normal Measurement D
istribution Distribution Nominal Mode Mode Ord
inal Medianmode Median /mode Interval/Ratio Me
anmedmode Mean/med/mode
52Descriptive Statistics
53Measure of Variability
- Two distributions can have the same mean, median
and/or mode and yet be very different. - Variability refers to how spread out (or close
together) the data are.
54Example
- 2 groups of men w/ mean SBP in each group is 120
mmHg. Are they similar? - First group BP 110,120,120,130
- Second group BP 80,90,150,160
- Both have mean 120
- Spread of data or range of data is much different
55Range
- Range the interval between the lowest and
highest values within a data group. - Can be significantly influenced by outlying data
(extreme values) - Used for ordinal, interval or ratio data
56Interquartile Range
- Measure of variability directly related to the
median. The median represents the 50th
percentile. - The interquartile range is that range described
by the interval between the 25th and 75th
percentile values. - Used for ordinal, interval/ratio data that dont
have normal distributions
57Standard Deviation (SD)
- Standardized measure of the spread of scores
around the mean. - Useful for continuous (ratio/interval) data
- When reported in a study /- 1SD
- Needs normal distribution of data
- The mean /- 1SD includes 68 of data points (34
on each side of the mean)
58Standard Deviation
- Mean /- 2 SD include about 95 of data points
(47.7 of the values on each side of the mean) - Mean /- 3 SD include about 97.7 of the data
points (49.8 of values on each side of the mean) - DBP 100 mmHg /- 5 mmHg includes data points from
95-105 mmHg(assume 1 SD unless tells you
differently)
59Standard Deviation (SD)
- The larger the SD, the further the data points
deviate from the mean (more variable data). - The smaller the SD, the closer the data points
are to the mean (less variable data). - Ex 0.9 /- 0.2mg and 1.1 /- 0.6mg
- Which has more widely scattered data?
- Answer 1.1 /- 0.6 mg (larger SD)
60Variance
- Variance is estimate of the study data. Obtained
by calculating the differences between each
individual value and the overall mean - Needed for calculating the SD
- SD variance
- SD2 variance
61Standard Error of the Mean (SEM)
- SEM is a way of estimating the variability of an
individual sample mean relative to the population
as a whole. - SEM SD/ variance or SD/ sample size
- SEM is used to calculate the Confidence Intervals
(CI) - Improperly used in place of SD because it is a
smaller number and looks better
62SD versus SEM
- Mean Serum Theophylline concentrations of a group
of patients was 13.6 /- 2.1 (1SD). - Conclude about 68 of patients had conc.
Somewhere in the range of 11.5-15.7. - Serum Theo conc. of a group of patients was 13.6
/- 2.1 (SEM) - could assume that if several addl samples of pts
with same characteristics were studied, their
mean values would fall between 11.5 and 15.7 68
of the time.
63Review
- Variance differences between each individual
value and the overall mean. - Variance used to calculate the SD
- SD variance or SD2 variance
- SEM or SE is derived from the SD
- SEM SD/ sample size
64ReviewSEM
- SD measure of the variability of individual
values about the sample mean - SEM/SE measure or indication of the variability
of individual sample means about the true but
unknown population mean. - SEM is used to estimate the reliability
(precision) of a study sample in terms of how
likely it is that the sample mean represents the
true population mean. - SEM is used to calculate the CI
65Example
- In a study of the effectiveness of Drug X on the
Blood Sugar concentrations in 15 patients, the
authors report the mean BS values in the patients
as 150 /- 2.3mg. - Would this represent the SEM or SD?
66Confidence Intervals (CI)
- Represents a range that has a high probability of
containing the true population value. - The likelihood that a study samples value
reflects the true value of the population. - Calculated for a desired level of probability
(95). A 95 CI means there is a 95 probability
that the true population value falls within the
CI range
67Confidence Interval Example
- The mean difference in healing rates between
placebo and penicillin was reported to be 59 (CI
24-72).
68Confidence Intervals
- Can be calculated for nominal level data
(proportions) and continuous level data - 90 CI assoc. with narrower range of values
(dont need to be as confident) - 99 CI assoc. with wider range of values (more
confident the CI will contain true population
value.
69CI is influenced by
- 1. Level of confidence selected
- 2. SEM (larger SEM, wider the CI)
- 3. Standard Deviation (SD) of the study sample.
(larger SD, then larger SEM, then wider the CI) - 4. Size of the study group. (The larger the
sample size, the smaller the SEM, and narrower
the CI)
70Confidence Interval Example
Two similar studies are published about efficacy
of Pravastatin for reducing cholesterol. Both
sets of patients are comparable. Study 1 enrolled
200 patients. Study 2 enrolled 50 patients. The
mean /- SD treatment reduction in cholesterol
concentrations in the Study 1 patients was 15.2
mg /- 2.0 mg. The corresponding values in the
Study 2 patients was 17.1 mg /- 2.0 mg Which
mean values (15.2 mg or 17.1 mg) would most
likely have a wider 95 CI associated with it?
71Confidence Intervals (CI)
- CI applies to continuous data, proportions
(nominal data), medians, regression slopes,
relative risk data, response rates, survival
rates, median survival duration, hazard ratios,
non-random selection or assignment between
groups.
72Measures of Variability
Level of Measurement SD SEM
CI Nominal No No Yes Ordinal No No
No Continuous Yes Yes Yes
73Statistical vs. Clinical Significance
- Example A new antihypertensive drug is studied
to determine whether it decreases the rate of
myocardial infarction. The results indicate that
the drug decreases MI by 11 with a 95 CI
-2-25.
74Statistical vs. Clinical Significance
- The 95 CI for the relative risk of headache
development with a new diabetes drug is reported
as 1.20 (CI 0.95-1.50) and for a placebo drug as
1.25 (CI 0.88-1.76). - Are these showing statistical significance?
75Ratios, Proportions and Rates
- Ratio expresses the relationship between two
numbers. Men Women (4590) - Proportion specific type of ratio expressed as a
percentage. 12 experienced cough when using
this drug) (12 of total study population) - Rate form of proportion that includes a specific
time frame. (18 died from influenza in the US
last year)
76Incidence and Prevalence
- Incidence Rate
- Number of new cases of a disease per
time - Total population at risk
- Prevalence Rate
- Number of existing cases of a disease per
time - Total population at risk
77Descriptive StatisticsMeasures of
Risk/Association
- Relative Risk
- Odds Ratio
- Relative Risk Reduction
- Absolute Risk Reduction
- Number Needed to Treat
- Number Needed to Harm
78Measures of Risk
- Relative Risk (RR)
- the risk or incidence of an adverse event
occurring or of a disease developing during
treatment in a particular group. - RR pts in treatment group w/ ADR
- Total of pts in treatment group
- pts in placebo group w/ ADR
- Total pts in placebo group
79Relative Risk Example
- A new drug is being compared to placebo to
prevent development of diabetic retinopathy (DR). - Treatment DR No DR Total
- New drug 50 75 125
- Placebo 65 55 120
- What is the risk of DR developing during
treatment in patients taking the new drug? - 50__ 0.4 40 Risk in placebo? 65
0.5454 - 125 120
- RR 0.4/0.54 0.74 or 74
80Relative Risk (RR)
- RR 1 When the risk in each group is the same
- RRlt1 When the risk in treatment group is smaller
than the risk in the placebo group - RRgt1 When the risk in the treatment group is
greater than the risk in the placebo group
81Relative Risk (RR)
- Example The risk of an adverse event developing
during therapy with an eye medication compared to
the placebo group was listed as 1.5. What does
this mean? - Answer That the eye med is 1.5 times more likely
to cause an adverse event than the placebo being
used.
82Relative Risk Example
- 92 men and women who were recovering from heart
attacks were followed and surveyed a year later.
14 of the 92 patients had died. When death rates
were calculated according to pet ownership, only
3 of the 53 pet owners (5.6) were no longer
living, compared to 11 of 39 (28) patients who
were without animals. - Relative risk 0.056/0.28 0.2
- What does this mean?
83Relative Risk...
- Relative Risk does NOT tell us the magnitude of
the absolute risk. - Example A RR of 33 could mean that the
treatment reduces the risk of an adverse event
from 3 down to 1 or from 60 down to 20. These
may or may not be significant depending on the
population and adverse event (minor or major
adversity)
84Odds Ratio (OR)
- Commonly reported measure in case control
designs. Case control starts with outcomes.
(looks back for risk factors) - OR pts taking drug w/ ADR
- pts taking drug w/o ADR__
- pts not taking drug w ADR
- pts not taking drug w/o ADR
85Odds Ratio cont.
- The Odds Ratio could also be expressed as
- Treatment A Deaths
- Treatment A Survival_____
- Treatment B Deaths
- Treatment B Survival
86Odds Ratio (OR)
Odds of developing a disease or ADR if exposed
(to drug) Odds of developing a disease or ADR if
not exposed (to drug)
OR Disease Present
Absent Exposed factor A
B Not exposed to factor C
D OR A/C A X D OR A/B A
X D B/D B X C
C/D B X C
87Odds Ratio Example
A case control study reported that 35 of 120
chronic renal failure patients took NSAIDs
compared to only 20 of 110 similar patients
without renal failure. What would be the odds
ratio of developing renal failure if taking
NSAIDs? 35 (taking NSAIDs w/ RF) A35
B 20 20 ( taking NSAIDs w/o RF) C 85
D 90 85 (not taking NSAIDs w/ RF) 90 (not
taking NSAIDs w/o RF)
88Renal Failure/NSAIDs
A/C A/B B/D C/D 35/ 85 0.41 or
35/20 1.75 20/ 90 0.22
85/90 0.94 0.41 1.86 1.75
1.86 0.22
0.94
89Odds Ratio
- OR l The odds of developing an adverse event
or disease in the exposed (treatment) group is
the same as the odds in the non-exposed
(non-treatment) group. - ORlt1 Odds of developing ADR in exposed group is
less than odds in non-exposed. - ORgt1 Odds of ADR in exposed group greater than
the odds in non-exposed.
90Odds Ratio (OR)
- Example The odds that ASA was taken by children
who developed Reyes Syndrome vs. the odds that
ASA was taken by similar children who did not
develop Reyes Syndrome was reported as OR3l. - The odds that Reyes Syndrome children had taken
ASA was approximately 3 times greater than for
the children who did not develop Reyes Syndrome.
91Interpreting the OR and RR
- 1. Degree of validity of the study design.
- 2. The confidence interval (CI)
- 3. Relative Risk Reduction (RRR)
92Relative Risk Reduction (RRR)
- Ex If a new drug is shown to reduce the risk of
cancer, what is the exact percentage of this
reduction? - RRR measure of the reduction in the relative
risk in the exposed group. - RRR Rate in control group-rate in tx group
- Rate in control group
- RRR 1-RR
93Relative Risk Reduction (RRR)
- Incidence of cancer was 7 in treatment group and
12 in placebo( control) group. - RRR 12-7 0.42 42
- 12
- Disadvantage of RRR- doesnt discriminate between
very large and very small actual incidence rates
in the groups.
94Relative Risk Reduction Example
- A study is performed to determine the efficacy of
a new LMWH, Drug H in preventing PE from post
surgical patients. 299 post surgical patients are
randomized to receive Drug H and 355 receive
placebo. 43 patients developed PE in the placebo
group, and 21 developed PE in the treatment
group. What is the relative risk reduction by
Drug H (reducing the risk of PE)
95Drug H Example cont...
Incidence of PE in placebo group 43/355 0.12
12 Incidence of PE in Drug H group 21/299
0.07 7 RRR Rate in control group - rate in
treatment group Rate
in control group RRR 12-7 / 12 0.12- 0.07/
0.12 0.42 42 OR another way to calculate is
RRR 1-RR 1- 21/299 / 43/355 1- 7/12
12/12-7/12 5/12 0.42 42 or 1- 0.07/0.12
1-0.58 0.42
96Absolute Risk Reduction (ARR)
- ARR Incidence rate in control group - incidence
rate in treatment group. - Ex cancer treatment 7, placebo 12
- ARR 12-7 5
- For serious conditions though, a small ARR can
still be very clinically relevant.
97Number Needed to Treat (NNT)
- NNT number of individuals that need to be
treated in order to prevent one adverse event or
one outcome. NNT 1 - ARR
- Ex study determine efficacy of drug preventing
cancer. Incidence of cancer in placebo 12, in
treatment group 7 - 12-7 5 1/5 20NNT (20 pts needed to
treat to prevent 1 case of cancer - NNT 1/ placebo - treatment group
98Number Needed to Harm (NNH)
- NNH 1/ treatment- placebo group
- Ex Headache occurred in 25 of placebo patients
and 75 of patients taking drug X. - The NNH 75-25 50 1/0.5 2
- Only 2 patients would need to be treated with
drug X in order to cause a headache occurrence.
99Review
- In a diabetes study, 4 of Glucotrol users and
18 of placebo pts. Developed CHF within 10
years. - RRR 18-4 14 0.77 77 RR
- 18 18
- ARR 18-4 14
- NNT 1/0.14 7 pts
- In Glucotrol group 26 had HA vs. 3 in placebo.
- NNH 26-3 23 1/0.23 4