Title: The Basics of Study Design
1The Basics of Study Design
Barry Braun, PhD, FACSM Associate
Professor Director, Energy Metabolism
Laboratory Department of Kinesiology University
of Massachusetts Amherst, MA
2A fairy tale
- While boardsailing in Belize,
physician/ - scientist Dr. Dulcinea Toboso gets hit on the
head by - her mast and knocked unconscious. She wakes up in
- a hut where she is cared for by a tribe of people
who - share a remarkable characteristic every person
is - lean and toned, even though they eat massive
meals - and do absolutely no exercise. They tell her the
- secret is the bark of a rare tree that only grows
in the - misty cloud forests that hide the interior of the
island. - The bark smells like elephant feces and somehow,
- tastes even worse.
3- Though it is strictly
forbidden, Dr. Toboso leaves
with several kilograms of bark hidden
in her bathing suit. She - flies to San Francisco and heads to her
laboratory - to isolate the active ingredient, which she plans
to - market as "Bark-a-lounge", a dietary supplement
- designed to cause fat loss and muscle growth
- without any need for exercise. As a conscientious
- scientist, she decides to do a research study to
- show how well it works. She writes the study
- design on her prescription pad and orders her
- long-suffering assistant to do the following
study
4- A group of 12 men she knows from her gym will
- participate in the study. They will weigh
themselves - at home and then come to the laboratory so their
- body fat can be measured using skin fold
calipers. - Then they will do as many pushup and situps as
- they possibly can. They will be given 30 doses of
- "Bark-a-Lounge" in pill form and
told to - take 2 per day for about 15 days.
Then, they - will re-weigh themselves, come back to the lab to
- have body fat re-measured and do as many
- pushups and situps as possible. Dr. Toboso is
sure - that the men will lose fat but gain strength
after - taking "Bark-a-Lounge" for 15 days.
5Objective
- Although we have to give Dr. Toboso credit for
- even considering actually subjecting her product
to - scientific testing, many of you recognize that
her - study design is not optimal. The overall goal of
this - lecture is to allow you to recognize the
strengths - and the flaws in published studies and media
- reports. If you plan to conduct your own studies,
this - lecture will aid you in designing them in a way
that - maximizes their contribution to the body of
scientific - knowledge that is used to enhance the performance
- of athletes and the health of the general public.
6Plan of attack
- Part 1 True Lies
- What kind of study? Epidemiology vs. experiment
- cross sectional vs. longitudinal, association and
- causality, validity and reliability
- Part 2 Of Mice and (Wo)Men
- Humans, animals or cells?
Controlling - confounding variables vs. real world application.
7More plan of attack
- Part 3 Sub-divide and conquer
- How do you attack big important questions?
- One big study or many small ones?
- Part 4 The Color of Money
- Can the funding source affect the
study - design? The results?
- Part 5 You cant always get what you want
- All studies have flaws. Why continue to do them?
8Some useful terms
- Subjects participants in a study (usually only
- used when participants are human)
- Variable Something that can be measured.
- Independent variables are controlled by the
- investigator (research scientist). Dependent
- variables are not.
- Treatment What subjects are exposed to. Also
- called exposure or condition.
9- Outcomes The dependent variables. The
- answers to the question you are interested in.
- Control group or condition What the treatment or
- exposure is compared with. Can be the initial
state - (baseline) or can be a group that is either given
no - treatment or a non-functional placebo.
- Relative to starting weight (baseline), what is
- effect on body weight (outcome) when I give 100
- people (subjects) three pints of ice cream per
day - for 6 months (treatment) as compared with 100
- people who get no ice cream (control group)?
10Epidemiological Studies
- One or more characteristics of a
- population (e.g. weight or blood lipids or
- dietary habits) are assessed (usually by using
questionnaires but other techniques used as
well). Subjects are not asked to change behavior
or subjected to treatments like exercise or diet
change. - Researchers do not control the experimental
conditions they are trying to understand
behavior or physiology or metabolism in a
natural setting.
11Cross Sectional
- The variables of interest are measured once.
E.g., survey 600 subjects (300 W and 300 M) and
measure height. Exposure is gender and the
outcome is height. - Mean (average) height for men 175 cm
- Mean height for women 165 cm
- Based on your data, you might conclude that men
are taller than women.
12- Note that EVERY man was not taller than
- EVERY woman. There is a lot of variation in
- human height (lets say men in your sample
- ranged from 155-195 cm and women from
- 148-185 cm).
- But the average or mean height for men (175 cm)
is - greater than the mean height for women (165 cm).
148 165 175 195
13- Because there is so much variation in height
within - each gender (about 30 cm in your sample)
- compared to the mean DIFFERENCE in height
- (only 10 cm), you need to study a lot of subjects
to - see a difference between men and women that
- accurately represents the population.
14- Although very useful to illustrate a relationship
- between exposures and outcomes, a problem with
- observational studies is that you often cant
- determine if the exposure caused the outcome.
- Lets say you are interested in whether doing a
lot of - aerobic exercise lowers the risk for getting
cancer - in particular, skin cancer. You send out surveys
to - hundreds of people asking about their exercise
- habits and whether they had skin cancer. This is
a - case-control study it compares people who got a
- disease (cases) with those who didnt
(controls).
15Retrospective studies
- You could do this study retrospectively, that
is, - you could look through medical records, find
cases - of skin cancer, and mail surveys to the people
you - identified asking them about their exercise
habits. - The downside to this approach is that you depend
- on peoples memory of their past habits. You
might - minimize this problem by having people mail you
- their training diaries but many will be
non-existent or - incomplete and you have no way to determine
- whether or not they are accurate.
16Prospective studies
- You can also do this study prospectively. You
start - with a group of individuals who DONT have the
- disease and track them for some period of time.
- Then, you look for differences between people
- who got the disease vs. those who didnt.
- You might randomly contact 5000 people from the
- phone book and assess their exercise habits every
- year. At the end of 5 years you would see who got
- skin cancer and if there was a relationship
between - time spent exercising and a diagnosis of skin
cancer.
17- The advantage of a prospective design is that the
- subjects are followed longitudinally, that is
over - time rather than cross-sectionally which only
- gives a single snapshot at one time point.
- But to get meaningful comparisons you need to
- have a fairly large number of people who get the
- disease so that you can separate them into groups
- that differ by exercise habits. And some of the
- subjects will move away or lose interest over
time. - So to get accurate results often requires
recruiting - and tracking thousands of people for multiple
years.
18Questions and answers
- Lets say that your results show that people who
run - and cycle and swim gt 20 hours/week have higher
- rates of skin cancer than people who dont
exercise - at all. Can you conclude that triathlon training
- causes skin cancer? Alert the media!
- Most triathletes spend an enormous amount
- of time outdoors with a lot of skin exposure
- to the sun. So is it exercise that causes more
skin - cancer or is it more exposure to UV radiation
from - the sun. Unless you collected data on sun
exposure - in your survey, you would have no way to know
19Isolating the outcome of interest
- With enough subjects and enough information
- there are statistical methods to separate the
- key variables. E.g., if you had good data on both
- exercise habits and sun exposure you would see
- that if you remove or factor out the sun
exposure - variable, there is no longer any association
- between exercise habits and skin cancer. So it is
- sun exposure, not exercise, that increases the
risk - for skin cancer.
20- Take another example. Lets say you want to test
the - hypothesis that a high intake of fat increases
the risk - for heart disease. You would need to
- 1. accurately identify the men and women in
- the population who get heart disease
- 2. accurately assess how much fat is in the
- diet of each person
- 3. compare dietary fat in people who get heart
- disease with dietary fat in people who dont
21 of people who get heart disease 0 20 40
60 80
10 30 50 70
dietary fat as a of total kilocalories
- This graph (I made it up) says that the number
- of people who get heart disease increases as
- the amount of fat in the diet increases.
- What are potential problems with this story?
Well, did - we measure what we thought we were measuring?
22Validity
- Validity refers to the accuracy or truthfulness
of a measurement. In other words, are you
actually measuring what you think you are
measuring? - This can be obvious (using a body weight scale to
measure body fat), less obvious (are lower blood
lipids after starting exercise training due to
training or accompanying weight loss?) or very
subtle (do athletes perform better when given
carbohydrate during exercise because the sugar
does something directly or because they think
they should do better when given carbohydrate?)
23Measuring physical activity
- Activity monitors are a good example of how
- difficult it can be to develop tools that yield
valid - measurements of physical activity. There are
- many types of activity monitors available
- pedometers, accelerometers, etc.
- If you are a scientist interested in accurately
- measuring daily physical activity how valid are
- these tools?
24- For example, you decide that collecting physical
- activity information using questionnaires is too
- subjective and prone to bias so you decide to
- measure it objectively using an activity monitor
- that is worn on the hip and is sensitive to
motion. - You give the accelerometers to 20 people and
- measure their activity for 7 days to assess their
- physical activity. 10 of your subjects
- are world class cyclists and 10 are typical
college - students. After 7 days your measurements
indicate - the college students are more active than the
elite - cyclists! How can this be?
25- Since the activity monitor only measures
- movement in the vertical plane, the 600 miles
each - of your cyclists covered during the week on their
- bicycles was not detected as movement by the
- monitor.
- This is an extreme case but researchers
- are constantly forced to consider am I
- really measuring what I need to measure?.
26What do your subjects eat?
- One of the most common measurements
- attempted in Sport Nutrition is diet analysis. It
- seems straightforward you collect information
- from subjects about what they eat over the course
- of a few days and enter the foods into a database
- which spits out grams of carbohydrate and protein
- and thiamine and iron and vitamin C, etc.
- In reality, the measurement is fraught with
- potential inaccuracy.
27Sources of potential error
- How do you account for portion size? Estimate
- based on showing the subjects plastic food models
- before you start the study? Have them weigh their
- food? Better but they have to carry their scales
- everywhere with them. What about combination
- foods? How do they tell you ingredients and
- portion sizes of the seafood paella they had at
- their best friends wedding? And how do you know
- they are remembering to report
- everything they ate?
28- And the process of having to weigh their food and
- write everything down changes their typical
behavior. - People avoid foods that are difficult to record
- accurately and start choosing easy things like
- prepackaged foods that are conveniently labeled.
- Diet records are often inaccurate even in
- the hands of experienced users. Many subjects
- under-report their actual food intake by hundreds
of - kilojoules/day. In contrast , women with eating
- disorders may OVER-report actual food intake.
29Internal Validity
- Chance what is the chance that the outcome you
observe could occur even with NO association
between the exposure and outcome you measure? - Measured statistically and reported as a
p-value showing probability of obtaining the
result by chance. Commonly define p-value lt.05
(5) as statistically significant. This means
there is a 95 chance that the observed effect is
NOT due to chance alone. -
- Is this good enough? Is it too restrictive?
30- What are the consequences of getting it wrong?
- Willing to accept an error rate higher than 5 if
the - consequence is getting the wrong sandwich.
- Not willing to accept error rate greater than
0.1 if - consequence is landing on jagged rocks.
- Every reader will have to use their own judgment
- regarding their comfort level with a given
- probability that the results are due to chance.
Most - journal editors have a comfort level right at 5.
31- Bias a systematic error that misrepresents the
- association between the treatment and outcome.
- Investigators may design the study in a way that
- makes it more likely to get a particular outcome.
- Or, in conducting the study, they may treat the
- subjects in one group differently than in the
other - group (e.g. more encouragement during a maximal
- exercise test with the treatment than the
placebo) - Subjects can bias a study as well. Food intake is
- often not accurately reported e.g. faulty
memory or - wanting to supply the right answer.
32Reliability
- Reliability refers to the reproducibility of a
measurement. Measurement tools (surveys,
activity monitors, etc) are often tested
extensively before being used in studies to
determine if the values they report are
reproducible. Reliability is the main reason
researchers often need to make multiple
measurements over several days .
33Reliability
It is important to be clear on the distinction
between validity and reliability. A measurement
can be reliable but not valid i.e., it measures
incorrectly every time. Investigators require
results to be both reliable and valid.
Reliable but not valid
Neither
Reliable AND Valid
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
34Reliability influences of measurements
- Some measurements, e.g. maximal oxygen
- consumption (VO2max) are very reliable. You can
- measure VO2max on different days, different times
- of day, before or after a snack, and the results
will - almost always be within a few of each other.
- On the other hand, resting metabolic rate varies
- day to day and is very sensitive to time of day,
- food intake, exercise, room temperature, etc.
- Need very controlled conditions and have to
repeat - measurements at least 3 times
35- Back to the made-up graph which indicates that
the - number of people who get heart disease increases
as - the amount of fat in the diet increases.
- What are other potential problems with this
story? - Did account for all the other confounding
variables?
36- A confounding variable is associated with both
the exposure and the outcome and that affects the
association between the exposure and outcome.
more exercise hours per week
more skin cancer
more sun exposure
The relationship between exercise and skin cancer
is confounded by strong relationships between
exercise and sun exposure and between sun
exposure and skin cancer. Trying to minimize
confounding variables is the most difficult and
time-consuming part of study design
37- Can we accurately measure the rate of heart
- disease (probably) and the amount of fat in the
diet - (much more problematic)?
- Do other factors need to be considered?
- gender (true for men AND women?),
- age (maybe elderly people eat more fat)
- ethnicity (directly or indirectly)
- other risky behavior (smoking, lack of
exercise, - less frequent physicals, etc.) in people who
eat more fat in diet?
38- Can you consider all the other factors?
- Clearly not b/c we dont even know what they all
are - (e.g. there is a lot of recent evidence that the
- conditions a fetus encounters in utero can have
an - impact on adult-onset disease).
- Even if you could, does a positive relationship
- between 2 things (as 1 goes up, the other also
- goes up) prove that one causes the other?
39price of gasoline
distance from the Earth to Saturn
- During this time period (2005), there was strong
- association between the distance from Earth to
- Saturn and the price of gasoline. Did gasoline
prices - rise because Earth was getting farther from
Saturn? - The relationship is a coincidence
- Association does not mean causality
40- So, epidemiological studies are difficult to
design - in a way that gives you clear, definitive
answers. - To get a sharper picture of the causal
relationships - between diet and health or performance you can do
- an experimental study.
- Take a group of healthy people, feed them
different - amounts of fat, and see who gets heart disease?
41Experimental Studies
- The key difference from an observational study is
- that the investigator actively manipulates the
- treatment instead of letting things happen by
- chance. Because the experimental conditions are
- controlled, there is a much greater chance that
- the outcomes are directly related to the
treatment. - A disadvantage is that by manipulating the
- conditions, the results may have less direct
- relevance to what happens in the real-world
42Experimental Studies
- In experimental research, study subjects (whether
human or animal) are selected according to
relevant characteristics and then assigned to
either an experimental group or a control group.
The subjects in the experimental group receive
treatment and the control group receives no
treatment or a placebo. If you do this
correctly, you can assume that differences
between the groups at the end of the study were
caused by the treatment.
43Experimental Cross Sectional
- Experimental studies can be cross-sectional
(multiple groups getting a single treatment) or
cross-over (one group getting multiple treatments
including control). In a cross-sectional design,
subjects are randomly assigned to either a
treatment or a control group. They are exposed
to the treatment or control for a period of time
and then the outcome is compared between the two
groups. Lets say you wanted to test whether
consuming only simple sugars for 28 days would
cause more synthesis of muscle glycogen compared
with a normal diet.
44- Your cross-sectional design might look something
- like this
Group 1
Group 2
28 days
Baseline test of muscle glycogen synthesis
Groups randomly assigned
Re-test of muscle glycogen synthesis
45Assigning subjects to groups
- One of the keys to doing this right is to ensure
that - the 2 groups of subjects are as similar as
possible. - To do this, subjects are usually randomly
assigned - to the placebo or control group.
- An alternative is to match subjects in each group
- on some key characteristics (e.g. age, weight,
- training status, aerobic capacity). This helps to
- distribute any characteristics that might
influence - the results across the groups.
46- An example of why randomization is important can
- be seen in the following example
- Researchers want to determine if a high fat diet
- during marathon training can improve performance.
- They do a baseline (before any treatment) test of
- aerobic fitness to all of the potential subjects.
Then - they assign them to different groups 20 to the
high- - fat diet group and 20 to the high-carbohydrate
diet - group. Then they train them using the different
diets - for 12 weeks.
47- At the end of that time, they redo the test of
- aerobic fitness and find that the high-fat group
- has improved considerably more (increased
- VO2max from 45 to 52 ml/kg/min) than the high-
- carbohydrate group (only increased from 68 to 70
- ml/kg/min). They report in all of the media
outlets - that runners can gain twice the training effect
by - using a high-fat diet. Is this reasonable?
48- Notice that the baseline VO2max was considerably
- higher in the high-fat group. Runners were
clearly - not randomly assigned the high-carbohydrate
- group seems to have contained really fit elite
- runners (whose VO2max is already about as high
- as it can be) and the high-fat group look like
mainly - novice runners (who can improve a lot with
training). - If the groups had been randomly assigned, the
- baseline VO2max would have been similar in the 2
- groups. In that case, a larger improvement in the
- high-fat group could be interpreted as due to the
- diet (assuming everything else had been done
right!)
49Blinding
- Randomization is often blinded to limit
- experimental bias (an interest in having a
particular - result). Blinding is used to prevent bias from
- influencing the behavior of both the
investigators - and the subjects. There are two types of
blinding, - single blind and double blind. In a single
blinded - study the investigators know which treatment the
- subjects are getting but the participants do not.
In a - double blinded study, a neutral third party
assigns - the groups and neither the investigators nor the
- participants are aware of the group assignments.
50- A drawback of cross-sectional study design is
that - no matter how well you match the 2 groups on
- important characteristics like age, height,
weight, - fitness, etc., there is no way to do this
perfectly. - Two groups may be similar but they cant be
- identical, meaning inter-individual variability
- (genetic and other differences between people)
will - be a limitation to showing clear differences
- between the treatment and the control groups.
- Wouldnt it be great if you could clone each
- subject and use their clone in the other group?
51Experimental Cross Over
- In a cross over design, subjects serve as their
own controls. Half of the subjects get the
treatment and the other half get placebo. Then
the same subjects undergo the opposite protocol.
½ of group
½ of group
28 days
Baseline test of muscle glycogen synthesis
order of treatment randomly assigned
28 days
Re-test of muscle glycogen synthesis
Final test of muscle glycogen synthesis
1 month washout
52Washout period
- A potential problem with the cross-over design is
- that effects of the first condition (e.g.
treatment) - may have an impact on the response to the second
- treatment (e.g. control). The solution is to put
a - washout period between the 2 conditions to
allow - the effects of the first condition to disappear.
- This washout period may be long (months for
- some interventions like training or lipid-soluble
- anabolic agents). This makes the study very
lengthy - and it can be difficult to keep subjects in the
study.
53External Validity
- Also referred to as generalizability meaning how
applicable are the results to the general
population. To increase the external validity,
investigators can study subjects varying in
gender, race, ethnicity, age, weight, etc. By
doing this, it is more likely that results can be
applied to the general population.
54Overgeneralizing
- Many classic studies in nutrition (for example
the - response to semi-starvation and re-feeding human
- protein requirements) were performed almost
- solely using Caucasian, male, healthy subjects in
- their 20s and 30s.
- Nutritional requirements were generalized from
- those studies to the entire population, despite
few - data on women, children, ethnic/racial minorities
or - people with underlying health problems
55Trade-offs
- All major funding agencies now mandate inclusion
- of women and minorities or require a strong
- justification for not doing that.
- Why not include as many types of subjects as
- possible in order to maximize the external
validity? - Increasing external validity also means
increasing - the number of potential confounding variables. In
- some studies, it is more prudent to use a
specific - population to minimize confounding variables
56Basic research studies
- Experiments under highly controlled conditions
are often necessary to confirm observations or
uncover how a process works (the mechanism of
action). They may be conducted in vitro (e.g.
with cell populations on culture plates) or with
animals. - These studies allow the investigator to isolate
one variable of interest without confounding
variables such as environmental factors, genetic
variation, and differences in dietary or physical
activity patterns.
57- One of the advantages of doing studies using
cells - or animals is that tissues not available in
humans - can be isolated (e.g. whole muscle, liver, heart,
- etc.) and life spans are much shorter. For
example, - if we were to do our study of high fat diets and
- heart disease in mice instead of humans, the
study - would take a couple of years instead of decades.
- And researchers could sacrifice the mice at the
end - of the study and look directly at the effects of
the - diets on their arteries, muscle, liver, etc.
58- Due to differences in physiology and the fact
that - animals are routinely exposed to levels of
- compounds far higher than those humans typically
- encounter, results from studies with animals are
- not directly generalizable to humans.
- In addition, there are moral issues regarding
animal - experimentation that cant be ignored. Some
people - feel strongly that no experimentation on animals
is - ever justified. Some people have no problem at
all - with scientific experimentation on animals.
59- The great majority of individuals, both within
and - outside the scientific community see this as a
- complicated issue. There are benefits to animal
- research (potentially lifesaving cures for human
- disease many dogs were sacrificed
- in the hands of Banting and Best
- before they were able to isolate and
- purify the insulin that has saved
- the lives of millions of people with diabetes).
- And certainly costs (nobody enjoys the idea of
- submitting creatures to experimental procedures
- that often end with their death).
60- And the type of animal is certainly a factor in
- peoples discomfort with animal research few
- people object to research on flies, a few more to
- fish or frogs, many become uncomfortable
- with experiments on mice, rats, and rabbits, and
- even more people feel strongly about research on
- cats, dogs and primates.
- To balance these competing forces, universities
- and other research organizations follow strict
- guidelines to help ensure that research on
animals - is conducted in the most humane possible way
61- Researchers are required to justify why the
- research is essential (disease yes, performance
- no), to use statistical analysis to minimize the
- number of animals they intend to study ,and to
- maximize the comfort and well-being of the
- animals in their care.
- As new experimental and mathematical modeling
- techniques are developed, the justification for
- doing research on animals is expected to diminish
- in the near future.
62Human Experimentation
- The moral issues of experimentation extends to
- humans as well. Before organizations began to
- regulate the conduct of experiments on humans,
- experiments were sometimes done without
- subjects consent and with little regard for their
- health or well-being.
- Human research in most countries
- is regulated to ensure that subjects
- can truly give informed consent to the procedures
- and that potential benefits outweigh risks
63- Potential subjects have to be recruited in ways
that - are not coercive and they must be in a position
to - refuse to participate or to leave the study
partway - though without adverse consequence (so no
- prisoners, people who are institutionalized,
- children unless with parental consent).
- What about students in a class being taught by
the - researcher? Grad students in the lab? The
- researcher has to convince an institutional
review - board that participation or non-participation
will - have absolutely no consequences with respect to
- their grade in the class or graduation, etc.
64- Review boards weigh the potential benefits from
- the research with the stress, physical and mental
- discomfort, time commitment, etc. that the
subjects - are required to undergo.
- During the study itself, procedures must be in
- place to ensure that health and well-being of the
- subjects are a higher priority than the data.
- Subjects are often compensated financially for
- their participation it is important that
compensation - be sufficient but not excessive (i.e. coercive)
65- Because the priority to maximize health and well-
- being of the subjects and to ensure they are not
- coerced into continuing participation in a study
can - conflict with the need to collect vital research
data, - doing human studies in a way that both
prioritizes - subject well-being AND maintains maximal
scientific - rigor is very difficult.
66- So lets return to this made-up association.
Could you - do an experimental study in which you recruit
- subjects without heart disease, feed them several
- different amounts of dietary fat and look at the
- relationship between dietary fat and the rate of
heart - disease over time?
67- Yes. But would require studying hundreds of
- people for decades, providing all of their meals
and - controlling dozens of other things that affect
risk for - heart disease (like smoking and exercise and
- aspirin use and on and on).
- This would cost tens of
- millions of dollars, take
- several decades and
- would still be almost
- impossible to
- do b/c most volunteers
- would leave the study
68- So, how do you design a study that can answer an
- important question and that is doable in a
- reasonable time frame and for a reasonable
- amount of ?
- You have to take a big important question and
- reduce it to a much smaller, more focused
- question. You have to do a series of small
studies, - each one building on the one before, until you
- accumulate enough evidence to support or
- disprove your idea.
69(No Transcript)
70Matching subjects on key characteristics
- To compare whether men responded to cookie
- cream similarly to women, you plan to do a
- 2nd group composed of men. What kind of men?
- Well, you can recruit men matched to the
- characteristics of the women. How about
- VO2max? OK, but a woman with VO2max of 60
- ml/kg/min is often a lean athlete in hard
training - whereas elite male athletes have a higher
- VO2max. So men and women matched on
- VO2max will usually differ on body fat and
training
71- How about body fat? OK, but an average body fat
- for a man, lets say 15 would be very low for a
- woman, and again you are likely to end up with
- trained female athletes with very high VO2max and
- moderately trained, recreationally active men.
- There is actually almost no way to match men and
- women for both aerobic capacity AND body fat.
- Need to choose the one that is MOST critical. Or
- another characteristic that CAN be matched (e.g.
- training status)
72- Recruiting subjects for studies requires
balancing - many competing factors. A more diverse subject
- pool gives you more generalizability but also
more - confounding variables, increasing the number of
- subjects required.
- A more homogeneous group (e.g. highly trained,
- college-age women in the luteal phase) reduces
- the confounding variables and allows you to do
the - study with fewer subjects but also makes the
- results less generalizable
73- The trade-offs illustrate why studies at all
levels of - generalizability are required to answer important
- questions. Epidemiological studies using large,
- heterogeneous sample sizes can point to
- interesting associations that are worth pursuing
- (e.g. more physical activity is associated with
lower - rates of diabetes).
- Basic scientists can look at potential mechanism
- (e.g. isolated rat muscle electrically stimulated
to - contract takes up more sugar than resting muscle)
74- In between are human experimental trials ranging
- from simulating the rat study (putting in
arterial and - venous catheters in the leg of a volunteer to see
if - exercised human muscle also takes up more sugar
- for the blood) to testing different intensities
and - durations of physical activity on groups of free-
- living people to determine which combination has
- the biggest impact to reduce the risk for
diabetes. - The best study designs build on the
- results that have come before and add
- another key piece to the jigsaw puzzle.
75New York Times, 5-27-01
- Coke formed a partnership with
- Procter Gamble earlier this spring.
- The companies are now preparing to introduce a
- drink called Elations. Each bottle of Elations
- contains 1500 milligrams of glucosamine, a
dietary - supplement that has been popular among people
- with arthritis for years.
- Procter officials insist that sound science is
what - distinguishes Elations from the many herbal
- concoctions currently transforming the market.
76New York Times, 5-27-01 contd
- The National Institutes of Health is conducting
a - comprehensive 4-year study on glucosamine. But
- neither Coke nor Procter felt they could afford
to - wait for the results. The game will be over if
- anybody isn't in it by then," said the assistant
- director of Procter's Nutrition Science
Institute. - Can research be done in a way that balances needs
- of the scientific community (a line of research
- studies that tell the whole story) with needs of
- industry (no research or a single study showing
the - product works)?
77For industry, enough research is ....
- sufficient to convince enough consumers
- to buy the product that from sales exceeds the
- costs of manufacture, distribution and
advertising. - To do more violates the interests of employees,
- shareholders and, in terms of price, consumers.
- Doing more than the minimum research needed
- to maximize sales is not only unnecessary but
- even incompatible with interests of the company.
78For academic scientists, enough research is ..
- First efficacy (does it work?)
- safety (does it harm?) but
also - Research scientists are charged with
understanding - context mechanism of action, effects on other
- metabolic pathways etc.
- Doing less than the minimum research required
- to understand the physiological context is
- incompatible with responsibilities as scientists.
79Is there a way to meet halfway?
- YES, in the sense that both groups share the same
- basic goals of optimizing safety, health, and
- performance
- NO, in the sense that there are fundamental
- disagreements about who (target population), what
- (top priorities), why (knowledge/sales), when
(how - soon), and how (single study vs. line of
research) - research is done
80A fairy tale revisited
- A group of 12 men she knows from her gym will
- participate in the study. They will weigh
themselves - at home and then come to the laboratory so their
- body fat can be measured using skin fold
calipers. - Then they will do as many pushup and situps as
- they possibly can. They will be given 30 doses of
- "Bark-a-Lounge" in pill form and told to take 2
every - day for about 15 days. After 15 days they will
re- - weigh themselves and come back to the laboratory
- to have body fat re-measured and do as many
- pushups and situps as possible. Dr. Toboso
- is sure that the men will lose fat but gain
strength - after taking "Bark-a-Lounge" for 15 days.
81Doing studies correctly is hard. Why keep doing
them?
- Its supposed to be hard. Thats what
- makes it great. If it was easy, anybody
- could do it.