VALIDITY

1
VALIDITY
2
Validity is the most important concept in
assessment

• Think of VALIDITY as accuracy

3
VALIDITY refers to the degree to which a test
measures what it purports to measure
4
A test is valid to the extent that it measures
what it is supposed to measure.
5
Teachers use assessments to make decisions about
students and those decisions can only be accurate
if the test results themselves are accurate.
6
The test results can only be accurate if the
questions are representative of the entire
subject area.
7
The principle premise of assessment is that you
can tell how much a person knows about an area by
asking a person to show a representative example
of his knowledge of the field.
8
Thus
The system works like this
The student studies and learns every aspect about
a subject.
The teacher develops a test to find out what the
student has learned.
The teacher forms an impression
The student does well
9
Based on the students good score on the exam
The teacher comes to the conclusion that the
student has leaned all the material.and also
that this student is
SMART!
SMART!
SMART!
10
Validity centers on the accuracy of the
inferences that teachers make about their
students.
11
A representative example would be some of every
different type of content within the field.
12
In theory, you would need only one example of
each different bit of information in the field in
order to say you have sampled the field
adequately.
13
Basic Math Knowledge

• All math operations

A good sample includes a little of each of the
skills
Addition
Subtraction
Multiplication
Division
14
Poor sampling results in anon-representative
sample which doesnt look like the original that
it came from
Too Much Addition
Too Much Division
Too Much Multiplication
Too Much Subtraction
15
A good test has some questions about every area
that was studied although areas that are more
important should have more questions.
16
The larger the field of study, the more test
questions it takes to get a true idea of all the
student knows.
17
ThereforeThe more questions that you have on a
test, the more accurate or valid the test becomes
18
Because you are increasing the likelihood that
you are getting a representative sample when you
increase the number of questions.
19
There are three types of validity
20
The 3 types of Validity are

• Content
• Criterion
• Construct

21
Content Validity
22
Content Validity describes the knowledge or skill
that we want the students to master.

• It was formerly called INSTRUCTIONAL OBJECTIVE

23
Content Validity

• How well the content of the test represents the
  content area of the subject being studied.
• This is more than just factual knowledge although
  facts are important. It includes reasoning and
  attitudes.
• This is the most important form of validity.

24
2 Ways to determine Content Validity

• The teacher reviews the subject material and then
  makes sure that each area or concept is
  represented on the test
• A knowledgeable third party reviews the test and
  offers an opinion as to how the test can be
  improved to make it more representative of what
  the students should know.

25
This third party can be

• Another teacher in the school who teaches the
  same subject
• The state or national standard for the grade
  level and subject
• Someone who is especially knowledgeable about the
  subject or a panel of specialists who work in the
  field

26
Purpose

• The purpose of Content Validity is to make sure
  that an assessment instrument gives a
  comprehensive and accurate measurement of the
  content of the original field under study.
• It shows mastery of the field which was studied.
• It allows for comparison between different
  students, schools or methods of instruction.

27
For Example

• Those students who perform well on tests are
  passed to the next grade while those students who
  do not do well are held back.
• College admissions take students grades into
  account even though the students come from
  different schools and backgrounds and have been
  exposed to different levels of instruction.

28
Criterion Validity
29
Purpose

• The purpose of criterion-related evidence of
  validity is to predict how well students will
  perform on a subsequent criterion.

30
For Example

• A scholastic aptitude test is used to predict
  future grades and success in school.
• The test is taken when the students are in high
  school and are supposed to predict the students
  college grades.
• Scholarships and honors are frequently awarded
  based on the results of these tests (i.e. the
  National Merit Scholarship).

31
Criterion Related Validity is not just related to
tests.

• It can be used in any circumstance where there is
  a need to predict future events such as success
  in basketball based on the students height at a
  certain age or the likelihood that a student will
  want to go into medicine based on having certain
  personality traits.

32
Methodology

• The standard method for computing criterion
  related validity is to give an assessment based
  on the first criterion and then measure the
  second criterion and calculate the correlation
  between the two.
• If the correlation is high between the two
  variables, we infer that the first variable is a
  good predictor of the second variable.

33
Example

• A test is constructed which will supposedly
  predict future income in the 5 years after
  graduating from college.
• The test is given to 1000 college seniors.
• Five years later, the former seniors are
  contacted and their income is recorded and
  correlated with their result on the initial test.

34

• Those questions which correlated highly with the
  students income are retained in the test.
• The students scores on the modified test (minus
  the poor questions which were eliminated from the
  test) should then again be correlated with their
  incomes to confirm a high correlation.
• The test is now ready to be given to a new group
  of college seniors in order to predict their
  earnings 5 years into the future.

35
This method of making a test is obviously time
consuming (i.e. waiting 5 years for the 2nd
criterion to occur.

• There is a faster way to make a test but it is
  not necessarily as accurate.

36
The researcher can start with a group of subjects
who have been out of college for 5 years,
administer the predictive test, and correlate the
two scores.The correlation between the two
measures is called concurrent criterion validity
37
Take care that you do not place too much trust in
tests which predict future success.

• Other factors such as study habits, motivation,
  interest, attitude, etc. frequently play such a
  large part in performance that many times test
  scores designed to predict success are rendered
  meaningless.

38
Construct Validity
39
Construct Validity measures intangible concepts
such as anxiety, motivation, attitudes, etc.
40
The more elusive the construct being assessed,
the more numerous and varied validation studies
you will need to make sure your test is
valid.(It is easier to prove that a test
measures addition ability and more difficult to
prove a test measures leadership ability)
41
There are 3 major ways of measuring Validity

• Intervention Studies
• Differential-Population Studies
• Related-Measures Studies

42
Intervention Studies

• One way to measure Construct Validity is to give
  a pretest which purports to measure the concept
  in question.
• Present a program or other intervention.
• Give a post-test to see if there are changes in
  the students scores.
• Attribute the difference in scores to your
  program.

43
Differential-Population Studies

• Based on his knowledge of the construct, the
  researcher hypothesizes that different
  populations will score differently of a test
  measuring this construct.
• For example, it could be hypothesized that a
  group of nurses would have higher scores than a
  group of juvenile offenders on a test to measure
  nurturing.
• The test could be given to both groups and the
  scores compared to see if there are significant
  differences.

44
Related-Measures Studies

• Another name for this is convergent validity
• There are 3 variables
• The Test which is being developed
• The concept which is being tested
• An existing test which has already been shown to
  measure the concept (e.g. a standardized test)

45

• Each student is given the test which is being
  developed and the existing (standardized) test
  which is already accepted as measuring the
  concept that is being studied.
• The results of the two scores are correlated.
• If there is a high correlation between the two
  tests, it can be inferred that because the second
  test is accepted as measuring the concept, the
  new test also measures the concept.

46
Regarding validity of a test the more evidence
you have to support the validity, the better
47
Validity is the keystone of educational
measurement
48
But Beware...
Even if all test items accurately reflect the
subject matter which is being taught
49
The teacher must make sure that the test is being
given to the group for which it was intended!
50
If an otherwise valid test is given to a group of
students other than those for which it was
intended
or
if it is given under the wrong circumstances
51
The inferences drawn from the test results will
not be valid.
52
What is Face Validity?

• Face Validity means that the test appears to be
  valid.

53
What is Consequential Validity?

• Consequential validity refers to whether the uses
  of test results are valid. Is the test being
  used for the purpose for which it was intended?

54
Standards for Educational and Psychological
Testing

• This joint publication of the American
  Educational Research Association, the American
  Psychological Association and the National
  Council on Measurement in Education (published in
  1999) outlines the latest thinking and practices
  in testing and education today.

55
If a test is found to bevalid it is also
reliable.

• The converse is not true
• (i.e. A test can be reliable but not valid.)

56
Review There are 3 types of Validity

• Content-Test samples the entire subject content
• Criterion-Test predicts future achievement
• Construct-Measures vaguely defined concepts
• This is the primary validity for classrooms