A Thesis in Instructional Systems by Judith Keegan Yoho

1
TECHNOLOGY LEADERSHIP, TECHNOLOGY INTEGRATION,
AND STUDENT ACHIEVEMENT A CORRELATION STUDY IN
K-12 PUBLIC SCHOOLS

• A Thesis inInstructional SystemsbyJudith
  Keegan Yoho
• March 22, 2006

2
Navigation

• INTRODUCTION
• Problem
• Purpose
• Significance
• Definitions
• Research Questions
• Hypotheses
• REVIEW OF THE LITERATURE
• Technology and Leadership
• Styles
• Traits
• NETS-A Standards
• Technology and Integration
• Technology Skills
• Technology Use
• Constructivist Pedagogies
• Technology and Student Achievement
• Computer Use and Pedagogy

• METHOD
• Design
• Instruments
• NETS-A Survey
• TAGLIT Survey
• PSSA
• Data Description
• Data Analysis
• RESULTS
• Correlation
• Multiple Regression
• DISCUSSION
• NETS-A and Skills
• Technology Use and PSSA
• Constructivist Pedagogy and PSSA
• Limitations
• Future Research
• Conclusion

3
INTRODUCTION

• ISTE (1) adopted the National Educational
  Technology Standards for Administrators (NETS-A)
  in 2002.
• The NETS-A provide
• Guidelines for administrators to assist in school
  reform, particularly as it relates to technology
  use.
• Agreement on the things P-12 school
  administrators need to know and do to effectively
  support technology integration in schools.
• A national consensus among educational
  stakeholders of what best indicates effective
  school leadership for comprehensive and
  appropriate use of technology in schools.

(1) ISTE (2002)
4
The Problem

• Two major phenomena characterized public
  education in the 1990s (1)
• Intense criticism of schools
• Proliferation of computer technology
• Many stakeholders came to believe that computer
  technology could transform education by making
  teaching and learning more effective and
  efficient, thereby increasing student
  achievement.
• The success or failure of technology in schools
  resided with teachers (2).
• Technology-rich classrooms were established for
  teachers to practice and emerge laden with
  technology expertise for their classrooms.
• The anticipated effect on teachers' instructional
  methods and increased student achievement failed
  to materialize (3).

(1) Brockmeier (2005) (2) Brooks (1997) (3)
Whitehead, Jensen, Boschee (2003).
5
The Problem

• Overlooked in the process of technology
  integration was that attaining the promise
  depended on school leadership.
• When principals are comfortable with technology,
  they foster technology use in their schools (1).
• The principal, more than any other educator, is
  the key to teachers' adoption and use of
  technology (2).
• The people who make decisions about policies and
  finances in schools have little or no training in
  educational technology and few resources to make
  informed decisions (3).

There Is No Strong Link Between School Leadership
And Educational Technology!
(1) The Office of Technology Assessment
(1995) (2) Cooley and Reitz (1997) (3) Thomas
(2001)
6
The Purpose

• This correlation study attempts to understand
  the nature and complexity of technology
  integration factors, and whether there is a
  corresponding correlation among these variables
  and student achievement that can possibly be
  explained, in part, by leadership.
• The present study examines the relationship
  among three main constructs
• K-12 leaders perceived adherence to the National
  Educational Technology Standards for
  Administrators (NETS-A)
• K-12 teachers technology integration as defined
  by their technology skills, use, and
  constructivist pedagogies (TAGLIT)
• K-12 student achievement on reading and math
  Pennsylvania System of School Assessment (PSSA)

7
The Significance

• The essential point in this study is that school
  leadership, as it concerns technology, is
  currently under profound renegotiation.
• There is no best leadership style for all
  situations (1). The context and situation
  specificity influence and so determine the
  leadership approach adopted.
• This study provides insight into the ways that
  the NETS-A can facilitate leadership for
  effective school reform and technology
  integration for improved student achievement.

(1) Manasse (1984)
8
Definitions

• Campus-Level Leaders Principals and Assistant
  Principals (1)
• NETS-A The National Educational Technology
  Standards for Administrators identify knowledge
  and skills every P-12 administrator needs to know
  about and be able to do with technology
  regardless of specific job roles.
• TAGLIT Taking a Good Look at Instructional
  Technology (2). The TAGLIT survey is used by
  schools nationally (over a million users thus
  far) as part of the work on technology leadership
  by Bill and Melinda Gates Foundation.
• Technology Integration Factors From the TAGLIT
  survey, technology integration factors include
  the average technology skills, use, and
  constructivist pedagogies.
• PSSA The Pennsylvania System of School
  Assessment (3) is a standards-based,
  criterion-referenced assessment in reading and
  math at grades 5, 8, and 11 used to measure
  student's attainment of academic standards.
  Schools are currently expected to score 45
  proficient and above in reading and 35
  proficient and above in math.

(1) Brooks-Young (2002) (2) Cory (2001) (3)
PDE (2001)
9
Research Questions 1-4 10(Technology
Leadership and Technology Integration)
RQ1. Is there a significant relationship between
the campus-level leaders perceived adherence to
NETS-A technology leadership standards and
teachers average technology integration
factors? RQ2. Is there a significant
relationship between the campus-level leaders
perceived adherence to NETS-A technology
leadership standards and teachers technology
skills? RQ3. Is there a significant
relationship between the campus-level leaders
perceived adherence to NETS-A technology
leadership standards and teachers technology
use? RQ4. Is there a significant relationship
between the campus-level leaders perceived
adherence to NETS-A technology leadership
standards and teachers constructivist
pedagogies? RQ10. To what extent is teachers
technology integration influenced or explained by
NETS-A predictor variables?
10
Hypotheses 1-4 10
Hypothesis 1. There is no significant
relationship between the campus-level leaders
perceived adherence to NETS-A technology
leadership standards and teachers technology
integration factors. Hypothesis 2. There is no
significant relationship between the campus-level
leaders perceived adherence to NETS-A technology
leadership standards and teachers technology
skills. Hypothesis 3. There is no significant
relationship between the campus-level leaders
perceived adherence to NETS-A technology
leadership standards and teachers technology
use. Hypothesis 4. There is no significant
relationship between the campus-level leaders
perceived adherence to NETS-A technology
leadership standards and teachers constructivist
pedagogies. Hypothesis 10. To what extent is
teachers technology integration influenced or
explained by NETS-A predictor variables?
11
Research Questions 5-8 11 (Technology
Integration and Student Achievement)
RQ5. Is there a significant relationship between
teachers technology integration factors and
student achievement on the PSSA in reading and
math? RQ6. Is there a significant relationship
between teachers technology skills and student
achievement on the PSSA in reading and
math? RQ7. Is there a significant relationship
between teachers technology use and student
achievement on the PSSA in reading and
math? RQ8. Is there a significant relationship
between teachers constructivist pedagogies and
student achievement on the PSSA in reading and
math? RQ11. To what extent is student
achievement on the PSSA in reading and math
influenced or explained by teachers technology
integration predictor variables?
12
Hypotheses 5-8 11
Hypothesis 5. There is no significant
relationship between teachers technology
integration factors and student achievement on
the PSSA in reading and math. Hypothesis 6.
There is no significant relationship between
teachers technology skills and student
achievement on the PSSA in reading and
math. Hypothesis 7. There is no significant
relationship between teachers technology use and
student achievement on the PSSA in reading and
math. Hypothesis 8. There is no significant
relationship between teachers constructivist
pedagogies and student achievement on the PSSA in
reading and math. Hypothesis 11. To what extent
is student achievement on the PSSA in reading and
math influenced or explained by teachers
technology integration predictor variables?
13
Research Questions 9 12(Technology Leadership
and Student Achievement)
RQ9. Is there a significant relationship between
the campus-level leaders perceived adherence to
NETS-A technology leadership standards and
student achievement on the PSSA in reading and
math? RQ12. To what extent is student
achievement on the PSSA in reading and math
influenced or explained by NETS-A predictor
variables?
14
Hypotheses 9 12
Hypothesis 9. There is no significant
relationship between campus-level leaders
perceived adherence to NETS-A technology
leadership standards and student achievement on
the PSSA in reading and math. Hypothesis 12.
To what extent is student achievement on the PSSA
in reading and math influenced or explained by
NETS-A predictor variables?
15
Technology Leadership Technology
Integration Student Achievement
TAGLIT Survey
NETS?A Survey
SKILLS (Learning to Use)
PSSA
Basic Tools Multimedia Tools Communication
Tools Research and Problem Solving Tools
TECHNOLOGY LEADERSHIP
STUDENT ACHIEVEMENT
Standard 1. Leadership and Vision Standard 2.
Learning and Teaching Standard 3. Productivity
and Professional Practice Standard 4. Support,
Management, and Operations Standard 5.
Assessment and Evaluation Standard 6. Social,
Legal, Ethical Issues
USE (Enhancing Teaching and Learning by Using)
Reading Math
RQ5-8 11
Basic Tools Multimedia Tools Communication
Tools Research and Problem Solving Tools
RQ 1-4 10
CONSTRUCTIVIST PEDAGOGIES(Technology and the Way
the Classroom Works)
Involve students in cooperative, not competitive,
learning Involve students in activities that
require higher level thinking skills Involve
students in interactions with the world outside
of school Involve students in interdisciplinary
activities Involve students in activities that
they find engaging Find the time to work with
students who need extra help Serve as coach, not
lecturer or whole-group discussion leader Assess
student achievement based on products, progress
and effort
RQ 9 12
16
REVIEW OF THE LITERATURE

• As the critical issue of school computer
  utilization shifts from mere access to the more
  fundamental issue of how to effectively integrate
  technology into the curriculum, there is
  increasing discussion of what role administrators
  should play (1).
• The present study examines how campus level
  leaders perceptions of their technology role,
  based on the National Educational Technology
  Standards for Administrators (NETS-A), relates to
  their teachers technology skills, use, and
  constructivist pedagogy (TAGLIT) and how this
  relates to student achievement on the
  Pennsylvania System of School Assessment (PSSA)
  in reading and math.
• There are three lines of research in the
  literature that are fundamental to this study
• Technology and Leadership
• Technology and Integration
• Technology and Student Achievement

(1) Slowinski (2000)
17
TECHNOLOGY and LEADERSHIP

Anytime we try to INFLUENCE the behavior of
another human being, we are engaging in an act
of leadership (1).

• This study examines
• To what extent is teachers technology
  integration influenced or explained by NETS-A?
• To what extent is student achievement on the PSSA
  in reading and math influenced or explained by
  teachers technology integration?
• To what extent is student achievement on the PSSA
  in reading and math influenced or explained by
  NETS-A?

(1) Blanchard, Zigarmi, Zigarmi (1987 p. 13)
18
Leadership Styles

• Transactional leaders depend on contingent
  reinforcement, either positive or negative (1).
• The transactional leader places an emphasis on
  exchanging one thing for another jobs for votes,
  or subsidies for campaign funds (2).
• The focus is on rewards, the taking of corrective
  actions, and other forms of reinforcement in
  exchange for satisfactorily carrying out the
  assignment (3).
• Transformational leadership is built on four main
  factors idealized influence, inspirational
  motivation, intellectual stimulation, and
  individual consideration (4).
• The transformational leader is a role model who
  inspires and motivates followers through his or
  her demonstration of commitment and through
  effective communication of expectations.
  Inspirational motivation fosters a shared vision
  that promotes team spirit, enthusiasm and
  optimism for the future (5).

(1) Bass (1985) and Burns (1978) (2) Burns
(1978) (3) Bass Avolio (1994) (4) Bass and
Avolio (1994) (5) Bass (1985)
19
Leadership Traits

• Early systematic efforts at understanding
  leadership focused on answering questions about
  the universal traits of great leaders.
• The findings were so inconsistent and the list of
  identified traits so long as to serve no
  practical purpose (1).
• New interest in the identification of leadership
  traits emanates from an exploration of the
  relationship between traits and leadership
  effectiveness of administrators (2).
• The emphasis of present trait studies is not
  based on a comparison of leaders vs. non-leaders,
  but on the relation of leader traits to leader
  effectiveness.
• Identifying these traits provides a standard by
  which individuals can measure the degree to which
  they possess these traits.
• It also establishes a framework for the creation
  of leadership profiles in organizations.
• The technology standards for school
  administrators (3) exemplify technology
  leadership traits that have been developed in
  relation to leader effectiveness.

(1) Moorhead Griffin (1998) (2) Hoy Miskel
(1991, p. 254) , Yukl (1994) (3) TSSA (2001)
20
NETS-A Standards

• Leadership and Vision - Educational leaders
  inspire a shared vision for comprehensive
  integration of technology and foster an
  environment and culture conducive to the
  realization of that vision.
• Learning and Teaching - Educational leaders
  ensure that curricular design, instructional
  strategies, and learning environments integrate
  appropriate technologies to maximize learning and
  teaching.
• Productivity and Professional Practice -
  Educational leaders apply technology to enhance
  their professional practice and to increase their
  own productivity and that of others.
• Support, Management, and Operations - Educational
  leaders ensure the integration of technology to
  support productive systems for learning and
  administration.
• Assessment and Evaluation - Educational leaders
  use technology to plan and implement
  comprehensive systems of effective assessment and
  evaluation.
• Social, Legal, and Ethical Issues - Educational
  leaders understand the social, legal, and ethical
  issues related to technology and model
  responsible decision-making related to these
  issues.

21
NETS-A 1Leadership and Vision

• The NETS-A standards in Section 1 on Leadership
  and Vision outline how technology leaders need
  to develop a schoolwide shared vision for
  technology and ensure that the resources,
  coordination, and climate are in place to realize
  it (1).
• Widespread involvement by stakeholders during the
  development of the technology vision and plan to
  foster commitment and ongoing support among all
  stakeholders is also important (2).

(1) ISTE (2002) (2) Costello (1997), Jewell
(1998-1999), Thomas Knezek (1991). Thorman
Anderson (1991)
22
NETS-A 2Learning and Teaching

• Technology leaders are also expected to
  understand how educational technology can be used
  effectively in classrooms and work to see that
  technology supports the needs of students
  learning and teachers instruction (1).
• NETS-A makes this explicit in Section 2 on
  Learning and Teaching where the creation of
  learning environments that support collaboration,
  higher level thinking, and other learner-centered
  methods is emphasized.

(1) Bailey (1997), Bozeman and Spuck (1991), and
Thomas and Knezek (1991)
23
NETS-A 3Productivity and Professional Practice

• The NETS-A standards in Section 3 on
  Productivity and Professional Practice promote
  technology leaders use of technology to increase
  productivity and model for others how to
  effectively use technology (1).
• The literature providing recommendations for
  technology leaders skill sets usually asserts
  that principals should learn how to operate
  technology and use it whenever possible for
  carrying out their own duties, especially to
  communicate with others (2).
• Several authors also state that it is the
  responsibility of the principal to ensure that
  faculty and staff receive learning opportunities
  by providing either release time (3) or
  professional development opportunities (4).

(1) ISTE (2002) (2) Dempsey (1999), Hall (1999),
Jewell (1998-1999), Thomas Knezek (1991),
Thorman Anderson (1991) (3) Kearsley Lynch
(1992) (4) Bailey (1997), Dempsey (1999), Hall
(1999), Thorman Anderson (1991)
24
NETS-A 4Support, Management, and Operations

• The NETS-A Section 4 on Support, Management, and
  Operations ensures that the systems in place
  support technology use in the school and that
  technology also supports the management of such
  systems, including coordinating and allocating
  decisions and spending for equipment, networks,
  software, staff, and support services of all
  types (1).
• Several authors identified providing access to
  equipment for staff as a major responsibility of
  the principal (2) seeking funding to provide
  this equipment and establish this and an ongoing
  budget for technology (3).

(1) ISTE (2002) (2) Bailey (1997), Dempsey
(1999), Hall (1999) (3) Costello (1997),
Kearsley Lynch (1992), Thomas Anderson (1991)
25
NETS-A 5Assessment and Evaluation

• Section 5 of NETS-A on Assessment and
  Evaluation covers various types of monitoring
  functions but emphasizes technology-based
  techniques for evaluation and accountability.
• Where other authors mentioned data collection for
  technology decision making, it was recommended
  that technology leaders work from a needs
  assessment when planning staff development (1) or
  to support a more general goal of seeing what is
  working (2).

(1) Dempsey (1999) (2) Thorman Anderson (1991)
26
NETS-A 6Social, Legal, and Ethical Issues

• Social, legal, and ethical issues associated with
  educational technology are addressed in Section 6
  which states that educational leaders should work
  to ensure equity of access, the safety of users,
  and compliance with social, legal, and ethical
  practices related to technology use.
• Principals should work to ensure equitable access
  and opportunity to technology resources (1).
• School leaders should ensure the teaching of
  students about new ethical dilemmas that might
  arise because of technology use and its
  capabilities (2).
• School leaders should consider legal, ethical,
  and security issues as risks in the management of
  technology (3).

(1) Kearsley and Lynch (1992) (2) Bailey
(1997) (3) Pereus (2001)
27
Leadership and Student Achievement

• Research supports that teachers have the
  greatest affect on student
• achievement. The link between principal
  leadership and student outcomes is indirect (1).
• Some studies found no statistically significant
  relationship between the principals role in
  instructional management and student achievement
  (2).
• Many researchers have concluded that principals,
  being one step removed from the direct
  instructional process (3) had an indirect effect
  on student outcomes (4).

• Hallinger and Heck (1996)
• Leitner (1994)
• Brandt (1987)
• Bossert (1988), Heck, Larsen, Marcoulides
  (1990)

28
TECHNOLOGY and INTEGRATION

• Although there is much variation in the body of
  research, many studies do show a relationship
  between the use of technology in classrooms and
  improved student achievement. For the purpose of
  this research, studies in technology use are
  classified in two ways
• Learning From Computers
• When students are learning from computers (1)
  the computers are essentially tutors. In this
  capacity, the technology primarily serves the
  goal of increasing students basic skills and
  knowledge.
• Learning With Computers
• By contrast, in learning with computers,
  students use technology in constructivist ways to
  develop higher order thinking, creativity,
  research skills, etc., rather than serving simply
  as an instructional delivery system.

(1) Reeves (1998)
29
Technology Integration Definition

• Everybody is talking about technology
  integration, but few practicing teachers profess
  to know exactly how to proceed (1).Teachers are
  often expected to integrate technology without
  having a working definition of the concept.
• Real technology integration requires change (1)
• To adopt new teaching tools such as the computer
  and the Internet.
• To change the way they teach their students,
  which may include changing the role they play in
  the classroom and the way their classrooms are
  physically arranged.
• Technology integration is a system of TECHNOLOGY
  USAGE and TEACHING PRACTICES that, when aligned
  properly, will likely promote increased student
  learning and engagement (1).
• Technology is truly integrated when it is used in
  a seamless manner to support and extend
  curriculum objectives, and to engage students in
  meaningful learning.

(1) Johnson Liu (2000)
30
EffectiveTechnology Integration
Figure 2.1 Piersons model of technology
integration showing the relationship among
content, pedagogical, and technological
knowledge. Section C represents
technological-pedagogical-content knowledge. Used
with permission.

• A teacher who effectively integrates technology
  is one who would be able to draw on extensive
  content knowledge and pedagogical knowledge, in
  combination with technological knowledge.
• The intersection of the three knowledge areas, or
  technological-pedagogical-content knowledge,
  would define effective technology integration (1).

(1) Pierson (2001)
31
Constructivist Pedagogies

• The use of new technologies in an educational
  setting has caused the theory of learning,
  constructivism, to receive new attention (1).
  Students in these settings become empowered by
  gaining access to real data and work on authentic
  problems.
• Comparing teachers who were computer users and
  those who were not, teachers who made the
  greatest use of computers uniformly exhibited
  constructivist practices, including
  collaboration, project-based work, and hands-on
  activities with their students, and that
  computers facilitated their teaching in these
  ways (2).
• The role of technology in education is so
  important, that it will force the issue of
  didactic versus constructivist teaching.
• Computers undermine the didactic, lecture
  methodology, and, instead promote the student as
  a self-directed learner. Teachers will no longer
  have a choice but will be compelled to use a
  constructivist approach in a technology-rich
  environment (3).
• Using computers entails active learning, and this
  change in practice will eventually foster a shift
  in society's beliefs toward a more constructivist
  view of education (3).

(1) Mann (1994) (2) Honey and Moeller (1990) (3)
Collins (1991)
32
Technology, Constructivist Pedagogy and New
Learning Environments
Figure 2.3 Characteristics representing
traditional approaches to learning and
corresponding strategies often associated with
new learning environments for P12 students.

• Characteristics that represent traditional
  approaches to learning are contrasted with new
  learning environments representative of
  constructivist pedagogy in Figure 2.3 (1).

(1) ISTE NETS-T (2000)
33
Technology and Pedagogical Practice
(Constructivism)
Figure 2.2 Piersons model of technology
integration (modified). Note. From Pierson
(1999). Diagram modifications have been made to
include student construction of knowledge. Model
serves to explain interwoven patterns of
knowledge which emphasize constructive teaching
strategies. Used with permission.

• "The key to success lies in finding the
  appropriate points for integrating technology
  into a new pedagogical practice, so that it
  supports the deeper, more reflective
  self-directed activity children must use if they
  are to be competent adults in the future" (1)

(1) Strommen and Lincoln (1992, p. 473)
34
METHOD

• This section describes the methodology used to
  guide the research.
• Included are a description and explanation of how
  the population sample were selected, a
  description of the instruments used, an
  explanation of the development of the survey
  instrument, a review of the procedures used for
  the data collection, and a discussion of how the
  data were analyzed.

35
Research Design

• The NETS-A survey was developed to collect
  information from campus-level leaders about P-12
  technology leadership.
• Data from the NETS-A survey was correlated with
  TAGLIT (1) secondary survey data about technology
  integration collected from teachers at the
  corresponding P-12 schools.
• Similarly, TAGLIT survey data was correlated with
  student achievement in reading and math on the
  PSSA.
• Multiple regression analysis identified the
  extent to which technology integration and
  student achievement were influenced or explained
  by technology leadership and technology
  integration.

(1) Cory (2001)
36
NETS-A Survey Development

• Survey items were reviewed by a group of
  knowledgeable scholars including two Pennsylvania
  State University Professors, one elementary
  principal, one high school principal, and nine
  university students in a quantitative research
  class.
• The 36 items were then administered to 25
  principals in a course at PSU. Their task was to
  align each item with one of the six NETS-A
  standards to demonstrate that the content of the
  item matched the appropriate standard.
• Results of this process averaged 43 alignment
  with the standards.
• Several items were revised and the same process
  was repeated with another group of principals.
  Alignment of items with appropriate NETS-A
  standards increased to 70 (see Survey Items Page
  1 and Page 2).

37
NETS-A Survey Development

• The final survey was piloted with 24 principals
  at the Principals Technology Leadership Academy
  opening workshop in Philadelphia on 11/5/02.
  Results were coded and entered into SPSS (see
  codebook).
• As presented in Table 3.1, the internal
  consistency estimate of reliability (Cronbachs
  Coefficient Alpha, 1985) for the pilot survey was
  high (.9709), therefore it was administered at
  future kick-off events in a similar manner.
  Cronbachs Coefficient Alpha for the total NETS-A
  public school surveys (n 233) was .9465, and
  for the schools participating in the TAGLIT
  survey (n 30) it was .9378.

38
NETS-A Survey Instructions
IRB Flyer for Login/Password
39
Final NETS-A Survey Page 1
40
Final NETS-A Survey Page 2
41
NETS-A Survey Administration
Total Campus-Level Leaders 233
42
NETS-A Descriptive StatsCampus-Level Leaders
Perceived Adherence to NETS-A

• For the population n 30, campus-level leaders
  see themselves most closely aligned with NETS-A
  standard 6 Social, Legal, and Ethical Issues
  (Mean 3.6, between Fairly Well and Well), and
  least closely aligned with NETS-A standard 5
  Assessment and Evaluation (Mean 2.8, between
  Need Improvement and Fairly Well).
• Means for the population n 203 follow a similar
  pattern (see Figure 3.2).

43
Comparison of Average NETS-A Means

• T-tests for the two data sets (n 30 and n
  203) showed no statistically significant
  difference between the average NETS-A standards
  responses for five of the six standards.
• Analyses of these five revealed effect sizes that
  met the minimum value for a small effect size (d
  .20) according to Cohens (1988) standards for
  interpreting effect sizes. This indicates a
  non-overlap of 14.7 in the two distributions.
• For NETS-A standard 3 (S3APPP), Cohens d 0.4
  which is between a small and medium effect size.
  An effect size of 0.4 indicates a non-overlap of
  27.4 in the two distributions.

• It was concluded that no differences existed in
  average NETS-A standard responses and the data
  were representative of the entire population of
  NETS-a surveys collected (n 233).

44
TAGLIT Survey Development

• TAGLIT was originally designed for use by North
  Carolina educators participating in the
  Principals Executive Program and has been
  web-enabled with support from Bill and Melinda
  Gates Foundation.
• With 66 items, it was designed specifically to
  give principals a current-status report of
  technology for teaching and learning at their
  school.
• The TAGLIT lead developer, Dr. Sheila Cory, said
  there has not been a formal reliability/validity
  study done on the survey, however there have been
  over 1,000,000 users.
• The Bill and Melinda Gates Foundation required
  the use of this survey for schools in the PTLA
  Program.
• Responses on 35 Items regarding Teacher
  Technology Skills, Use, and Constructivist
  Pedagogies were used in this study (see Skills,
  Use, Constructivist Pedagogy) and coded (skills,
  use, pedagogy) for data entry.

45
TAGLIT Survey Descriptives Teachers Technology
Skills

• Means for average teachers technology skills
  were all mid-way between 2 (I can do this but
  sometimes I need help) and 3 (I can do this by
  myself). The lowest mean was AMTS Average
  Multimedia Tools Skills, and the highest mean
  was ARPTS Average Research and Problem-Solving
  Tools Skills.

46
TAGLIT Survey Descriptives Teachers Technology
Use

• Means for average teachers technology use ranged
  from a low of 1.85 for AMTS - Average Multimedia
  Tools Use - which is between 1 (I do not use it
  in teaching and learning) and 2 (I am beginning
  to understand its relevance in teaching and
  learning and to experiment using it with
  students) to a high for ARPTS - Average Research
  and Problem-Solving Tools Use - of 3 (I make a
  conscious effort to include it in teaching and
  learning and to integrate it effectively into my
  lessons

47
TAGLIT Survey Descriptives Teachers
Constructivist Pedagogies

• Teachers responded to the question As a result
  of your use of technology in teaching and
  learning, are you more inclined to...
• Means for average constructivist pedagogies are
  all between 2 (Yes, Somewhat) and 3 (Yes, Quite a
  Bit).

48
PSSA Development

• The Pennsylvania Systems of School Assessment
  (PSSA) in reading and math were developed as part
  of the Chapter 4 Regulations in 1996.

49
PSSA Descriptives
50
Data Analysis Continuous Variables

• The survey data and student achievement scores
  collected in this research study were treated as
  continuous variables.
• There continues to be ongoing debate regarding
  the treatment of ordinal data from a Likert scale
  as interval data. Some argue that Rasch models
  (Rasch, 1961) should be used to make that
  conversion.
• In a February 16, 2006 (338 pm) e-mail posting
  to Evaltalk_at_Bama.UA.EDU (the discussion list
  serve for the American Evaluation Association),
  Dennis Roberts, Professor Emeritus at Penn State
  University noted "For me, if one sums together
  scores on a set of items ... let's say a 30 item
  attitude scale where Likert response categories
  are used, the issue of whether the data are
  ordinal or interval is not worth the time to
  figure it out. Do your regular analysis and be
  done with it.
• This investigator examined the data from two
  perspectives - treated as interval or treated as
  ordinal. The mean values when treating the data
  as interval were literally the same as the
  medians when treating the data as ordinal (see
  Comparison). Thus the investigator for this study
  treated the data as interval.

51
Data AnalysisPart 1 - Correlation

• Assumptions for Pearsons r include (1)
• There has to be a relationship between the
  variable and the dependent variable. This
  relationship must be linear.
• There should not be a relationship between any of
  the independent variables (multicollinearity)
• Successive observations of the dependent variable
  must not be correlated (autocorrelation)
• The variance of Y is the same for any fixed
  combination of independent variables
  (homoscedasticity)
• The dependent variable must be continuous and at
  least interval-scale.

(1) Mason, R. D., Lind, D. A., Marchal, W. G.
(1988)
52
Data AnalysisPart 1 - Interpretation
(1) Levin Fox (2000)
53
Data AnalysisPart 1 - Correlation

• An important point to consider in interpreting
  the magnitude of correlation coefficients in
  educational research is that many factors
  influence the behavior patterns and personal
  characteristics of primary interest to educators
  (1).
• Therefore, the influence of any one factor is not
  likely to be large.
• Correlations in the range of .20 to .40 might be
  all that one should expect to find for many of
  the relationships between variables studied by
  educational researchers.
• Correlation analysis indicates the strength and
  direction of the relationship between two
  variables, while regression analysis identifies
  the form of that relationship (2).

(1) Gall and Borg (1996) (2) Cadwallader (1985
43)
54
Data AnalysisPart 2 - Multiple Regression

• Multiple correlation and regression is used in
  those situations where one is investigating the
  interrelationship between more than two variables
  (1) .
• For the purpose of this study multiple regression
  will be the most relevant method as there are
  various factors or variables that play a role in
  technology integration and in student
  achievement.
• Backward elimination is the method that starts
  with all variables in the equation then
  eliminates independents one at a time, based on
  removal criteria, until such an elimination makes
  a significant difference in R-squared. If
  suppression is suspected, backward elimination
  should be chosen as the stepwise option.
• In some cases, it's very important to include all
  variables, even if they are not individually all
  that important.

(1) Cadwallader (1985 55)
55
Data Analysis Backward Elimination Procedure

• For this research study, a variable was removed
  if it no longer met the criterion of .05 to
  enter, .10 to remove.
• These values can be changed, but it always must
  be harder to get in than to get out.
• The variable selection process terminates when
  all variables in the model meet the criterion to
  stay (see Figure 3.3).
• Often the backward method results in an equation
  with the strongest explanatory power (1).

(1) Mason (1988)
56
Checking Regression Assumptions
See Example

• Normal Probability Plot
• In the Normal Probability Plot, you are hoping
  that your points will lie in a reasonably
  straight diagonal line from bottom left to top
  right. This would suggest no major deviations
  from normality.
• Scatterplot
• In the Scatterplot of the standardized residuals
  you are hoping that the residuals will be roughly
  rectangularly distributed, with most of the
  scores concentrated in the center (along the 0
  point). What you dont want to see is a clear or
  systematic pattern to your residuals (e.g.,
  curvilinear, or higher on one side than the
  other).
• Residuals can also be detected from the
  Scatterplot. Outliers are cases that have a
  standardized residual of gt 3.3 or lt -3.3 (1)
• The plot should show a random pattern, with no
  nonlinearity or heteroscedasticity. In jargon,
  this will show the error vector is orthogonal to
  the estimate vector. Non-linearity is, of course,
  shown when points form a curve. Non-normality is
  shown when points are not equally above and below
  the Y axis 0 line. Non-homoscedasticity is shown
  when points form a funnel or other shape showing
  variance differs as one moves along the Y axis.
• Equal variance with mean zero

(1) Tabachnick and Fidell (1996)
57
RESULTS

• Findings for Correlations RQ 1-4 and Regression
  RQ 10
• (slides 58 62)
• Findings for Correlations RQ 5-8 and Regression
  RQ 11
• (slides 64 71)
• Findings for Correlations RQ 9 and Regression 12
• (slides 73 74)

58
NETS-A and Teachers Technology Skills
Correlation RQ2
59
NETS-A and Teachers Multimedia Skills
Correlation RQ2
60
NETS-A and Teachers Technology Use Correlation
RQ3

• No statistically significant correlations

61
NETS-A and Teachers Constructivist Pedagogies
Correlation RQ4

• No statistically significant correlations

62
NETS-A and Teachers Average Multimedia Skills
Regression RQ10
63
Summary RQ 1-4 10 Correlations and Regressions
r .516r .644r .454r .533r
.391r .393
Y .856(S2ALT)
64
Teachers Technology Skills and PSSA
Correlation RQ6

• No statistically significant correlations

65
Teachers Technology Use and PSSA Correlation
RQ7
66
Teachers Constructivist Pedagogies and PSSA
Correlation RQ8
67
Technology Skills and PSSA Reading and Math
Regression RQ11
PAREAD
PAMATH

• No statistically significant regression models

68
Technology Use and PSSA Reading Regression RQ11
69
Technology Use and PSSA Math Regression RQ11
70
Constructivist Pedagogies and Reading PSSA
Regression RQ11
71
Constructivist Pedagogies and PSSA Math
Regression RQ11
72
Summary RQ 5-8 11 Correlations and Regressions
Y -.788(ACTU) .524(ARPTU) Y -.490(ACTU)
.649(ARPTU)
r -.447 (PAREAD)
Y .970(CP29HLT) -.524(CPTEH)
-.726(CP35PPE) Y .747(CP29HLT) .710(CP33EA)
-.743(CPTEH)
r -.466 (PAREAD)
73
NETS-A and PSSA Reading and Math Correlations
RQ9

• No statistically significant correlations

74
NETS-A and PSSA Reading and Math Regression RQ12
PAREAD
PAMATH

• No statistically significant regression models

75
Summary RQ 9 12 Correlation and Regression
76
DISCUSSION

• This study was designed to add to the general
  body of knowledge about the relationship between
  technology leadership aligned with NETS-A and
  technology integration as defined by teachers
  technology skills, use, and constructivist
  pedagogy. Additionally, the association of NETS-A
  and technology integration with student
  achievement in reading and math were investigated.

77
Leadership/Technology Integration/ Student
Achievement Correlation/Regression Results Model
TAGLIT Survey
NETS?A Survey
SKILLS
PSSA

• ABTS
• AMTS
• ACTS
• ARPTS

TECHNOLOGY LEADERSHIP
r .516r .644r .454r .533r
.391r .393
STUDENT ACHIEVEMENT
USE
Y .856(S2ALT)

• S1ALV
• S2ALT
• S3APPP
• S4ASMO
• S5AAE
• S6ASLE

• ABTU
• AMTU
• ACTU
• ARPTU

Y -.788(ACTU) .524(ARPTU) Y -.490(ACTU)
.649(ARPTU)

• PAREAD
• PAMATH

r -.447 (PAREAD)
CONSTRUCTIVIST PEDAGOGIES

• CP28CL
• CP29HLT
• CP30RW
• CP31IA
• CP32EA
• CP33TEH
• CP34CO
• CP35PPE

Y .970(CP29HLT) -.524(CPTEH)
-.726(CP35PPE) Y .747(CP29HLT) .710(CP33EA)
-.743(CPTEH)
r -.466 (PAREAD)
Pearson r Regression
78
DiscussionNETS-A AMTS Correlation

• NETS-A and Teachers Technology Skills
• All six average NETS-A standards correlated
  significantly with teachers average multimedia
  tools skills (AMTS). Four areas make up the
  multimedia tools skills category
• use drawing or painting software to create
  pictures
• use a digital camera and/or scanner to get
  pictures into the computer
• use presentation software to create a
  presentation
• use multimedia software to create a product
• The two multimedia categories that showed
  significant correlations with NETS-A were (a) use
  drawing or painting software to create pictures
  and (b) use a digital camera and/or scanner to
  get pictures into the computer (see Table 4.3).

79
DiscussionNETS-A AMTS

• Some explanation for these findings may come from
  the fact that technology staff development has
  been so focused on the computer, and not on
  technology's role in learning and teaching.
• The President's Commission on Web-Based Learning
  found that teacher training was "usually too
  little, too basic, and too generic to help them
  develop real facility in teaching with
  technology (1).
• 96 reported that the most common training they
  received was on basic computer skills (1).
• With the rise in multimedia computers and
  peripherals, the emphasis on teachers multimedia
  skills, especially learning to use digital
  cameras and scanners, and handle digital images
  is predictable (see QED Chart).
• The lack of correlation between NETS-A and
  teachers multimedia skills with multimedia and
  presentation software may be explained by
  findings that most of the computers in schools do
  not have the capability to run a large variety of
  multimedia software and are also limited in how
  they can access graphical information on the
  Internet (2).

(1) Web-Based Commission (2001) (2) Anderson and
Ronnkvist (1999)
80
Discussion NETS-A and AMTS Regression
ALT has a positive influence on AMTS Y
.856(S2ALT) R2 .486 Adjusted R2 .426

• For the criterion variable teachers average
  multimedia tools skills (AMTS), regression
  results indicated that teachers average
  multimedia skills (AMTS) were specifically
  influenced by one NETS-A standard Average
  Learning and Teaching (S2ALT) which accounted for
  49 of the variance in AMTS.
• NETS-A standard 2 on Learning and Teaching
  states that technology leaders are expected to
  understand how educational technology can be used
  effectively in classrooms and work to see that
  technology supports the needs of students
  learning and teachers instruction (1).The
  creation of learning environments that support
  collaboration, higher-level thinking, and other
  learner-centered methods is emphasized.

(1) Bailey (1997), Bozeman and Spuck (1991), and
Thomas and Knezek (1991)
81
ConclusionNETS-A and Teachers Technology Skills

• With regard to the integration of computers into
  the classroom, the results in this study indicate
  that the NETS-A for campus-level leaders
  correlate with one main aspect of teacher
  technology integration teachers skills with
  multimedia tools and the main influence on these
  skills is NETS-A standard 2 Learning and
  Teaching.
• Importantly, there were no significant
  correlations between NETS-A for campus-level
  leaders and teachers technology use or
  constructivist pedagogies.
• Although research has shown that technology
  training for teachers promotes the use of
  technology as an instructional tool (1)
  administrators have often failed to schedule and
  fund technology training for teachers (2).
• Effective use of technology requires more than
  the basic knowledge of how to operate a computer,
  digital camera or scanner. Because the
  appropriate amount of attention has not been
  given to individual teacher concerns and
  readiness to change, technology has not been
  fully integrated into the fabric of teaching and
  learning.

(1) Atkins, 2000 Casey Rakes (2002), Martin
Lundstrom (1988), Smith (1998) (2) Eastwood et
al. (1998), Fitzgerald, Krueger, Kaczka (1999),
Zehr (1999)
82
DiscussionTechnology Use/Constructivist Pedagogy
and PAREAD - Correlation
ACTU has a negative correlation with PAREAD

• The negative association of teacher technology
  use and student achievement in reading finds some
  explanation in general reviews of observational
  studies tending towards a negative assessment of
  the potential of using computers for
  instructional purposes in classrooms to improve
  students educational achievement (1).

CP33TEH has a negative correlation with PAREAD

• According to a report of the National Education
  Commission on Time and Learning (2), the whole
  question of teachers and time needs to be
  rethought in a serious and systematic way.
  Technology integration induces reallocations,
  substituting alternative, possibly more effective
  forms of instruction. Given a constant overall
  instruction time, this may decrease student
  achievement.

(1) Cuban (1993), Oppenheimer (1997) and
Kirkpatrick and Cuban (1998) (2) NECT (1994)
83
DiscussionTechnology Use and PSSA - Regression
ACTU has a negative influence on PAREAD and PAMATH

• Wenglniski (1) found that using computers to
  teach low order thinking skills was negatively
  related to academic achievement.

ARPTU has a positive influence on PAREAD and
PAMATH

• Wenglniski (1) found that teachers who had
  students use computers to solve simulations saw
  their students math scores increase
  significantly.

Wenglinski (1) found that teachers who had
training and skills used technology in ways that
focused students on simulations and applications
that encouraged students to develop problem
solving skills. Those teachers who hadnt had
training used skill and drill software.
(1) Wenglniski (1998)
84
ConclusionTeachers Technology Use and Student
Achievement

• Teachers use of average communications tools
  correlated negatively with student achievement in
  reading and math.
• Communications Tools (teacher use of e-mail and
  web authoring) is not embedded in curriculum
  application?
• Teachers use of average research and
  problem-solving tools correlated positively with
  student achievement in reading and math.
• Similar results were presented in a report from
  Canadas Learning Policy Directorate, Strategic
  Policy and Planning Branch (2004) on the impact
  of computer use on reading achievement of
  15-year-olds.
• This concurs with Wenglinskys findings (1998) in
  the importance of teachers computer skills where
  he concluded that the proper thinking fostered by
  the computer application along with the teachers
  skill in using that technology will provide
  increased scores in student testing.

85
DiscussionConstructivist Pedagogies and PAREAD -
Regression
Y .970(CP29HLT) -.524(CP33TEH) -.726(CP35PPE)

• Three constructivist pedagogies explain 45 of
  the student achievement in reading.
• HLT has a positive influence on PAREAD
• TEH has a negative influence on PAREAD
• PPE has a negative influence on PAREAD
• The one exerting the most influence on the
  dependent variable (PAREAD) is higher level
  thinking (.970 CP29HLT), which has 1 ¾ times
  more influence than assessment based on
  progress, product, and effort (-.726 CP35PPE),
  and 2 times more influence than time for extra
  help (-.524 CP33TEH).

86
DiscussionConstructivist Pedagogies and PAMATH -
Regression
Y .747(CP29HLT) .710(CP32EA) -.743(CP33TEH)

• Three constructivist pedagogies explain 51 of
  the student achievement in math.
• HLT has a positive influence on PAMATH
• EA has a positive influence on PAMATH
• TEH has a negative influence on PAMATH
• They all exert approximately the same influence
  on the dependent variable (PAMATH), higher level
  thinking (.747 CP29HLT), engaging activities
  (.710 CP32EA), and time for extra help (-.743
  CP33TEH).

87
ConclusionTeachers Constructivist Pedagogies
and Student Achievement

• Research indicates that computer technology can
  help support learning and is especially useful in
  developing the higher-order skills of critical
  thinking, analysis, and scientific inquiry by
  engaging students in authentic, complex tasks
  within collaborative learning contexts (1).
• Results from other studies (2) also suggest that
  students can benefit from technology-enhanced
  collaborative learning methods and the
  interactive learning process.
• Four fundamental characteristics of how
  technology can enhance both what and how children
  learn in the classroom (3)
• (1) active engagement,
• (2) participation in groups,
• (3) frequent interaction and feedback, and
• (4) connections to real-world contexts.

(1) Roschelle, Pea, Hoadley, Gordin Means
(2000), Means (1994) (2) Perez-Prado and
Thirunarayanan (2002), Smith, Ferguson and Caris
(2001) (3) Roschelle, Pea, Hoadley, Gordin,
Means (2000)
88
Discussion Campus-Level Leaders

• On average, campus-level leaders perceive
  themselves as being only fairly well aligned
  with the NETS-A (see Table 3.4).
• Peoples judgments of how well they will be able
  to perform largely determine what outcomes they
  expect their actions to produce (1).
• The less control and direct impact that
  individuals perceive to have over accelerating
  changes determines their level of anxiety and
  focus on perceived deficiencies.
• In contrast, those individuals with a positive
  sense of efficacy help mobilize effort and
  resources to overcome challenges in the change
  process (1)
• Perceiving themselves only fairly well aligned
  with NETS-A may help explain why the lack of
  correlation between NETS-A and teachers
  technology use and constructivist pedagogies.

(1) Bandura (1998 p. 53)
89
Conclusion Campus-Level Leaders

• Clearly, this study shows that through their
  perceived adherence to NETS-A standard 2
  Learning and Teaching, campus-level leaders can
  influence teachers technology multimedia skills.
  BUT they need to influence teachers technology
  use and constructivist pedagogies in order to
  indirectly influence student achievement.
• Leadership that promotes change is the missing
  factor when it comes to merging technology and
  instruction (1). No matter how much training
  teachers receive to prepare them for technology
  integration, most will not successfully employ
  that training without the leadership of the
  principal. Other studies have supported this
  claim (2).
• Constructivist reforms contain elements that
  focus leaders on the essential intellectual work
  of schools.

(1) Merkley, Bosik, and Oakland (1997) (2) Await
Jolly (1999), Maxwell (1997), National Center
for Education Statistics (2000), Sandholtz,
Ringstaff, Dwyer (1997)
90
Conclusion Campus-Level Leaders

• Leadership that seeks the purposeful integration
  of technology into the culture of schools can
  improve student achievement.
• Principals should identify key NETS-A behaviors
  that are important to them in promoting and
  supporting technology integration in their
  schools, and develop and sharpen these technology
  leadership skills through participation in NETS-A
  aligned professional development.
• Principals should tie their learning about NETS-A
  to their leadership behaviors. Transformational
  leaders style inspire and motivate followers
  through their demonstration of commitment and
  through effective communication of expectations.
  Inspirational motivation fosters a shared vision
  that promotes team spirit, and enthusiasm and
  optimism about the future.
• Vehicles such as project-based learning (1) and
  other forms of pedagogy rooted in constructivism
  (2) may give leaders a way to infuse technology
  into buildings that more easily fits the
  education paradigm.

(1) Blumenfeld et al., (1991) (2) Duffy
Jonassen (1992)
91
ConclusionTechnology Leadership, Technology
Integration, and Student Achievement

• As principals become more adept at guiding
  technology integration, more efficient and
  effective technology use should become prevalent
  in schools.
• The principals' increased knowledge of the
  benefits and uses of technology should lead to
  more support of teachers' attempts to infuse
  technology into the teaching and learning model.
• The principals' improved technology skills should
  lead to increased use of technology tools,
  thereby producing principals who are models of
  technology use.
• So, who will be the catalyst to initiate this
  shift? An educational culture supportive of
  change is necessary if a general transformation
  in instructional methods is to occur (1). Like it
  or not, principals are the protagonists when it
  comes to creating a positive school culture (2).
  Simply put, a principal is responsible for
  fostering a supportive climate. Without
  administrative support, integrating technology
  into the classroom is dead before it even begins.

(1) Becker and Ravitz (1999) (2) Peterson Deal
(1998)
92
Limitations

• The study included not just principals but also
  assistant principals.
• The survey results are based on self-reported
  data.
• Generalizing findings from technology research is
  difficult because it is a rapidly moving target
  due to changes in technology and educational
  vision.
• The small sample size (30 schools) in this
  research makes generalizing to the population
  less reliable.
• The fact that TAGLIT technology skills and use
  questions were very general and did not focus on
  content-specific software applications means this
  data element was not reported.
• The use of standardized tests might miss-measure
  the impact of computers on student learning.
• Regression analysis and the use of beta
  coefficients can inform us only of the relative
  importance of the various predictor variables,
  not the absolute contributions, since there is
  still the joint contributions of two or more
  variables taken together that cannot be
  disentangled.

93
Future Research

• NETS-A and technology integration in schools
  where principals attended PTLA training - this
  investigation can serve as the baseline.
• NETS-A and technology integration in schools that
  have identified effective instructional leaders -
  significantly greater achievement in reading and
  math has been exhibited (1).
• NETS-A and gender differences in administrators -
  women are more likely to possess characteristics
  associated with effective leadership and
  effective schooling (2).
• NETS-A and grade-level specific technology
  integration - science teachers are three times as
  likely as English teac