MCI WorldCom Board Meeting

1
Point of ViewThe Intersection of Measurement
and ManagementCaleb M. SchutzPresident,
MarcoPolo Education FoundationVice President,
MCIJune 12, 2003
2
The CP Management Dilemma

• Create favorable business conditions and
  relationships to help the company and
  stakeholders prosper

• Minimize or eliminate societal problems
• Maintain or enhance quality of life

3
The CP Management Dilemma Regard for Corporate
Executives
4
The CP Management Dilemma Measurement is Power

• Judges and juries are created
• Conclusions are reached
• Counter-intuitive results (surprises) emerge
• Changes are made
• New solutions are found
• Careers are started and ended

5
The CP Management Dilemma What is Missing from
Measurement in Corporate Philanthropy?

• Academic interest degrees, research, careers
• Industry expertise and funding
• Theories and models e.g. giver/receiver
  compatibility, customers, employees, stockholders
• Standard measures surveys, assessments, norms,
  benchmarks
• Outside pressure
• CP management belief, interest, commitment,
  expertise
• Lack of concrete, shared examples

6
Vision and Structure MarcoPolo Mission Since 1997

• To create premier, standards-based Internet
  Content for the Classroom and provide the
  requisite Professional Development to every
  teacher in the United States
• To provide the infrastructure, distribution and
  services that ensure MarcoPolo is scalable and
  will readily enrich teaching and dramatically
  affect student achievement for generations to
  come
• To be the benchmark in our areas of expertise,
  sharing results openly and contributing to
  overall excellence in education

The MarcoPolo program has had a profound impact
on American education and it has the potential to
be a national treasure serving our teachers and
students for generations to come. -- Ray
Simon, Director, Arkansas Department of Education
7
Vision and Structure MarcoPolo Program Values

• Curriculum free to educators and students
• Relationships built on long-term mutual
  commitments and goals, and based on integrity,
  trust and honesty
• Comprehensive, real time measurement to diagnose
  and improve and to encourage external
  verification of results
• Commitment to customer service and agreement that
  teachers are the final arbiters
• Ongoing innovation and leadership which helps set
  the industry standards in all programmatic and
  technological endeavors

8
Vision and Structure MarcoPolo Program Elements
Resources and Infrastructure
Content
Training
9
Goals and Outcomes MarcoPolo Goals Progress To
Date
Goals for 2005
Current Progress
10
Impact Monthly Web Site User Sessions
A user session is defined as a session of
activity (all hits) for one visitor of a web
site. A visitor session is terminated when a
visitor is inactive for more than 30 minutes.
These numbers reflect the average number of
monthly user sessions per quarter.
11
Impact Program Comparison Web Links to Program
12
Impact MarcoPolo Professional Development
Educators trained through 5/15/03 176,873
Public schools covered through 5/15/03 14,115
Number of completed trainings
Trainings do not include Distance Learning
Sessions Source Rusticello Professional
Development Database
13
Impact Average Daily Activity
Content
Professional Development

• 7 registered trainings
• 72 educators trained and rostered
• 60 training surveys submitted with trainer
  quality rating of 4.62 out of 5.0
• 174 Teachers Guides downloaded
• 490 in sales of training and materials

• 144,478 page views
• 49,530 user sessions
• 16,139 unique visitors to MarcoPolo Web sites
• 6,049 searches on the MarcoPolo Search engine
• 11 Listserv messages reaching 16,608 subscribers
• 3,500 MarcoGrams delivered to educators nationwide

Total March activity was divided by 31 days in
March to calculate average daily activity.
Average Sales are based on 21 business days.
14
Impact Rusticello Real Time Tracking
Roster information
Training survey results
Training survey comments
Trainers report back on their trainings
15
QualityUsing Feedback
Xpeditions-specific write-in comments
Prior to Redesign
After Redesign
Source Teacher Training Survey Partner-specific
comments, May 1- Jul. 31, 2001
Source Teacher Training Survey Partner-specific
comments, Feb 1- Apr. 30, 2001
16
Quality Survey Data MarcoPolo Content Quality
How would you rate the QUALITY of the content
(lesson plans, links, and other resources) on the
MarcoPolo web sites?
Source Teacher Training Survey, January 1, 2000
May 15, 2003 108,245 respondents
17
Quality One-Year Follow-up Survey
Content Quality
Teacher Use
How often do you use MarcoPolo in your teaching
(for preparation or with students)?
How does the quality of the MarcoPolo content
compare to other online educational content that
you use?
Source MarcoPolo One-Year Follow-up Survey.
2,332 respondents June 11, 2003
18
Quality Survey Results Comparison
National Teacher Survey
MarcoPolo Survey
Please evaluate the overall training session.
How satisfied are you with the professional
development in which you have participated during
the last 12 months?
Source The MetLife Survey of the American
Teacher and Harris Interactive Inc., 2001--1,273
respondents.
Source Teacher Training Survey, January 1, 2000
April 4, 2003 -- 106,066 respondents
19
QualitySearch Engine Comparison
Sample Comparison Search Poet
MarcoPolo 678 Partner-developed or panel-reviewed
K-12 resources
Yahoo Sponsors, advertising and 2.4 million
resources for education, commerce, entertainment,
etc.
Google 2.7 million commercial and non-commercial
web sites
Category Match Content Management Software
Advertisement Poet In My Window CD
Sponsor match Poetry contest 58,000 in prizes!
Poet Software Corporation
Poet Software Corporation
The Poets Corner File not Found
20
Preliminary AssessmentsUsing Survey Data to
Guide Development
How important are the following features of
Internet content to your teaching? (5most
important 1least important)
Source MarcoPolo Teacher Training Survey
10,132 respondents
21
EffectivenessResearch
Standards-based Lesson Plan
Interactive Applet
Pre- / Post-data
Setting Clear Goals and Benchmarks
Research from Michigan State University
22
Effectiveness Letters of Support
23
Effectiveness MarcoPolo Field Trainer Recognition
24
Effectiveness Transparent, Real-Time Data
25
Value to the Corporation Branding
26
Value to the Corporation Growing the Customer
Base
Why did you start using the Internet?
Source Surveying the Digital Future The UCLA
Internet Report, Year Three UCLA Center for
Communication Policy, February 2003 2000
households responding
27
Value to the Corporation Telecom Products

• Internet Access
• Dedicated Web Hosting
• Security
• VPN
• Paging
• Audio conferencing
• Toll Free Service
• Local and Long Distance
• Calling Cards

28
Value to the Corporation Measuring Return on
Investment
Cumulative Investments in MarcoPolo

Contributions include costs for substitute
teachers, trainer stipends, technology labs, and
other training costs
29
Recommendations

• Ensure there are sustained efforts to increase
  the sophistication of measurement in the field
  societal and business impact
• Aim for a set of standard CP outcomes/impact
  measures that will be used by all industries
• Initiate systematic evaluation of CP performance
  in helping communities and society
• Establish a culture in which measurement is used
  first and foremost for continuous improvement,
  not as a guillotine
• Freely share results so that maximum gains can be
  made in society through collective CP investments