Nebraska 4-H Robotics

1
Nebraska 4-H Robotics

• Effectiveness of Educational Robotics in the
  Classroom

2
Presenters

• Dr. Bradley S. Barker, 4-H Science and Technology
  Specialist
• Email bbarker_at_unl.edu
• Dr. Neal Grandgenett, Professor of Mathematics
  Education
• Email ngrandgenett_at_mail.unomaha.edu
• Dr. Gwen Nugent, Associate Research Professor,
  Nebraska Center for Research on Children, Youth,
  Families and Schools
• Email gnugent1_at_unl.edu

3
Presentation Overview

• Provide background of using robots in non-formal
  education
• The three iterations (pilot study, large scale
  study, NSF ITEST program) and impact data from
  the 4-H robotics program.
• Embedded assessment (Reasoning/ Communication)
• Effects of robotics on attitudes
• Summary
• Questions

4
Purpose of Program

• To address the shortage of students pursuing
  careers in science, technology engineering, and
  mathematics (STEM).
• Goals
• promote youths interest in STEM
• fields (including IT),
• introduce basic STEM skills,
• foster problem solving and inquiry,
• and encourage teamwork

5
Background Why Robotics?

• Integrate many content areas
• (CS, Engineering, Mathematics)
• LEGOs are familiar to youth
• Durable and reusable
• Relative low-cost
• Programming language works
• on Macs and PCs
• Widely available
• Motivating

6
NXT Robot with GPS trailer
7
RCX Robot
8
Review of Literature

• Fundamental gap in literature
• what is the impact of using
• robotics on STEM learning?
• Most studies are qualitative,
• examine self-evaluations, project
• descriptions, and student reactions.
• We set out to examine the impact on STEM learning
  in a nonformal environment.

9
Pilot Study Gibbon Elementary

• Purpose was to develop assessment instrument and
  examine impact on learning STEM concepts.
• Developed 24-item multiple choice test one item
  from each unit in the curriculum.
• Test items reviewed by CMU robotic experts and
  revised.
• Robotics group met twice a week for six weeks.
  Pre test prior to intervention and post on last
  day for both groups.

10
Pilot Study Participants

• Participants
• 32 students in study ages 9-11
• median age was 9.0
• 14 students in (9 male,
• 5 female) experimental group
• 18 students in control group (11 male, 7 female)
• Control group selected by instructor in same
  grade but not part of the afterschool program.

11
Pilot Study Results

• Cronbachs alpha score of 0.86 on posttest.
• LEGO Robotics questions removed
• STEM assessment items alpha was calculated at
  0.76
• The assessment instrument seemed to be valid and
  reliable.
• Used Pell and Jarvis instrument to measure
  attitudes towards science.

12
Pilot Study Results

• Observed impact on learning
• No significant difference (t(30) 11.60, p .70
  on pretest scores between groups (M7.50, SD
  2.58, control) and M 7.93, SD3.71,
  experimental).
• Significant difference (t(22,17) 12.93, P lt
  .000 between groups on posttest (M7.44, SD
  2.98, control) and M 17.00, SD .88,
  experimental).
• No change in attitudes

13
Pilot Study
14
Pilot Results

• Refined the assessment instrument.
• Felt confident students had an increase in STEM
  content areas as well as specific robotic
  concepts.
• Used instrument for larger study.

15
Large Scale Study (RCX)

• Participants
• 121 students ages 7-14 from 6 afterschool
  programs and 3 4-H clubs.
• 36 youth ages 11-14 acted as a control group from
  3 afterschool programs
• Interventions (not concurrent) lasted for 8
  weeks. The pretest was administered prior to the
  intervention and the post test was administered
  immediately after.

16
Large Scale Study Results

• Impact on learning
• ANCOVA analysis used posttest as dependent and
  pretest, gender as covariates
• Main effect was significant F(1,141) 11.04
  p.001
• Posttest (M10.68, SD 3.93, control) and (M
  11.09, SD 3.93, experimental).
• No difference based on gender F(1, 141) .833
  p.478

17
Boxplots Pretest to Posttest
18
NSF ITEST Program

• Expanded 4-H robotics program
• Includes the integration of robotics
• with Geospatial technologies
• (GIS, GPS, aerial photography)
• Looking at applications in precision
• agriculture and natural resources
• Provide career exploration with visits from
  scientist and engineers.
• New robotic kits

19
ITEST Program Components

• Open to middle school students (200)
• Year 1 - Pilot camps
• Year 2
• Start with 40-hour summer camp
• 1 day camp for educators and leaders
• Youth then complete 80 hours in clubs and
  after-school programs
• Year 3
• 40-hour summer camp
• 80 hours in clubs and after school programs

20
Results of Pilot Camps

• Site 1 Gretna, NE 6 day overnight camp.
• N 12 ages, 11-14, median age 12.50
• 8 males, 4 females
• Paid to attend the camp
• Site 2 - Grand Island, NE 5 day camp.
• N 26 ages 11-15, median age 12.00
• 18 males, 8 females
• Offered through CLCs at Barr M.S.
• Modified content examine and added new questions,
  piloted an embedded assessment

21
Camp Results Academic

• Site 1 pretest m 14.5, sd 4.42 and posttest m
  17.5 sd 4.89
• Site 2 pretest m 11.77, sd 2.99 and posttest
  m 16.04, sd 3.68
• Overall significant increase in scores using
  ANCOVA analysis using posttest as dependent and
  pretest as covariate F(1,31) 21.24, p .000.
• KR20 score .81, Alpha score .799

22
Boxplots Pretest to Posttest
23
Boxplots by Content Area
24
Camp Results Attitude

• Developed new attitude instrument focusing on
  robotics
• Modeled after the Motivated Strategies for
  Learning Questionnaire (Pintrick, Smith, Garcia,
  McKeachie, 1991)
• Multiple scales focusing on task value,
  motivation, self-efficacy, problem solving,
  cooperative learning
• Conducted interviews with selected youth in Grand
  Island

25
Camp results Attitude

• Goal - promote youths interest in STEM fields

4-point scale
26
Camp results Attitude

• Promote youths interest in STEM
• Six of the seven Grand Island youth interviewed
  said the camp made them like math and science
  more
• Four said it increased their interest in a career
  in STEM

27
Camp results Attitude

• Goal foster problem solving and inquiry skills

Significant increase for make a plan
28
Camp results Attitude

• Interviews
• There was kind of a big task, so it made us break
  it down into little parts and then get it done
  step by step.
• For the robot, you have to start with nothing and
  tell it exactly what to do.
• If your robot did something wrong, you have to
  figure out after how many steps it was and then
  you fix the particular problem.

29
Camp results Attitude

• Goal Encourage teamwork

Significant increase for work with others and
listening to others
30
Camp results Attitude

• Interviews
• I like the hands on aspect of it. I hate just
  being stuck in a chair and writing stuff down.
• It was fun when we did challenges.

31
Future Directions Attitude

• Revise the instrument
• Factor analyze the instrument and revise
• Use results to guide instructional improvement
• Initiate new study that measures attitude mid-way
  through the camp

32
Piloting Embedded Assessment
Robotics Instruction
Pretest Measures
Posttest Measures
Content Tests Attitude Test
Content Tests Attitude Test
Envelope Activity
Think-Aloud Interview
Task Rubrics
Design Tasks
Control Groups Partners and Partner Schools
33
Embedded Asssessment Envelope
Activity(mathematical communication/reasoning,
and also used to discuss programming instruction)
Steps to Mailing a Letter Student
Directions Directions In order to do something
well, like playing a sport or a board game, you
usually need to know the "procedures" of how to
do it. Thus, it is often helpful for you to know
the steps in doing some task, before you are able
to explain that task or procedure to someone
else. For example, if you are given a letter
that you need to mail, with an envelope, a stamp,
a ruler, and an address of where you want to send
the letter, can you describe how you would mail
that letter? Or in other words, can you describe
the general procedure for mailing a letter?
Let's further pretend that this person who will
be mailing the letter is a friendly alien from
Outer Space, and that they have absolutely no
idea about how to mail a letter here in the
United States. List the steps that the alien
should use to mail a letter. Use the blank
sheets of paper provided to make your list. You
have 15 minutes to accomplish this task.
34
Sample Products
35
Scoring
36
Envelope Activity Analysis
37
Student Interviews

• Are there any similarities between the
  envelope activity and programming?

• Are there any differences between the
  envelope activity and programming?

It is easier to see the result when you tell a
robot
You have to do the steps very carefully
It is more fun to write steps on the computer
than with pencils
Both big things need to be broken down into
little steps
I am better at thinking than a robotthey just
do what they are told
I had to tell a thing that knows nothing how to
do something
38
Summary

• Robotics seems to have a promising potential
  impact on academic achievement
• Robotics impact on attitudes is difficult to
  measure current results suggest that the impacts
  are limited to specific areas.
• More research is needed into long term effects

39
Summary

• Robotics seems to have a potential impact on
  learning of robotics concepts and principles
  based on pre to post test scores
• More research is needed into long term effects
• Intervention seems to increase interest in STEM
  but it is difficult to measure directly and
  mathematical interest is particularly challenging
• Embedded assessment is a promising area of
  investigation for robotics activities

40
Questions?
41
Back Page
A final quote following questions..
We have not succeeded in answering all of your
problems. The answers we have found only serve
to raise a whole set of new questions. In some
ways, we feel we are as confused as ever, but we
believe we are confused on a higher level and
about more important things.
Omni Magazine, 1992