Assessment: between Civilization and Culture

1
,
Assessment between Civilization and Culture
  The concept of zivilisation (civilization)
encompasses a wide variety of facts technology
levels, types of housing, kinds of behavior, the
way in which the food is prepared, religious
ideas, customs or the development of scientific
knowledge.
  Thus in Europe, the concept of zivilisation
also seems to express a type of awareness by a
Western superiority, in comparison to older
societies or contemporary societies that are
more primitive (Norbert Elias, 1994). The
concept of civilization minimizes national
differences stressing what should be common to
them all.
2
In contrast, the German concept of kultur places
special stress on special national differences
and specific group identities. kultur is a
concept linked to the expression of philosophic
systems, individual characteristics and the quest
for identity.   But there is a tremendous
difference between civilization and culture, as
already explained by Kant misleading utter
courtesy vs true virtue.
Thus, we believe more in educational processes,
in science linked to culture, that do not
minimize the differences among peoples and avoid
holding up certain standards as ideals,
enshrining the gross outcome rather than
construal of facts based on analyses that strive
to understand the different local realities.
3

• Some important viewpoints on educational reforms
  and assessments
•  
• An assessment (whether in school or not), only
  makes sense if it is underpinned by the by
  reflection and success/failure achieved in terms
  of strengthening citizenship and
• social inclusion
•  
• an assessment that helps highlight
  stumbling-blocks and hurdles along the way,
  enhancing development/self-esteem and opening up
  new horizons.
• - an assessment that helps build up a critical,
  creative and cooperative sense (Novak, Ausubel)
  an educational process (in school, for the
  teaching-learning process)
•  
• School assessment challenge
•  
• - need to create a new assessment culture that
  surpasses technical boundaries, and includes its
  ethical dimension in its dynamics
•  
• - procedures that foster cooperation (instead of
  keener competition), allowing expressions and
  respect for a diversity of voices, and cultural
  standards
•  
• - an evaluatory oversight that takes local
  challenges into account, the development of
  universal scientific thought through drawing up
  hypotheses and proposing original solutions to
  problems

4

• Tension between continuity and breaking-away
  actions
•  
• Dilemma through some superficial reforms, how
  can the solely quantitative approach be
  maintained with objective tests, such as
  assessing or redefining the course to build up a
  more democratic and humanistic approach to
  assessment?
•  
• What is the meaning of this widespread interest
  in evaluatory practices at this point in time?
  What are the objectives of the alternatives
  proposed in discussions of assessment? What
  concepts underpin the modifications implemented?
• Does the discourse on assessment transform or
  merely reform its theory and practice? Will we be
  able to get away from the exclusionary dimension
  of assessment?
• Redefining classroom routines assessment as a
  significant issue
•  
• Three key perspectives
•  
• Chart 1
• Core issue of the discourse education quality
  however, quality is assessed solely through the
  quantification of cognitive performance and
  acquired skills, through tests, meaning what has
  been presented to the students and absorbed by
  them
• Assessment as a control mechanism classroom
  time, content, processes, subjects and outcomes
• - There is no place for divergence, there is no
  place for relationships based on dialogue only
  for antagonistic discourse denying diversity -
  those who do not comply with pre-set standards
  in the classroom routine, there is no room for
  expression, acknowledgement, questioning and the
  strengthening of different voices.

5

• Chart 2
• Hybrid model
• Establishing a qualitative dynamics that,
  although relativizing right/wrong, understands
  mistakes and tolerates them. But mistakes are
  viewed as part of a process that accepts only a
  correct answer, maintaining the perception that
  there is a specific pace and only one answer as
  the most appropriate outcome - right as a
  pre-determined outcome.
•  
• - Maintain the assessment approach as a tool for
  control, adaptation and selection, although
  accepting the socio-historic nature of the
  subjects.
•  
• - Apparently more democratic, this model allows
  a more open approach, but still does not
  underscore the key aspects to be transformed,
  indicating merely superficial changes.
•  
• real heterogeneity, ideal homogeneity?!
• Stress on outcome, disregard for process
•  
• Examinations under permanent supervision, control
  (Foucault continuous surveillance aspects)
  checking the learning process, where there is no
  teaching-learning dialectic process but rather
  fine-tuning measurement techniques with an
  identified outcome, the product, reduced to
  numbers
•  
• The educational process and the complex
  pedagogical culture is reduced to figures, charts
  and statistics

6
DEPLETION / EXHAUSTION OF THE DENIAL, SELECTION
AND EXCLUSION PROCESSES, AND THE APPEARANCE OF
POSSIBILITES OF BREAKING AWAY FROM THESE
PROCESSES. Chart3 Building up a democratic
assessment based on an inclusionary standpoint.
The process of constructing a multicultural
pedagogy that sees the school as a cultural
crossroads it replaces the daily logical
exclusion based on a non-existent socio-cultural
homogeneity by the logic of inclusion, grounded
on respect for heterogeneity. Conceptualizing
the learning process and the product as a way to
denying the need felt by anyone to build up
information, for leaning purposes, as the
learning process is shaped by the reference
scheme of the student, meaning the set of the
knowledge, feelings and affects through which he
thinks and acts. This scheme has been built
throughout the lifetime of the subject, and when
entering into contact with the new information,
he may be mobilized.               
7
ASSESSMENT IN ACTION a study on the development
of a shift in trend towards research in the
teaching/learning process in the sciences.
Research Project - tutoring for Masters
Dissertations at the Oswaldo Cruz Foundation
Fiocruz on the topic of Evaluatory Study on the
process of implementing the ABC Maõ na Massa
Scientific Education Project, in the Northwestern
region of Rio de Janeiro State. Providing input
for a proposed evaluation method based on the
concept that the school has multiple territory,
where educators and students may be found, as
well as the relatives of the latter and
scientists from various fields of knowledge
ranging from the hard sciences through to the
education sciences work philosophy of the ABC
Mão na Massa Scientific Education project.
8
Justification Systematic evaluatory monitoring
gt would the project effectively foster an
investigative process? General Objectives
1. Monitor the scientific training process
during the project implementation phase at some
schools in the Miracema and Santo Antônio de
Padua municipalities, seeking to examine its
contribution to the development of science
teaching, from a critical standpoint, while
encouraging ongoing questioning among the
students 2. Research the possibility of a
task that fosters the exchange of dialogue among
the students in an environment that stresses
creative flair on the one hand, with different
lines of thought and better development of
argument by the students, and on the other the
possibility of coping with uncertainties by the
teacher 3.Describe and analyze
systematically the main educational practices and
some prototype material appropriate for the
development of the project. 4.Build up a
participatory assessment proposal for the
project, committed to cultural plurality.
9
Specific Objectives 1   This study is
designed to chart the possible ways of
implementing this project in two municipalities,
with special attention to the specific
characteristics of the local schools involved.
2  Strive to show how and to what extent the
development of the project would pave the wave
for a type of teaching that is better integrated
with the many different expressions of culture
and language, as well as building up increasingly
stronger links between teaching the sciences and
the daily life of the students. 3  Assess the
potential of the project for fostering social and
cultural inclusion for children whose families
come from more underprivileged segments of
society drops-outs and those who repeat years.
4  Analyze to what extent the contribution
of the various partnerships between
government-run primary schools, teacher training
colleges, institutes of higher education and
academic research centers, an essential
characteristic of projects in this State that
would foster the development of teaching the
natural sciences in ways that encourage the
investigative process in all its multiple
educational possibilities.
10
Evaluatory Monitoring The philosophy of the
ABC Mão na Massa Scientific Educational underpins
the development of strategies and materials
tailored to the local context one question ,
many replies. Analyzes of the meaning of the
issues related to the starting-point and the
stages of building up knowledge do they
encourage cognitive, psychomotor, social and
affective development in class? Analyzes of
the suitability of the form and raw materials
from which new materials are made durability,
format and relevant efficacy for their purpose,
as well as their exposure to risks (elements that
should be removed/substituted due to hazards).
Example little boat experiment. During the
use of the strategy or material, assess whether
a- this favors exchanges among peers in order
to answer the question (challenge)?. b- assists
in serving up elements that are hard to visualize
in reality? c-  triggers the appearance of
other questions and, tailored to the basic idea
of the project allows an assortment of
replies (equally correct) to a single
question? d-  encourages participation and
enhances spoken and written expression? e-
contributes to strengthen a critical, creative
and cooperative stance?     
11

• We are faced by powerful challenges
• Evaluate OR measure? EVALUATION IS FAR MORE
  COMPLEX
• Even if we want to measure, what or how should
  this be done?
• Latin American experiences -
• a designing test to measure the learning process
• When preparing educational tests for measuring
  the learning systems in Latin America, there are
  two alternatives
• prepare tests that allow the comparison of the
  learning process in specific groups of students
  OR
• b) prepare tests that disclose what specific
  aspects or a knowledge or objectives the students
  have learned.
• Coping with this dilemma requires developing
  tests based on paradigms that ARE GROUNDED ON
  VERY DIFFERENT THEORETICAL BASES.
• In the fist case (A), the measurement is related
  to STANDARDS privilege of the function of
  arrangement or discrimination among groups and
  individuals
• MOST OF THE LATIN AMERICAN COUNTRIES
  HAVE DEVELOPED TESTS FOR THEIR LEARNING SYSTEMS
  IN COMPLIANCE WITH THE PSYCHOMETRIC MODEL THAT IS
  BENCHMARKED TO STANDARDS.
• WHAT INFORMATION CAN SIMPLE AVERAGES
  RELEGATE TO EDUCATORS, WHO CAN INTERVENE TO
  UPGRADE THEIR WITH CHILDREN IN THE CLASSROOM?

12

• Can simple averages provide the educator with
  much information or the administrator for drawing
  up a syllabus for the classroom?
• Reasons why the benchmark approach is used
• abundant prior experience and information
  available internationally on the procedures
  traditionally followed for validating tests of
  this type
• knowledge on this type of approach in the
  technical department of the Ministries
• - availability of computerized statistical
  packages that allow the issues to be analyzed as
  well as other technical analyses.
• While For the later (B), the measurement
  paradigm based on CRITERIA must be used. In
  counterpart, this type of measurement is intended
  to show mainly whether individuals have mastered
  a certain field of contents and/or skills.
•  
• Few measurement systems in Latin America have
  developed criteria-based tests.
• This measurement alternative can provide
  information of educational significance. In this
  case, the first basic question is what is to be
  measured?
• What are the core objectives? Who are the
  professionals who could stipulate them? For
  example, this reply will always depend on the
  purpose that is going to be measured.
  Nevertheless, there are many professionals
  called upon to scientifically draw up a
  criteria-based measurement tool and they should
  be very familiar with the priorities of the
  Program in order to define their order.

13
Conceptual and methodological differences between
benchmark-based tests and criteria-based tests
14
CONCERN OVER THE PROPER USE OF THE FINDINGS
OBTAINED THROUGH THE TESTS MEANS CONCERN WITH
UNDERSTANDING THE TYPE OF LEARNING INTERPRETATION
THAT THEY ALLOW (MEANING WHICH MEASUREMENT IS
TAKEN AS VALID) VALIDITY HOW THE LEVEL TO
WHICH EMPIRICAL AND THEORY UPHOLD THE
INTERPRETATION OF THE FINDINGS PRODUCED BY A
MEASUREMENT PROCESS. CASE VALIDATIONS - EXAMPLES
Case 1 Knowledge measurement X or reading
capacity? A low average will indicate that the
student has not learned (failed to acquire the
knowledge) that it is intended to measure or
rather does it measure the reading capacity of
the children? (ability to understand the written
tests or a highly developed vocabulary) Case 2
Is this a problem of educational efficacy or
access to resources? By comparing the averages
between public and government-run establishments,
the averages posted by the former are seen to be
higher does this lead to the conclusion that
these schools are better? (even when the
government schools do not have access to attempt
addressing one of the most important topics in
the test?) WOULD THIS NOT BE A PROBLEM OF LACK OF
EQUITY? Case 3 General mastery of knowledge
or lacking familiarity with the content area that
was measured most? The various components of
the syllabus must be measured with equal
stringency.
15
Case 4   Pertinence of educational evaluation
and the danger or comparisons When comparing
the outcomes in different regions, it appears
that some of them posted higher scores ? would
their schools be better? Might the selected
topics not be making a difference? These
different averages may merely be indicating a
difference in terms of the pertinence of the
tests between each one of the countries, states
or regions involved in the comparison.
VALIDATION (ACCUMULATION OF EVIDENCE UPHOLDING
THE CONSTRUALS THROUGH MEASUREMENTS) IS A CONCEPT
THAT FORCES ACROSS-THE-BOARD-EVALUATION OF ALL
THE EVIDENCE AVAILABLE THAT JUSTIFIES THE
CONSTRUAL OF THE DATA, AS WELL AS THE USE OF THE
INFORMATION GATHERED HOW CAN EVIDENCE BE
OBTAINED TO QUESTIONS THAT ARE CONSTRUED IN THE
SAME WAY IN DIFFERENT PARTS OF A SINGLE COUNTRY
OR AMONG GROUPS THAT ARE DIFFERENT IN
LINGUISTICS, CULTURAL, SOCIAL AND ECONOMIC
TERMS? A VALIDITY IS SPECIFIC TO EACH
POPULATION A TEST VALIDATED FOR ONE COUNTRY
CANNOT BE CONSIDERED AS AUTOMATICALLY VALIDATED
FOR OTHER POPULATIONS. CONGRUENT TO THE
MEASUREMENT AND THE EDUCATIONAL REALITIES THAT
ARE BEING MEASURED ? TAKING A POSITION BASED ON
AN INVALID INFERENCE THAT IS EQUIVALENT TO TAKING
A GROUNDLESS DECISION ? WARNING TO ANALYZE THE
INFORMATION PRODUCED BY THE EVALUATION
SYSTEMS. AT THE SAME TIME AS THERE IS AN
AWARENESS OF THE NEED TO BE SUPPORTED BY
INFORMATION, KNOWING ITS SCOPE AND LIMITATIONS
16
New horizons evaluating real performance
Brazil old difficulties, new
challenges
STRESSING THE MEASUREMENT OF SKILLS AND KNOWLEDGE
AS WELL AS REASONING WITHIN A CONTEXT OF THE REAL
WORLD TO WHICH SUCH KNOWLEDGE AND SKILLS ARE
APPLIED ? POSSIBILITY OF APPLYING SOME OF THE
PERFORMANCE ASSESSMENT TOOLS ON SMALL SUB-SAMPLES
OF STUDENTS IN ORDER TO POSSIBILITY KEEP COSTS
LOW, IN ORDER TO OBTAIN INFORMATION ON COMPLEX
KNOWLEDGE AND LEARNING SKILLS THAT COULD NOT BE
MEASURED BY USING THE TRADITIONAL MUTIPLE CHOISE
TESTS.
17
An ethical modernization project, with major
impact on the quality of life among the
population.
18
Directive assessment is not judgment!
  The examination says you DO NOT know. It
classified and excludes. The assessment says you
STILL have not got there and indicates a path
for building up this learning process. It
diagnoses and includes.