Title: Developing and Assessing the Research Skills of Students
1Developing and Assessing the Research Skills of
Students
-
- Dr Said Al-Sarawi
- Research Skill Development and Assessment
- ALTC Project Member
- Education Research Group of Adelaide (ERGA)
- School of Electrical and Electronic Engineering
- University of Adelaide
- A seminar at Edith Cowan University
2Outline
- Motivation
- Students and staff issues
- RSD Framework in Australia
- Benefits of using RSDF
- What is RSD Framework?
- Case Studies From 1st Year to Master Level
- Other Dimensions of RSD Framework
- RSD at the Program Level !
- RSD in conclusion
- Discussion and future work
3Motivation (1/2)
- PhD completion rate were doubled for students who
had participated in undergraduate research (Bauer
Bennett, 2003) - Performance based research funding (old RQF,
ERA) - Performance based learning and teaching support
Learning and Teaching Performance Fund (DEST,
2006) - Undergraduate research has been associated with
higher level of student satisfaction and generic
skills development (Kardash, 2000) - Factors affecting skills implementation are
(Lucas et al, 2000) - Scepticism of the message, the messenger and its
vocabulary - The skills demanded lack clarity, consistency and
recognisable theoretical base - The skills are dependent on discipline area
4Motivation Student and Staff issues (2/2)
Staff Student
How can I develop and assess students research skills? What is meant by Research and what skills are needed?
Is there a systematic, explicit approach that can be used? How I can acquire and develop these skills?
Is this issue only limited to the engineering discipline? How are they relevant to my career aspirations?
How can I remove the subjectiveness from the assessment? How will my skills be assessed?
5RSDF in Australia
- The University of Adelaide (John Willison)
- Human Biology (Eleanor Peirce Mario Ricci)
- Electrical Engineering Masters by Coursework
(Said Al-Sarawi and Brian Ng) - Clinical Nursing (Frank Donnelly)
- Petroleum Engineering (Steve Begg)
- Introductory Academic Program (Richard Warner)
- English (Joy McEntee)
- Dentistry (Vicki Skinner and Leonard Crocombe)
- Oral Health (Sophie Karanicolas and Cathy
Snelling) - Software Engineering (Li Jiang)
- Veterinary Science (Susan Hazel)
6RSDF in Australia (Conts.)
- Macquarie University (Psychology Judi Homewood)
- Monash University (Business Ethics Jan Schapper
Sue Mayson Business Glen Croy Tourism) - University of Melbourne (Business Law Eu-Jin
Teo) - University of South Australia (Introduction to
Tertiary Learning, 2008 Rowena Harper)
7(No Transcript)
8The facets of student research
- In researching, students
- embark on an inquiry and so determine a need for
knowledge/understanding - find/generate needed information using
appropriate methodology - critically evaluate information/data and the
process to find/generate - organise information collected/generated
- synthesise and analyse and apply new knowledge
- communicate knowledge and the processes used to
generate it, with an awareness of ethical, social
and cultural issues. - (Willison ORegan, 2007)
9Learning to Frame Research Questions
- Level 1
- Facet A) Embark on Inquiry
- Respond to questions / tasks arising explicitly
from a closed inquiry. - Facet E) Synthesis, analysis, application Ask
questions of clarification / curiosity.
10Learning to Frame Research Questions
- Level 2
- Facet A) Embark on Inquiry
- Respond to questions / tasks required by and
implicit in a closed inquiry. - Facet E) Synthesis, analysis, application Ask
relevant, researchable questions.
11Learning to Frame Research Questions
- Level 3
- Facet A) Embark on Inquiry
- Respond to questions / tasks generated from a
closed inquiry. - Facet E) Synthesis, analysis, application
- Ask rigorous, researchable questions based on
new understandings.
12Why use RSD approaches? (1/2)
- Benefits of using the RSD for students (according
to Eleanor Peirce and Mario Ricci, Medical
Sciences, Uni of Adelaide) - Their research skills in our course have
improved. - They understand much more clearly what is
expected of them. - They know exactly where they need to develop,
thanks to feedback.
13Benefits for Lecturers (2/2)
(according to Eleanor Peirce and Mario Ricci,
Medical Sciences, Uni of Adelaide)
- We can give feedback on assessment tasks more
accurately and efficiently we can give the same
quality feedback with less writing, and faster. - We can get a much better idea of where our
students are from a quick analysis of the RSD
results than from a detailed analysis of standard
marks. - We can easily match assessment tasks with course
objectives, and course objectives with the
Universitys Graduate Attributes.
14Case Study 1 Human Biology - 1st Year
- Read the two short then complete tasks 1 and 2.
- Task 1
- Integrate the information presented in the two
articles to write your own dot-point notes - on the worksheet attached. To do this
- Identify 3-4 key ideas from the articles
- Use these key ideas to formulate headings and
underline each - Make bullet-point notes and list them under these
headings. - After each point, indicate its source, i.e.
whether the idea came from article 1, article 2,
or both - Provide a title that embodies the content of
your notes.
O Week RSD O Week RSD O Week RSD
I II
A
F
Task2 Which of the two articles do you consider
to be the better source? On what
characteristics/features of the article have you
based your choice?
15Case Study 1 Human Biology - 1st Year
16Case Study 1 Human Biology - 1st Year
Literature Research Skill Stream
O Week RSD O Week RSD O Week RSD
I II
A
F
Small Group Inquiry Small Group Inquiry Small Group Inquiry Small Group Inquiry Small Group Inquiry
I II III IV
A
F
Lit Week RSD 1 Lit Week RSD 1 Lit Week RSD 1 Lit Week RSD 1
I II III
A
F
Lit Week RSD 1 Lit Week RSD 1 Lit Week RSD 1 Lit Week RSD 1
I II III
A
F
Laboratory Research Skill Stream
Individual Open Inquiry Individual Open Inquiry Individual Open Inquiry Individual Open Inquiry Individual Open Inquiry
I II III IV
A
F
Lab RSD 1 Lab RSD 1 Lab RSD 1
I II
A
F
Lab RSD 2 Lab RSD 2 Lab RSD 2
I II
A
F
Field and literature research
Semester 1
Semester 2
17Novelty in the engineering case
- The differentiation points
- Clearly state the need to research skill
development - The students are required to identify a gap in
knowledge for each of the chosen topics (the
unknown) - Rigorous literature research
18Case study 2 Photonic and Communication Master
Course
- Masters (coursework) course at EEE
- Course run by senior colleague in EEE
- 2 units of lectures/exams
- 1 unit of literature research project
- Students seek supervisors individually
- Topic chosen by student, but requires approval by
supervisor - Demographics
- Almost all are international students usually
East Asian background - Technically capable, but generally lack prior
experience in conducting research - Goal produce a high quality review paper on
chosen topic - Mostly negative experiences in 2005
- Cohort lacked basic literary research skills
- Unstructured approach towards project
- Low quality final review papers
19Process Details
- Initial diagnostic task
- Supervisor supplies two technical papers
- Different levels (magazine, journal) for contrast
- Students summarise and compare the key points
from both sources in one structured, bulleted
list - Detailed supervisor feedback in written form
- Knowledge accumulation phase
- More articles added to the reading list
- Continually add to an organic structured,
bulleted list - Strategy on further reading
- Student applies critical evaluation on
suitability of sources with supervisor input - Fortnightly workshops for group presentations
discussions - Supervisor supplies feedback optional external
advice (CLPD) - Writing phase
- Student writes review paper based on list
- Supervisor feedback on first draft approx a week
before submission
20Outcome and Evaluation
- Outcomes from 2006
- Total of 6 students plus1 external (industry)
student as control - Qualitatively, much improved papers compared to
2005 - Coherent structures, logical arguments,
conciseness, respect of referencing practice - Student perspective
- struggled to cope with demands of research among
the pressures of regular coursework - English as second language remained a great
barrier - Framework matrix useful for quantitative
assessment - Unexpected benefit reduction in plagiarism
- Turn-it-in software reported major improvements
from 2005 - Reports tend to be on conservative side.
21Case 3 Final Year Project (in Progress)
We have been concerned about a degree of
disconnect between the desired leaning outcomes
and the assessment methods used in these
projects. HoS of School of EEE, The University
of Adelaide.
The set of project deliverables are The set of project deliverables are
Semester 1 Formative literature assessment (diagnostic value?) Proposal seminar design document/interim report Critical design review (peer review)
Semester 2 final report final seminar project log books/wiki/blog/email log etc
22RSD Useful First-year to PhD?
23Variety of RSD approaches
- Five distinct approaches have emerged in the use
of the RSD so far -
- Rubrics base to assess the profile of skills for
each student, as demonstrated by Human Biology
rubrics. this is the most common approach, and
is used by most disciplines. - A lock-step approach, whereby students are kept
'in formation' and progressively and corporately
develop one level of skills at a time. This is
shown by the Nursing RSD rubrics. - Grading with a specific grading, by incorporation
into SOLO taxonomy to define grading within a
specific level set, as used by Dentistry. - Skill Evaluation, to evaluate the skills and
levels required by existing assessments. - Identification of research skills of higher
degree students, Masters by Research and PhD
students and/or their supervisors/advisors to
locate their present skill set and plot future
directions and development needs.
(SOLO Structured Observed Leaning Outcomes)
24Other Dimensions of RSD
- Degree of Autonomy
- Degree of Academic Rigour
- Degree of Conceptual Demand
- Degree of disciplinary knowledge required
- Status of knowledge being pursued
25RSD Framework at Program Level !
No studies that consider student outcomes of the
explicit development and assessment of research
skills over a whole undergraduate or
masters-by-coursework program
- Undergraduate Level
- Bachelor Oral Health, Adelaide Uni, AU
- Bachelor of Media Studies , Adelaide Uni, AU
- Bachelor of Science, Adelaide Uni, AU
- Bachelor of Nursing, Adelaide Uni, AU
- Bachelor of Business, Monash Uni, AU
- Bachelor of Science (Psychology), Macquarie
University, AU - Postgraduate Level
- Master of Engineering (Advanced) (Electrical),
Adelaide Uni, AU - Bridging Program for International PhD students,
Adelaide Uni, AU - Masters and PhD (Nursing), PhD (Nursing)
- Trinity College Dublin, Ireland
- Master of Business, Monash University, AU
26RSD in conclusion
- Provides the Big Picture and relates this to the
assessment details for course coordinators,
lecturers, tutors, and especially students - Informs assessment-first curriculum redesign
- Same facets for multiple assessments, various
levels - Explicit Transparent assessment criteria
- Coherent Incremental skill development
- Revisited (potentially) Cyclic Conceptual
Framework
27Discussion and Future work
- How to integrate the framework into other
Programs? - How will the implementation of RSD framework
affect academic workload? - How to monitor and assess students progress?
- How can this be implemented for larger class
sizes? - How can the framework adopted for non-literary
research skills?
28References
Acknowledgement
This session was funded by an Australian Learning
and Teaching Council Grant
RSD Web Site
http//www.adelaide.edu.au/clpd/rsd
- Bennett, N., Dune, E. Carré, C. (2000). Skills
Development in Higher Education and Employment.
(Society for Research into Higher Education
Open University Press). - Stevens, C. Fallows, S.J. (2000). Integrating
Key Skills in Higher Education employability,
transferable skills and learning for life.
Routledge, ISBN 0749432659. - Lucas, U., Cox, P., Croudace, C. and Milford, P.
(2004). Who Writes This Stuff? Students
Perceptions of Their Skills Development. Teaching
in Higher Education, 9(1), 55-68. - Willison, J.W. ORegan, K. (2007). Commonly
known, commonly not known, totally unknown A
framework for students becoming researchers.
Higher Education Research and Development, 26(3).