Session 2:
Learning goals and operationalisations (4.2)

Why learning goals need to be operationalised

As was shown and discussed in the first session of this module, it is important that the learning outcomes and the assessment criteria (and the assessment procedure!) should fit together.

Assessment criteria in learning and teaching should be identically or similarity with the learning goals, when the teachers wants to reach the learning goals (and not only the assessment criteria).

Nevertheless, learning goals (e.g. in curricula) are very often very general. So it is very difficult to assess an e-portfolio for its creativity or self organisation. It is hard to say, how and when a learner reached this goal.
There are different levels of learning goals and for that, different levels of assessment (Hager & Butler, p. 68):

• knowledge, skills and attitudes
• performance in simulated or practice domains
• personal competence in the practice domain

To make a fair, objective and correct assessment possible a teacher should "operationalise" the learning goals. In e-portfolio work, normally learners are involved in that process.

Sources for evaluation and assessment

Not every potential operationalisation of learning goals can be found in an e-portfolio. But in portfolio work, there exist several sources which can be used for the assessment.

From the viewpoint of assessment, the rationale for portfolios is clear: there are a number of valuable activities and attainments that cannot be assessed using the format of timed tests. The ability to create, design, reflect, modify and persevere are all important goals of education. It is entirely appropriate to assess these processes by collecting evidence on the ability to engage in an extended piece of work, and to bring it to a successful conclusion by the creation of some product – lab report, video, installation etc. Part of the portfolio can (should) provide evidence of the range of personal skills demonstrated, perhaps under the headings suggested in the Tomlinson Report (2004): student selfawareness – of themselves and the ways they learn and what they know; how students appear to, and interact with, others; thinking about possible futures and making informed decisions. A section of the portfolio in the form of a viva, or simply annotations of products where students show their attainments in these three aspects of performance is appropriate. (Source: Future Lab Review )

In general, the following different sources for assessment can be described Hager & Butler, 217).The popular techniques for assessment and their brief description:

• Journal: shared account of a person’s actions, thoughts and feelings written by the person himself or herself, usually on a daily basis.
• Diary: private account of a person’s actions, thoughts and feelings written by the person himself or herself, usually on a daily basis.
• Verbal Report: account given by individuals of their thought processes, feelings, ideas etc.
• Questionnaire: form on which there is a set of questions to be answered by a number of people so that information about those people which is of interest to the researcher can be collected.
• Checklist: a list of items to be checked by a person.
• Self-rating forms: persons’ critical rating of their own work.
• Self reports: persons’ report on their own work.
• Field-notes: written comments made in the process of professional action.
• Interview: conversation or meeting intended to gather certain information.
• Case Study: an in-dept study of one particular student, teacher, class, school etc.
• Observation: process of watching or listening to professional action either while it is happening, or from a recorded sequence.
• Teachers' portfolios (Teaching portfolios): an anthology of achievements that the teacher has accomplished, both in the classroom and elsewhere. Items often found in a portfolio include a statement of the teacher's philosophy of teaching, teacher's curriculum vitae, examples of materials, activities, or lesson plans that the teacher has developed, video clips of teacher's classroom teaching, samples of student, peer, or administrative assessments of the teaching, and so on.

Assessing E-Portfolio itself

Sometimes, e-portfolio work in itself should be assessed.

Assessment with e-portfolio has to consist of objective criteria on the one hand (like completeness) and criteria, which show the degree of the ability to reflect on the learning process on the other hand (Richter 2006: Portfolios im universitären Kontext: Wann, wo, wie? In: Brunner, I., Häckert, T.; Winter, F. (Ed.). Das Handbuch Portfolioarbeit, S. 234-241.).
E.g. the description of a mature e-portfolio by Challis is helpful (Challis, D. (2005): Towards the Mature ePortfolio: Some Implicatios for Higher Education. In: Canadian Journal of Learning and Technology. Vol. 31(3). :

• Selection of material (e.g. relevant – everything tied to the set purpose and audience, carefully selected to make obvious specific points, no unprocessed or trivial material)
• Level of reflection (e.g. reveals deep understanding, illustrated self-awareness and growth, incorporates and is responsive to feedback from others)
• Content (e.g. reveals considerable thought over a period of time, is contextualized, reveals personality as well as thought, all text is accurately)
• Use of multi-media (enhances content and engages, appropriate and purposeful, high quality audio/video, non-distracting)
• Design (e.g. uncluttered and elegant, graphics are in accord with portfolio’s purpose and its creator, no distracting elements, connections are readily made
• Navigation (e.g. clear - intuitive, allows users to select their own pathway(s), fully hyperlinked).

The following general checklist for the criteria areas that help to define presentational e-Portfolios based on E-Portfolio Assessment Criteria, proposed by Penn State (Penn State, 2006 http://psu.edu ).

Operational (e-Portfolio functions well). Indicators:

• navigation is clear and consistent
• all links work
• media displays as intended
• all programming is appropriate (not too limited or too flashy)
• spelling and grammar are correct
• published materials respect copyright laws

Appearance (e-Portfolio looks well). Indicators:

• appearance and navigation are clear and consistent
• images are optimized for the web
• text is readable (fonts, sizes, and contrast)

Evidence (academic, co-curricular and personal evidence). Indicators:

• organizational scheme connects all evidence into an integrated whole
• features or showcases a specific piece of evidence
• shows depth in major and related experience
• shows breadth of knowledge and experience
• includes a resume (one page, printer friendly)

Reflection (personal message is integrated into the e-Portfolio). Indicators:

• audience and purpose of e-portfolio is described or is obvious
• addresses the career and own personal development
• reflective comments about evidence as well as reflective comments about what this evidence says about the student is integrated into the e-portfolio
• includes short-term goals (skills learner needs to add/improve)
• includes long-term goals (professional and/or personal aims)
• interpretation of learner's own achievements is expressed