Collaborative Planning for Electronic Portfolios: Asking Strategic Questions

In the March, 1994, issue of The Computing Teacher, I discussed the technology to support alternative assessment that was beginning to appear in the commercial marketplace as well as some practical strategies for implementation. That article began with a series of questions to keep in mind when considering the use of technology to support alternative assessment. This article expands on those questions that need to be asked (and answered) before decisions are made about a major implementation such as electronic portfolios or using technology to support observational assessment.

There are two broad categories of applications in the use of technology to support alternative assessment:
1. Programs to electronically record and store observations and/or anecdotal data about student learning, mostly by the teacher; and
2. Electronic portfolios, digitizing and storing collections of student portfolio artifacts, using a range of technologies and multimedia elements.

This article will focus more on the second category, electronic portfolios, since there seems to be a broad interest and more variability in that implementation than in the observational assessment software, which is limited to two commercial packages (Sunburst's Learner Profile and Aurbach's Grady Profile).

This article will also use a series of questions to begin to focus the discussion of the use of electronic portfolios with the various stakeholders in the assessment process, under the assumption that developing a shared understanding within a collaborative model will lead to a more useful, productive and successful assessment process.

Assessment systems must be judged based on the value of the information they provide for students, teachers, curriculum specialists, principals, school board members, parents, and community members. All these stakeholders make choices about students, programs, curriculum and instruction. They must be considered within the context of intended use (Baker, 1992 in Fenton, 1996).

Before addressing these different strategies for implementing electronic portfolios, a few general questions may be appropriate to form a context from which to make decisions about assessment in general.

What is assessment and evaluation?	Assessment is the collection of relevant information which may be relied upon for making decisions. Evaluation is the application of a standard and decision-making system to assessment data to produce judgments about the amount and adequacy of the learning which has taken place. (Fenton, 1996)
What is a portfolio?	Rick Stiggins (1994) defines a portfolio as a collection of student work assembled to demonstrate student achievement or improvement. The material to be collected and the story to be told can vary greatly as a function of the assessment context. Vicki SpandelÌs definition (NWREL): A purposeful collection of students' work that illustrates efforts, progress, and achievement. Stiggins (1994) also states, ".. portfolios are a means of communicating about student growth and development--not a form of assessment"(p.87).
How are portfolios usually stored (without a computer)?	Teachers and students have devised a number of strategies, from notebooks and file folders in file drawers, to pizza boxes and larger containers for more creative projects. Some teachers use photographs, audio tape and video tape to store evidence of student work.
NOTE:	(The information immediately below is provided by Skip Via, Fairbanks North Star Borough School District)
What are the advantages and disadvantages of traditional manila folder portfolios?	Advantages: Child-centered, minimum teacher time, easy storage and retrieval, user-friendly interface Disadvantages: Difficult to back up lack of portability, limited shelf life
What are the advantages and disadvantages of electronic portfolios stored as hypermedia stacks?	Advantages: Easy to back up, good portability, good shelf life Disadvantages: Teacher-centered, increased teacher time, difficult storage and retrieval, cross-platform compatibility
What are the attributes of an Intranet-based system for storing electronic portfolio information?	A new model: Child-centered, minimum teacher time, easy storage & retrieval, portable, cross-platform, automatically updated, accessible user interface. New model prerequisites: Distributed storage, ubiquitous high bandwidth access, common document formats, transparent authoring tools, integrated with other applications. Web-based documents: It's here (everywhere) now, accepted standard, distributed storage and processing, common document standard (HTML), platform independent

What are the elements to include in any portfolio (whether traditional or electronic)?

• Learner Goals
• Guidelines for selection of materials (to keep collection from growing haphazardly)
• Work samples, chosen by both student and teacher
• Teacher feedback
• Student self-reflection
• Clear/appropriate criteria for evaluating work (rubrics based on standards)
• Standards and Exemplars - examples of good work

Why use technology to store portfolios in multimedia format?

• To make work in many media accessible, portable, examinable, widely distributable
• To make performance replayable and reviewable; it is important to see more than once
• To address ownership issues of student-created work
• To address storage issues (source: Sheingold, 1992)

Developing a Decision Matrix: Questions to ask before making decisions about implementing electronic portfolios

School teachers and administrators need a decision matrix or a template that will help them decide which programs or strategies to use, based on the Human and Financial Resources available. I am often asked, "What is the 'best' portfolio program?" And my answer is always, "It depends!" (on the assessment context and a variety of other factors, human and technological, that exist in a classroom, school or district). Here are a few of the questions that need to be answered before a definitive response can be made; and an important part of a collaborative decision-making process is including the major stakeholders in answering the questions that directly affect them.

Resource Questions

• What is the stakeholders prior experience in using traditional portfolio-based assessment?

• What is the level of computer skills of the teachers?

• What is the access to computers by students?

• What is the level of technology competency of the student and independence in using a computer? (age-dependent)?

• What technology is already available in the classroom?
  --describe computer(s) including RAM & hard drive storage capacity - every 18 months, look for the minimum technology capability to double, and costs to decrease by half for the same power/capacity.

• Is networking available in building or classroom or district? Is there a server?

• How much budget do you have for additional hardware, software?

• How much budget do you have for staff development (time and cost) and support?

Portfolio Context Questions

Before building a portfolio, educators need to ask a few questions about the assessment and portfolio context.

• Will you take a Teacher-Centered or Student-Centered approach?
  • -determined by who is in charge of the portfolio collection and publication

• What is the purpose of the portfolio?
  The purpose of the portfolio and the varied audiences will determine much of the following context factors. These context factors relate to not only the purpose of the portfolio, but some other characteristics of the learner.
  • Various Assessment Purposes with my estimate of the primary audience (below are examples from a database of alternative assessment strategies published by CRESST (UCLAÌs National Center for Research on Evaluation, Standards, and Student Testing).
    • Diagnosis of Student Learning (teachers, parents)
    • Selection/Assignment to Groups (teachers)
    • Grading/Course Exam (teachers)
    • Proficiency Testing (teachers, administration)
    • Program or Curriculum Evaluation (teachers, administrators)
    • Research (administration)
    • School Accountability (administration)
    • School/Instructional Improvement (teachers, administration)
    • Promotion/Certification (student, parent, community, i.e., college, employer)
  • Assumption: Different ages and audiences = different portfolios and purposes = different formats for storage and publication.
    • Elementary/middle school students and their parents primarily want to see growth/progress over time. Schools often want to retain these files as part of the studentÌs permanent record until graduation, which requires a format that is compact and transferable between schools. School districts may also wish to use student portfolios for program assessment or to document progress in studentsÌ achieving standards, which may require a format with links to a centralized student database.
    • High school students often use their portfolios as exhibitions for graduation, college application, or future employment, which requires a format that is cross-platform and playable in many different contexts. Students can use their portfolios to demonstrate both academic achievement as well as personal characteristics.
    • College students often use their portfolios for graduation or professional certification, and often for employment applications, which requires a format that is cross-platform and playable in many different contexts as well as demonstrating achievement of professional standards.
• How will you store the portfolio?
  • Computer diskette
  • Paper
  • Compact Disc-Recordable
  • Video Tape
  • High density floppy (Zip disk)
  • Intranet (a network within a building or district) or password-protected server
• How will publish you the portfolio? (many of the same strategies shown above)
• How will you provide for security and confidentiality of student assessment information stored in electronic form?
• Do you want to use technology to help collect observational assessment data? If so, there are only two programs available commercially, Learner Profile and Grady Profile, and only Grady includes capability to store portfolio items.

Other Assessment Context Factors

• What is the age of student?
• What is the time frame to be covered in the portfolio?
• What are the nature of the outcomes to be assessed?
• What is the nature and focus of evidence being collected?
• What multimedia formats do you need to include to illustrate student efforts, progress, achievement?
• Do you want to correlate student performance to state or district Standards; that is, document achievement of specific standards by linking them to specific evidence (artifacts, exhibitions, or performances)?

What are the multimedia elements that could be included in an electronic portfolio?

The following multimedia elements are often included in electronic portfolios. These elements are illustrated below, with comments about the type of competencies needed to collect data using the digital medium

• Images - digitize graphics with a scanner or camera
  - Most people know how to use a copy machine, but may not know how to scan and display visual images on a computer. The importance of visual images is.....
• Sound - digitized with a microphone or camera
  - Most people know how to use a tape recorder, but may not know how to record sounds for storage and play back on a computer. The importance of sound is .....
• Video - digitized directly with a camera or video digitizer
  - Most people can handle the rudimentary aspects of a VCR or video camera, but may not know how to record video for storage and play back on a computer. There are also portfolio strategies for recording computer screens to video tape, especially for sharing with families. The importance of video +++
• Text - most often work completed by the student using a word processing program
  - Student portfolios often consist of work that is text-based. +++
• Mixed media products - students are beginning to produce multimedia projects, integrating graphics and text, and sometimes sound and video, using an authoring program, such as HyperStudio, KidPix, Macromedia Director, or creating WWW pages.

What are the support technologies needed to manage this digitization process?

• Authoring Software - a program to construct and organize portfolios or presentations
  - Most people know how to store work in paper files and folders, but do not know how to organized information electronically on a computer for easy storage and retrieval. (explained below)
• Hardware Add-ons - the equipment needed for each of these functions
  - Many people are learning to use desktop computers for personal and professional productivity, but may want to know which pieces of equipment could be added to a desktop computer to enable multimedia production for presentations and portfolios.
• Platform - The combination of operating system running on a specific computer: Multimedia can be implemented on the Macintosh OS, Windows 3.1, and Windows 95.

Comparing Multimedia Presentations and Electronic Portfolios

The difference between constructing multimedia presentations and creating electronic portfolios that contain multimedia elements lies in the assessment purpose and context. Many of the hands-on technology skills are the same.

Process for Constructing Multimedia Presentations

Process for Constructing Electronic Portfolios

decide on goals of presentation describe the audience decide on audience-appropriate content/sequence of presentation decide which tools is most appropriate for the presentation context gather multimedia materials to include in presentation organize in a sequence (or with hypermedia links) for the best presentation of the material conduct presentation for audience evaluate effectiveness of presentation

decide on goals of portfolio based on learner outcome goals that should be based on national/state/local standards with associated evaluation rubrics decide on and describe the assessment context (see above) decide on and describe the audience(s) for the portfolio (student, parent, college, community?) decide on content of portfolio items (determined by context) decide which software tools are most appropriate for the portfolio context decide which storage and presentation medium is most appropriate for the situation gather multimedia materials to include in the portfolio which represent learner's achievement (preferably linked to standards, preferably in a relational database) record student self-reflection on work and achievement of goals record teacher feedback on student work and achievement of goals organize with hypermedia links between goals, student work samples, rubrics, and assessment present portfolio to appropriate audience (by student, in age-appropriate situations) evaluate effectiveness of portfolio related to the purpose and assessment context

What are some multimedia tools to develop portfolios?

• It is important to choose software tools that allow teachers and students to create hypermedia links between goals, outcomes and the various student artifacts (products and projects) displayed in multimedia format that demonstrate their achievement. There are a number of generic types of software with examples shown of brand name products.
  • Relational Data Bases, such as FileMaker Pro 3.0 or Oracle
  • Hypermedia "card" formats, such as HyperStudio (Roger Wagner Publishing), HyperCard (Apple Computer), Digital Chisel (Pierian Springs), or SuperLink. There are commercial electronic portfolio templates available.
  • Multimedia authoring software, such as Macromedia Authorware, Apple Media Tool, Macromedia Director, Oracle Media Objects
  • Network-compatible hypermedia , such as HTML/WWWeb Pages, Adobe Acrobat (Portable Document Format)
  • Other tool software, such as KidPix, integrated works programs, especially the slide show features.
  • OpenDoc & CyberDog, AppleÌs new component software that allows links between many different types of applications within a single document.
• What are some commercial Portfolio Software?
  There are several commercial programs available that support electronic portfolios:
  • Grady Profile (Aurbach & Associates)
  • Scholastic Electronic Portfolio
  • Designer Software Electronic Portfolio Toolkit (a HyperStudio template)
  • Pierian Springs Digital Chisel with their Electronic Portfolio template
  • SuperSchool Electronic Portfolio
  • LearningQuest' Electronic Portfolio

Conclusions

There are many options available for implementing electronic portfolios. The best solution is dependent on an assessment of many factors that affect each of the stakeholders in the assessment process: teachers, students, parents, administrators, and the general public. Further research is being conducted to determine the best type of electronic strategy to use, based on the answers to the questions posed in this article. Preliminary results can be found on my electronic portfolio web site which can be found at: http://transition.alaska.edu/www/portfolios.html

References

• Baker, Eva L.(1992), Pamela R. Aschbacher, David Miemi, and Edynn Santo. CRESST Performance Assessment Models: Assessing Content Area Explanations. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing.
• Fenton, Ray (1996) "Performance Assessment System Development" in Alaska Educational Research Journal, Vol. 2 No. 1, pp. 13-22.
• Sheingold, Karen (1992) presentation at a conference on Technology & School Reform, Dallas, June, 1992
• Stiggins, Richard J. (1994) Student-Centered Classroom Assessment. New York: Merrill Publishing Co.

Copyright © 1997, Helen C. Barrett; All Rights Reserved