User:Mydogategodshat/temp
From Wikipedia, the free encyclopedia
September 16, 2003
Dear XXXXXXX:
The purpose of this letter is to request your participation as an Peer Reviewer for MERLOT (Multimedia Educational Resources for Learning and Online Teaching). The MERLOT project is an international initiative enabling faculty to integrate technology in higher education. Twenty systems and institutions of higher education as well as the National Science Foundation support MERLOT. Over 100 faculty from these institutions have been performing the peer review of instructional technology, modeled after the peer review processes for research and scholarship. The Business Discipline Editorial Review Board is now recruiting Peer Reviewers who are experts in their discipline and are recognized for their excellence in teaching
The MERLOT project is an online community of faculty and institutions collaborating to increase the quantity of high quality, web-based, interactive teaching and learning materials. A continually growing collection of high quality online teaching and learning materials is realized through the peer review process. You can obtain more details about the MERLOT project, the peer review process and evaluation standards at http://merlot.org.
The Business Discipline Editorial Board has established criteria and guidelines for Peer Reviewers at the following website:
http://www2.gasou.edu/cgb/MERLOT.htm (no longer active)
Also included here are sample completed peer reviews and the letter that you will receive once the review is completed. In order to assist you with the review, the first time you are assigned a module to review, you are paired up with one of our veteran reviewers. We estimate that a complete review will take three to six hours to complete.
Please review the materials at this website and let me know (cswift@gasou.edu) of your willingness to serve as a Peer Reviewer. In your response, please include evidence of your meeting the criteria for review, as well as the fields within which you would be comfortable reviewing.
Sincerely,
Cathy Owens Swift, Ph.D. MERLOT Business Discipline Co-Editor cswift@gasou.edu 912-681-5217
[edit] A Guide for Writing Peer Review Reports for MERLOT
Draft 2.0
The purpose of this document is to provide Editors a guide that they can distribute to peer reviewers to help their peer reviewers learn to write peer review reports that meet MERLOT standards. Editors will be providing peer reviewers discipline-specific guidelines as well. This guide will also be used in the development of the MERLOT online tutorial to train new peer reviewers. The guide and tutorial will be populated with discipline-specific examples in later versions.
Table of Contents
General Report 1 I. The Peer Review Report 1 The Description Fields 2 II. Overview 2 III. Learning Goal(s) 3 IV. Target Student Population 3 V. Prerequisite Knowledge 3 VI. Type of Material 4 VII. Recommended Use(s) 5 VIII. Technical Requirements 5 The Evaluations and Observations Fields 6 IX. Evaluations and Observations 6 X. Quality of Content -Strengths and Concerns 6 XI. Potential Effectiveness as a Teaching/Learning Tool - Strengths and Concerns 7 XII. Ease of Use- Strengths and Concerns 8 XIII. Star Ratings 8
General Report I. The Peer Review Report A. Peer Review Reports include seven fields that describe the teaching-learning material and six fields that evaluate the teaching-learning material. 1. Description a. Overview b. Learning Goal(s) c. Target Student Population d. Prerequisite Knowledge e. Type of Material f. Recommended Use(s) g. Technical Requirements 2. Evaluations and Observations a. Quality of Content i. Strengths ii. Concerns b. Potential Effectiveness as a Teaching/Learning Tool i. Strengths ii. Concerns c. Ease of Use i. Strengths ii. Concerns B. Guidelines: Keep the following guidelines in mind when you are writing your peer reviews. 1. Complete all fields. If a field does not apply, write "none." 2. Follow the guidelines for each field and include only the information that is asked for in each field. 3. Write to your primary audience - faculty searching for online materials they can successfully use in their courses. 4. Be comprehensive. 5. Always spell check and edit for grammar.
The Description Fields
II. Overview A. Description: This field contains a brief overview of the teaching-learning material. It highlights the subject matter, features, type of material, and learning goals. It provides a brief overall description that allows the user to decide if the material is worth looking into further. B. Guidelines: Keep the following guidelines in mind when you are writing the Overview. 1. Answer and include a description of each of the following: a. What is the material about? What subject matter does it teach? b. What types of learning components and features are included on the site? Features might include: i. Images/graphics ii. Video iii. Audio iv. Quizzes (with immediate feedback?) v. Other Interactivity vi. Links to related material vii. Learning assignments viii. Teacher’s guide ix. Glossary of terms c. What is the Type of Material? (e.g., Simulation; This should correspond to the Type of Material field. See part VI of this Guide.) d. What is the Learning Goal? (This should correspond to the Learning Goal Field. See part III of this Guide.) e. What is the Target Student Population? (This should correspond to the Target Student Population Field. See part IV of this Guide.) 2. Other descriptions of the material you may want to include in the Overview: a. Reference or credit to the author or host of the material. b. A statement describing the material as part of a series of sites or learning modules (e.g., "This is the first section of a three-section tutorial."). List the sites that follow and/or precede the material you are reviewing. c. Information about whether or not there are any costs involved in using the module. d. Additional websites by the author that might be used with the module. 3. Be descriptive only. Avoid evaluative statement of the material in the Overview. Evaluations should be limited to the Evaluations and Observations section of the report. III. Learning Goals A. Description: Describes the skills and knowledge the learner will acquire by using the learning material. B. Guidelines: Keep the following guidelines in mind when you are writing the Learning Goal(s). 1. A learning goal is knowledge or skills that the learner will gain or develop upon using the learning material. For example, a learning goal is a statement that the learner "will develop a deeper understanding ", or "will be able to accurately identify " A learning goal is not a statement that the learner will "practice", "explore", or "observe." 2. Describe the learning goal from the perspective of the student/learner. 3. Describe the learning goal using your best judgment. 4. If there is a learning goal stated on the site, or otherwise by the author, and you feel it is clearly stated and accurate, feel free to use it. Be sure to credit the author or site by stating "As it is stated by the author (or on the site), the learning goal is ‘ ’." Conversely, if there is a learning goal stated by the author and you do not believe it is correct, substitute your description. IV. Target Student Population A. Description: Description of the student population(s) who would use the learning materials B. Guidelines: Keep the following guidelines in mind when you are writing the Target Student Population. 1. It is important to express a STUDENT population, even if the learning material supports a wider range of people (from students, to faculty, to special interest groups). 2. Describe the discipline area (academic major, minor, program area) and their educational level (freshman, lower division, upper division, etc). V. Prerequisite Knowledge A. Description: Describes the knowledge or skills a learner needs prior to using the learning object. This may include proficiency of a subject matter or skill. B. Guidelines: Keep the following guidelines in mind when you are writing Prerequisite Knowledge. 1. Be as specific or general as necessary. 2. Describe the prerequisite academic concepts and skills. 3. Describe the computer and information competence skills.
VI. Type of Material A. Description: Gives the type of material using nine standard MERLOT categories. The choices are: 1. Simulation: Users participate in an approximation of a real or imaginary experience where their actions affect the outcome of the activity. Users must determine and input, on their own, the initial conditions of some dynamic scenario or set of circumstances that generate an output that is different from, and changed by, the initial conditions. A simulation involves an engine that drives the dynamics of the learning module in accordance with specific rules and that simulates real phenomena. 2. Animation: The dynamic and visual representation of concepts, models, processes, and/or phenomena that allows users to view, on their own, such processes in space or time. Users can control the pace of the visual presentation and can step backwards and forwards through the processes being viewed, but cannot determine and/or influence either initial conditions or outcomes/results of the visual presentation. 3. Tutorial: Sequentially organized information and activities with specific instructional objectives structured to integrate conceptual presentation, demonstration, practice, and testing to teach specific concepts or skills. Users navigate through electronic workbooks to study, practice, and be tested on information designed to meet stated learning objectives. 4. Drill and Practice: Activities that require users to respond repeatedly to questions or stimuli presented in a variety of sequences. These exercises allow users to practice on their own and at their own pace and to develop the ability to reliably perform and demonstrate knowledge and skills. 5. Quiz/Test: This can be any assessment device intended to serve as a test or quiz. 6. Lecture/Presentation: This category includes lecture and presentation support materials such as presentation graphics (e.g. PowerPoint slide shows), lecture notes, or audio-visual materials that are not intended to be used outside the presentation. 7. Case Study: Illustrates a concept or problem by using a real-life example that can be explored in depth. 8. Collection: A collection of subject-specific materials; for example, a collection of web sites, images, or applets. 9. Reference Material: Material similar to that found in the reference area of a library. This can include subject specific directories to other sites, texts, or general information. The material here has no specific instructional objectives. B. Guidelines: Keep the following guidelines in mind when you are writing Type of Material. 1. Choose the best Type. The Type of Material you select can be different from the Type selected by the original contributor of the material. VII. Recommended Use(s) A. Description: A description of how to use the site, or for what purpose the site can be used. B. Guidelines: Keep the following guidelines in mind when you are writing the Recommended Use(s). 1. How might the materials be used in a course. 2. Recommend concurrent materials or activities VIII. Technical Requirements A. Description: Describes the technical specifications (hardware, software, network) that are required to use the learning material. B. Guidelines: Keep the following guidelines in mind when you are writing the Technical Requirements. 1. Technical requirements include: a. Specific browser ("Best if viewed using Explorer") b. Specific browser versions ("Must use Explorer 5.0 or greater, or Netscape 4.70 or greater.") c. Specific operating system ("Windows XP/2000/98," or "Not available for the Mac") d. Specific display or other settings ("Display should be set at 800 x 600") e. Specific plug-ins or other software requirements ("Must have Shockwave. Download Available at site" Or "Need Adobe Acrobat Reader to view") f. Hardware requirements ("Need sound card and speakers to hear audio." Or "Need a printer to use assignments.") g. Specific network or Internet access speed ("Must connect with a 56K Modem or better.) h. Java and Java-Script must be enabled on browser. 2.
Technical requirements does not include: a. Technical skills of the user. b. Technical glitches or bugs. c. Technical usability issues, such as slow response time. 3. When technical requirements are listed on the site, state them in this field. 4. If specifics are not listed on site, use the following questions and guidelines to help you write the requirements: a. Is sound required? - Specify "Audio Capability required" b. Is it a Java Applet? -Specify "Java enabled browser" c. Requires disc space (to download features of the site)? -Specify how much space is required. d. Requires plug-in or software? -Be as specific as you can. (popular ones include: Shockwave, Windows Media Player, Real Player, Quicktime, Flash, Adobe Acrobat Reader)
The Evaluations and Observations Fields
Each MERLOT Editorial Board defines the evaluation standards it used for its discipline. The discipline-specific evaluation standards must fit within the general evaluation standards that apply across all Editorial Boards. The following guidelines provide these general evaluation standards. IX. Quality of Content A. Description: Describes two general elements about the content of the material-its validity and its significance. B. Guidelines: Keep the following questions in mind when writing the strengths and concerns for this evaluation standard. 1. Use the following questions to help you evaluate and provide reasoning for your evaluation of Quality of Content. a. Is the content valid/accurate/reliable? i. Does it accurately depict reality? ii. Is it valid compared to convention or other resources? iii.
Is the content complete in scope without missing important and relevant information? iv. Is the content current/up-to-date? b. Does the content teach important/valuable/educationally significant concepts, models, or skills in the discipline? i. Does the content cover core curriculum within the discipline? ii. Is the content a pre-requisite for understanding more advanced material in the discipline? iii. Does the content cover material that is difficult to teach/learn? c. Is the validity and significance consistent throughout the material? d. If the site contains links to other resources, are they appropriate and valid? 2. Write a paragraph with "bullets" for each of strengths and concerns. You MUST have something in the Concerns area unless you award a rating of 5. If there are no concerns for this standard, write "none" in the field. X. Potential Effectiveness as a Teaching/Learning Tool - Strengths and Concerns A. Description: Describes the material’s likely ability to improve teaching and learning given the ways faculty and students could use the material. B. Guidelines: Keep the following guidelines and questions in mind when writing the strengths and concerns for this evaluation standard. 1. Describe the context for using material. Defining the purpose of the learning material is critical to evaluating its potential effectiveness. You can describe if the materials can be used as (1) an explanation or description of the problem, (2) a demonstration or exploration of the problem, (3) a practice problem, or (4) an application to "new" problems. 2. Describe how using the materials enhances the quality of the teaching and learning process. a. Does the material facilitate learning? Will learners be able to effectively achieve the learning goals? b. Are concepts, models, or skills presented with clarity, focus, and organization? c. Compared to other methods of teaching the same concept, models, or skills, is this learning material just as effective or better? Is it an innovative, new, original presentation of the concept? d. Does it engage the learner, create intrigue, or otherwise motivate the learner to achieve? e. Does it appeal to multiple learning styles? Multiple learning processes? f. Does it engage multiple senses through audio, video, images, and text? g. If it is interactive, does it do so effectively? h. If it is interactive, does it provide immediate feedback regarding the learner’s response accuracy? i. Does it provide examples when appropriate that help illustrate concepts? j. Does it demonstrate relationships between concepts? k. Does it provide effective introductions, overviews and summaries when applicable? l. Does it have flexibility or versatility of use? 3. Write a paragraph with "bullets" for each of strengths and concerns. You MUST have something in the Concerns area unless you award a rating of 5. If there are no concerns for this standard, write "none" in the field. XI. Ease of Use- Strengths and Concerns A. Description: Describes how easy it is for students and faculty to interact with the learning material. B. Guidelines: Keep the following questions in mind when writing the strengths and concerns for this evaluation standard. 1. Describe the features of the material that makes it easy to use a. Is information presented in ways that are familiar to students? b. Is the material easy to navigate? c. If it is interactive, does it provide feedback for user actions? Will the user always know if they are waiting for a response from the system, or if the system is waiting a response from the user? d. Is it self-contained, or are instructions necessary? e. If there are instructions or "help", are they clear, relevant, and complete? Are they available when needed? f. If applicable, does it clearly tell users when an error is made, and how the user should continue? g. Is the presentation clearly designed with no distracting design elements (e.g., color, animation, too much on a page)? h. Are the terms/new jargon, defined? i. Are related parts of the site clearly related, while parts that offer different content areas, or audiences are clearly separated? j. When the site requires plug-ins, does it provide links to easily access the plug-in for downloading? k. Are there any major bugs (e.g., links that do not work) 2. Write a paragraph with "bullets" for each of strengths and concerns. You MUST have something in the Concerns area unless you award a rating of 5. If there are no concerns for this standard, write "none" in the field. XII. Star Ratings A. Description: Peer Reviewed materials are rated 1 - 5 stars, with 5 stars having the highest excellence, for each evaluation standard and overall. YOU CAN ASSIGN RATINGS WITH DECIMALS (eg. 4.3). The purpose for these ratings is not to take the place of text in the peer review report, but to act as search indicators. Users of MERLOT are able to search or sort materials by number of stars. The rating scheme is as follows: 5 Stars: The material is excellent all around. 4 Stars: The material is very good overall but there are a few minor concerns. 3 Stars: The material meets or exceeds standards, but there is some significant concerns. 2 Stars: The material does not meet minimal standards, but might have some limited value. 1 Star: The material is not worth using at all. B. Examples: Rating the three evaluation standards (Quality of Content, Potential Effectiveness, and Ease of Use) allow users to separate the different aspects of the learning material. Three combinations of ratings are provided below as examples: 1. Example 1: High Quality, High Potential Effectiveness, and Low Ease of Use. Possible interpretation: Good curriculum but will require extensive user support and training. 2. Example 2: Low Quality, Low Potential Effectiveness, and High Ease of Use. Possible interpretation: Bad curriculum but easy for a new users to get accustomed to using instructional technology. 3. Example 3: Medium Quality, Low Potential Teaching Effectiveness, and High Ease of Use. Possible interpretation: Materials are accurate and easy to navigate but require extensive background knowledge. In all three examples, people reading the reviews should be better informed about how they might (or might not) use the material in their classes. C. Guidelines: What constitutes excellence and minimal standards for each standard and the material overall will be determined by each discipline.
