Preparing Questions for Brownstone's Software Formats: Guidelines for Editing Questions with Word Processors This document was developed for people who are using word processing programs to edit questions which are destined to be used in test generation programs like Diploma and on-line products like Blackboard, EDU, egrade, GradeAssist, and WebCT. These guidelines spring from experience gained converting millions of word processed questions. If you follow these rules and guidelines and arrange your questions in a simple column (like the sample at the end of this document), your questions will migrate easily from your word processor files to test generation and on-line products. The Most Important Thing to Consider Students and instructors won't see your questions as they appear in your word processed files. Instead, they will view your questions on-line after they've been converted to computer databases. Consequently, it's far more important to consider how your questions will appear in web browsers than how they look in your word processor. First, browsers are finicky about fonts, graphic images, tables, and information shared by questions (such as case studies). The issues associated with these items are discussed in the coming pages. Pay attention to how they're treated in your word processing files. On the other hand, you needn't fret about page layout issues. The less page layout work you do in your word processed files, the better. For example, we recommend that you do not apply special formatting such as hanging indents or leading space to the paragraphs of your questions. Likewise, we recommend that you don't insert page breaks or alter your page margins while working with questions in your word processor. Three Simple Rules Use the Simple, Indicated Tags--The tags that you'll apply to your questions will help distinguish their components when the questions are imported into a database. As you'll see in the examples on the following pages, the tags are simple and English-like. Isolate Questions with Three Blank Lines--Put at least three blank lines around each and every question. Like the tags just mentioned, this will help isolate your questions so that they can be placed into a database properly. Save Question Files in the Rich Text Format--You may use any popular word processor to write and edit your question files, so long as they are ultimately saved in a Rich Text Format (RTF). Most popular word processing programs can save documents in the RTF format. Copyright 2002, Brownstone Research Group, Inc. Page 1 of 10
Question Format Tags At the beginning of a series of questions of the same type, indicate the type of questions by using a question format tag. The following question format tags can be used: Multiple Choice, True/False, Short Answer, Fill-in-the-Blank, and Essay. The tag should be on a line by itself, separated from questions by at least three blank lines. (The question format tags in the example below have been highlighted and offset for emphasis. The tags you use in your word processing files do not have to be bold or offset.) True/False 1. True or False? The five rings in the Olympic flag represent the five continents: Africa, America, Asia, Australia, and Europe. Answer: True 2. True or False? During the opening ceremonies of the Olympic games, a single flagbearer from the previous Olympic games carries the flag in front of the Olympic flame. Answer: False Multiple Choice 3. What happens to the Olympic flame between the Olympic games? A) It's extinguished. B) It's preserved in Greece on Mount Olympus. C) It's preserved in Athens, Greece. D) It's preserved in the city where the previous Olympic games were held. Answer: A Copyright 2002, Brownstone Research Group, Inc. Page 2 of 10
Basic Question Tags The beginning of each question is indicated by a number followed by a period. Correct answers are indicated with a simple "Answer:" tag. Multiple choice distracters can be written "A) B) C) D)" or "a) b) c) d)" or "A. B. C. D." as long as their style remains consistent throughout the file. Up to ten distracters may be written for each multiple choice question. (The tags in the example below have been highlighted for emphasis. The tags you use in your word processing files do not have to be bold.) 1. What do the eight bearers of the Olympic flag represent? A) The seven continents and the one planet B) The five Olympic ideals and the three aspects of time C) The eight original Olympic sports D) The five continents and the three Olympic ideals Answer: D Topic, Difficulty, and Similar Tags Optional information such as topics and page references can be associated with questions in the same manner as the correct answer. This is done by typing the name of the information, followed by a colon and the information's value. The information tagged to questions in this manner must be text, but the information can be of any type: author names, dates, page references, etc. Be consistent when typing this information. For example, use "Page:" to indicate page numbers regardless of whether one page or many pages are associated with the question. (The tags in the example below have been highlighted for emphasis. The tags you use in your word processing files do not have to be bold.) 1. How is the Olympic flame ignited? A) It's ignited by sunlight. B) It's ignited using a flame preserved in Greece on Mount Olympus. C) It's ignited using a flame preserved in Athens, Greece. D) It's ignited with a flame from the previous Olympic games. Answer: A Topic: Olympic Flame Difficulty: Moderate Page: 53-57, 60 Copyright 2002, Brownstone Research Group, Inc. Page 3 of 10
Feedback Tags Optionally, feedback--the information that you might share with students to explain the correct answer after a test has been graded--can be associated with questions. If needed, feedback can be several paragraphs in length (which can be useful when explaining the steps in a complicated proof). (The feedback tag in the example below has been highlighted for emphasis. The tags you use in your word processing files do not have to be bold.) 1. What happens to the Olympic flame between the Olympic games? A) It's extinguished. B) It's preserved in Greece on Mount Olympus. C) It's preserved in Athens, Greece. D) It's preserved in the city where the previous Olympic games were held. Answer: A Feedback: There is no continuous Olympic flame. The Olympic flame is extinguished during the closing ceremonies of each Olympic games. A new flame is ignited by the sun's rays on Mount Olympus in Greece prior to every Olympic games. Algorithms and Variable Tags A word processor is not a good environment in which to develop algorithmically driven questions. However, if you do not have access to Brownstone's Diploma tool, you can tag your questions with your word processor in such a way that someone using Diploma can later edit the questions and enable their algorithms. In the text of your questions, mark variable locations with brackets, then put the definitions for the variables on individual lines behind a "Variables:" tag as indicated below. (The tags in the example have been highlighted for emphasis. The tags you use in your word processing files do not have to be bold.) 1. Assume that an old Olympic record in a distance event was [X] minutes and [Y] seconds. If the record is broken with a time of [A] minutes and [B] seconds, by how many seconds was the old record broken? Answer: [S] seconds Variables: X = 50 to 54 Y = 2 to 59 A = 47 to 49 B = 2 to 59 S = (X - A) * 60 + (Y - B) Copyright 2002, Brownstone Research Group, Inc. Page 4 of 10
Fonts, Symbols, and Other Character Attributes The text of your questions, answers, and feedback can incorporate bold, italic, underlined, superscripted, and subscripted characters. We highly recommend that you limit the fonts you use to Times, Arial, Courier, and Symbol. Most of the foreign language characters and math symbols that you'll need can be found in these fonts. Important: Other fonts might make your question look better in your word processor, but remember: students aren't going to see your questions in word processors. They're going to see your questions on-line. Therefore, it's important to use only fonts that their browsers are likely to have. In particular, avoid fonts that are unique to word processors, such as Word Perfect's WPTypographical font. If you use such specialized fonts, students might not be able to read your questions on-line. 1. Who was the founder of the modern Olympic games? Édouard Cézanne or Pierre de Coubertin? Answer: Pierre de Coubertin Tables Browsers handle consecutive space characters poorly, therefore do not use spaces to align or "bump" text into position in tables. When such tables are displayed to students on-line, their contents will often be misaligned. For best results, use your word processor's table tools to create tables. If table tools are not available in your word processor, use tabs. (Place one and only one tab between the text in each column. Use your word processor's tab settings feature to set the tabs' locations. Use the tabs' left, right, and center settings to define how the tabs should align text within their columns.) Important: Students will see your tables in web browsers, not in your word processing program. If a table is particularly wide, students will have to scroll left and right to see it. To avoid that, we recommend that you use small font sizes and narrow columns when working with large tables. (The cell borders in the table in the example below have been left visible for emphasis. The cell borders on the tables in your word processing files do not have to be visible.) 1. According to the following table, when were the Olympic games hosted by Atlanta? Answer: 1996 Year Host Country Host City 2000 Australia Sydney 1996 United States Atlanta 1992 Spain Barcelona 1988 South Korea Seoul 1984 United States Los Angeles Copyright 2002, Brownstone Research Group, Inc. Page 5 of 10
Graphic Images Images can be placed anywhere in the text of a question, answers, or feedback. There is no limit to the number of images that can be embedded into any one question. Important: For the benefit of students taking on-line tests, we recommend that images be small, crisp and easy to read. When images are large, students will have to scroll to see them. Try to confine large images to a width of 400 pixels. Also, use only the depth of color needed to capture the artist's work. If the image is two-color line art, save it as a two-color, black and white image. Saving two-color art at color depths of 16 million colors wastes enormous space. 1. What flag is depicted below? A) The Antarctic Expedition Flag B) The Flag of Athens C) The Olympic Flag D) The Flag of the Games Answer: C 2. Which of the following Olympic mascots was used during the summer games of 1988? A) B) C) Answer: B Copyright 2002, Brownstone Research Group, Inc. Page 6 of 10
Case Studies and Other Shared Information When several questions refer to the same figure, table, or case study, the shared information can be associated with the questions using "Reference:" and "Refer to:" tags. Note that the "Reference:" tags appear immediately before the shared information. Important: When developing shared information, it is important that you do not use specific question numbers in their text. For example, when writing a case study that will be shared by several questions, don't write "Use this information to answer questions 45-48 below." Instead, add text to each question that refers the student to the information above. This is important because when students see your questions in a test, the questions will probably not be numbered as you wrote them. (The tags in the example below have been highlighted for emphasis. The tags you use in your word processing files do not have to be bold.) Reference: Case Study 1 Assume that Country A sends 100 athletes to the track and field events in the Olympic games, and that Country B sends 50 athletes to the track and field events. Furthermore, assume that the athletes all possess the same levels of training, experience, and skill. 1. Read the case study above. Statistically speaking, which country is more likely to win track and field events? Country A or Country B? Answer: Country A Refer to: Case Study 1 2. Read the case study above. What is the likely ratio of medals that the athletes from Country A might win relative to the athletes from Country B? Answer: 2 to 1 Refer to: Case Study 1 Reference: Table 1 The Summer Olympic Games Since 1984 Year Host Country Host City 2000 Australia Sydney 1996 United States Atlanta 1992 Spain Barcelona 1988 South Korea Seoul 1984 United States Los Angeles 3. According to the table above, when were the Olympic games in Seoul held? Answer: 1988 Refer to: Table 1 Copyright 2002, Brownstone Research Group, Inc. Page 7 of 10
Sample Set of Formatted Questions The following set of questions has been formatted using the suggestions found in this document. Without further modification, these questions could be successfully imported into a database. (Some of the tags in this sample set of questions have been offset for emphasis. The tags you use in your word processing files do not have to be offset.) Multiple Choice 1. What flag is depicted below? A) The Antarctic Expedition Flag B) The Flag of Athens C) The Olympic Flag D) The Flag of the Games Answer: C 2. What happens to the Olympic flame between the Olympic games? A) It's extinguished. B) It's preserved in Greece on Mount Olympus. C) It's preserved in Athens, Greece. D) It's preserved in the city where the previous Olympic games were held. Answer: A Reference: Table of Host Cities Use the information below to answer the following questions. The Summer Olympic Games Since 1984 Year Host Country Host City 2000 Australia Sydney 1996 United States Atlanta 1992 Spain Barcelona 1988 South Korea Seoul 1984 United States Los Angeles Copyright 2002, Brownstone Research Group, Inc. Page 8 of 10
3. When were the Olympic games in Seoul held? A) 2000 B) 1996 C) 1992 D) 1988 E) 1984 Answer: D Topic: Olympic Hosts Refer to: Table of Host Cities 4. How many times have the Olympic games been hosted by the United States since 1984? A) Zero times B) One time C) Two times D) Three times Answer: C Topic: Olympic Hosts Refer to: Table of Host Cities True/False 5. True or False? The five rings in the Olympic flag represent the five continents: Africa, America, Asia, Australia, and Europe. Answer: True 6. True or False? During the opening ceremonies of the Olympic games, a single flagbearer from the previous Olympic games carries the flag in front of the Olympic flame. Answer: False Short Answer 7. How many flag bearers carry the Olympic flag during the opening and closing ceremonies of the Olympic games? Answer: Eight 8. Where is the Olympic flame ignited? Answer: Mount Olympus Topic: Olympic Flame Fill-in-the-Blank 9. There are colors on the Olympic flag. Copyright 2002, Brownstone Research Group, Inc. Page 9 of 10
Answer: six 10. The Olympic flame is ignited by on Mount Olympus before it begins its journey to the site of the Olympic games. Answer: sunlight Topic: Olympic Flame Essay 11. During opening and closing ceremonies, what do each of the flag bearers of the Olympic flag represent? Answer: Eight people carry the Olympic flag during opening and closing ceremonies. Five people represent the five continents: Africa, America, Asia, Australia, and Europe. Three people represent the three Olympic ideals: sport, environment, and culture. 12. What do the rings on the Olympic flag represent? What is the significance of their color? Answer: The rings represent the five continents: Africa, America, Asia, Australia, and Europe. The colors do not correspond to the continents however. The ring colors were carefully chosen so that each nation on Earth would have at least one color of their national flag represented by a ring or by the white field upon which the rings are depicted. Copyright 2002, Brownstone Research Group, Inc. Page 10 of 10