Write, edit or translate questions


How to use this procedure

To use "Edit questions" procedure, there are several steps to follow.
First, change Language, Field, Course, Chapter and/or Section, if the course page shown is not the one you want.
Second, either
     - select a question and click on "Obtain", or
    - click on "Start new question" after selecting the anchor and type of question.
Third, write, edit or translate the text in the TEXTAREA keeping in mind the recommendations indicated below; verify that the answer shown in the "Answer" box is correct and that the anchor and concept match.
Fourth, save the question. Keep a copy of your work on your computer.
(Note that if you are translating, Language must be the language into which you are translating - not English - although the text of the question itself might be initially in English.)


Layout of the screen

The full screen of "Edit questions" procedure is divided in several parts:
- the upper yellow shaded part identifies the material selected and allows to move to another selection;

        - buttons are present in this part to show the corresponding question in English (if the text is a translation of an English question) or to show the entire questions file instead of the text of the chapter;
        - to see what resources can be used in questions, checkmarks are provide for List audio, case, images, tables, pdf or othe documents;
- next yellow shadded box is the editing of the selected question, once a question was selected:
  • - this part also offers the possibility to change the question parameters (explained below) "Type", "Graph", "Text", "Aswer", "Anchor" and "Concept", which you do not need to change when translating or editing;
  • - right below it, is the TEXTAREA where appears the plain text of the question selected, and that is where all the writing, editing or translating work is to be performed;
  • - just below the TEXTAREA several buttons to Save or to perform different other steps;

- below the yellow shaded area, a pink "Preview question" box presents the text of the questions, the resources (graph, table, audio or document) used in the question, the correct answer and a button linking a window with the concept where the questions is explained; the possible answers of multiple choice questions are reshuffled exactly as they will be in graded automated tests in which the question is used;
- further below the yellow shaded areas and/or "Preview question" box, if the "Show questions" button was clicked, a pink box presents the entire contents of the test questions file;
- even further below the yellow shaded area, a pink "Preview box" presents the text of the chapter to which the questions pertain, with the anchors appearing in red and a highlight on the anchor for the question shown; note that the text appears as it will for students in their browsers, except that the images are missing and the links do not function;
- a "Work status" schedule shows the ID numbers of those who worked on the different questions files for that course;
a colour scheme in the schedule (see below) indicates the status of all the sections in the course:
- at the very bottom of the screen, these explanations are provided;
- in between these parts, various small messages may appear to indicate the actual name of the file saved or retrieved, the name of the person to whom the corresponding task is assigned or who last worked on the material selected, and a list of anchors not matching in the text (if any), as well as the resources that were selected to be listed.


Test question type, structure and parameters

When writing new test questions, the following few guidelines should be followed.
Keep in mind that PEOI's test questions are conceived to offer automated tests that cover all the material in a chapter with a diverse content and format to be an accurate assessment of student knowledge of a chapter. Automated tests must also be a learning experience with an immediate explanation of a question incorrectly answered by showing the underlying concept and/or the paragraph in a course page on which the question is based. It is recommended that question authors experience taking PEOI's automated testing in subjects such as accounting, economics or ESL.


Type 0: Multiple choice
For multiple choice questions, the wording of the question is typed in the first lines in the TEXTAREA, then each possible answer must be typed in one subsequent line and be preceded by a capital letter (A, B, C, D, E or F) followed by a dash "-" and a space,. Most multiple choice questions have four possible answers, but can have as many as six. It is not necessary to set the correct answer other than "A- ", the first possible answer, because test questions are rearranged each time they are used. When "Start new question" is clicked, the format of the new question will appear in the TEXTAREA with lines for four possible answers.
Parameters of multiple choice question:
  • parameter for "Type" box must contain 0;
  • "Graph" usually has 0, except if the question is base on a graph, image or audio;
  • parameter "Text" is usually 0, except if the layout of the question is other than plain text;
  • parameter "Answer" must contain the capital letter corresponding to the correct answer;
  • parameter "Anchor" must have the anchor label;
  • parameter "Concpet" should have the concept which corresponds and is extracted at the location of the anchor, but only if concepts have been extracted from the course page and placed in a concepts data bank and in the "Concepts" section for the chapter.

Audio multiple choice question: such question must have the name of the audio file from the course "Resources" folder to be entered in the "Graph" box, and the "Text" box have the word "audio".
Image multiple choice question: such question must have the name of the image file from the course "Resources" folder to be entered in the "Graph" box, and the "Text" box have the word "image".


Type 1: True/False

True or false questions are processed as multiple choice questions, but for consistency sake it is recommended that they start with True/false and the possible answers be:

A- true
B- false
C- no valid answer
True or false questions must have 1 in parameter "Type" box and the letter corresponding to the correct answer in the "Answer" box. When clicking "Sart new question" for a Tru/False type question you have the choice of creating a question with the answer being "B- False" rather the usual A as correct answer.

Type 2: Fill-in

Fill-in questions can be written with a missing word marked by underlined spaces "_" (with a suggestion of using as many spaces as there are letters in the missing correct word or words), or it can be written in the form of a question without any actual space to fill. In both cases the student will have to type in his/her answer in a separate box. For fill-in questions, the author should enter 2 in the "Type" box and the correct answer in the "Answer" parameter box. The other parameters are similar to those in multiple choice questions. It is possible to allow more than one correct answer, which then should tyed in the "Answer" box and be separated by commas with no space after the commas; each answer itself can nevertheless contain spaces. (Note that the test procedure will count answers as correct whether written in singular or plural, in capital letters or not, with or without currency sign, but not with articles preceding them.)


Type 3: Calculation

Calculation questions are processed as either multiple choice (with A-, B-, C- or D- answers) or fill-in depending on how they are written. If they are written as multiple choice, the "Answer" parameter box must contain the capital letter corresponding to the correct answer. If they are written as fill-in, "Answer" must be a numerical value of the correct answer. Note that if the answer is a number that exceeds 1,000 it should be written without commas (e.g. $2,432,611 should be written as 2432611). Currency signs (such as $) are acceptable, but generally not necessary. The "Type" box must contain "3".


Type 4: Graph

Graph questions can give the choice of up to four graphs or images to be offered as possible answers. Following the text of the question, the possible answers contain nothing but the capital letters followed by a dash "-" (i.e. A-, B-, C- or D-). As for multiple choice and true or false questions, the answer box must have the capital letter of the correct answer. The graph box must contain the name of the graph or image files names separated by a commas with no space after the commas. The "Type" box must have the digit 4. (Note that multiple choice, true or false and fill-in questions may be based on a graph, image, audio, video or portion of text which appears in a window in the upper left corner of the screen when a question is asked, but in this case these questions are not graph questions.)


Type 5: Essay question

Essay questions contain text only and nothing in the answer box. The "Type" is 5. These questions are not answered by students in a quiz. They can be present for review purpose.


Additions to text of quesions

In the text area of any of the questions, in addition to the text of the questions itself, short explanations can be entered by an author below the question itself and the possible answers, if any. If the explanations are to appear only after student's answer, the first character on the first line of the explanation (i.e. below the text and answers) should be a close square bracket "]". Then, the explanation will appear in a special window at the bottom of a student's screen just below the title of the concept only after the student answers the question. Such explanation can be most appropriate for giving the solution to a calculation question. If, on the contrary, the explanation are necessary to understand the question itself (such as in the case of definitions of variables in an equation, for instance), then the explanation need not have the bracket. Note that in that second case, the explanation will appear to the student in the box together with the question and its lay out is not HTML but exactly as written.

To assist in writing test questions and especially in entering the correct anchor, a list of anchors is shown and the entire text of the chapter, section or subsection appears below the question editing box. The anchor is highlighted in the text with bold and underlined fonts.


Recommendations on how to write questions

As previously indicated, all the work to be performed is in the TEXTAREA. It is advisable not to erase or damage the structure of the question. It is possible to copy and paste any portion of the question in the TEXTAREA. But the material pasted must be in text format with line breaks. Pasting material copied directly from a word processor in another format than plain text is likely to result in a corrupted file and an error message.

These questions are test questions which are retrieved randomly when a student takes a test. They differ significantly from the practice questions in their format (although, naturally, they cover the same material and may even contain identical wording). Whereas practice question are plain HTML pages, test questions files are text files with very rigid parameters. The reason for the rigid structure is that each multiple choice question is restructured each time it is used. Also, for fill-in type questions and for calculation questions, this permits more than one possible answer whenever that is necessary. Finally, there are graphic questions that are also rearranged so that no test looks the same as another, and no student will almost never take an identical test twice. Because of the rigid structure of the questions file, care must be used to keep lines in the multiple choice questions in the order and format they first appear. Especially, it is of utmost importance to have a line break at the end of each line.


What to do

Once a question is fully written and its parameters are correctly entered, the question is then saved in a temporary question file. The question must then be reviewed and evaluated by another volunteer. Once verified by the project team leader, the temporary file is moved to its permanent folder. The "Edit questions" procedure is not the only procedure to create PEOI's test questions. Test questions can also be seeded from review questions, and review questions are themselves converted concepts which have been extracted from course pages. The process creates data banks of questions and data banks of anwers.

Clicking on the "Notify work completed" button sends out an automated message that alerts the webmaster and the coordinator who will decide if the questions need proofreading and editing or be moved to permanent folder. This also changes the color code for the page in the schedule of work done from grey to blue.


What file is saved and retrieved

When a page is retrieved, it is initially the latest temporary file. If a different version of the page is desired, it can be selected in the "temporary file" pull-down menu and the "Get" button should be clicked. If there is no temporary file, the only available file for the page is the permanent file. When the "Save" button is clicked, the page in the TEXTAREA is always saved in a temporary file which bears the numeral of the latest temporary file plus one.

Warning: Please note that the procedure is believed to contain at least one bug. Some users have reported that a portion or the entire file can be deleted occasionally. This has occurred when a totally empty question appears on the screen, and the user clicks on "Save". Normally, no empty question should ever appear. If it does appear, please, report that to the webmaster immediately indicating with as much detail as possible what steps you took that led to the appearance of an empty question. It is believed that the problem was caused by a missing parameter, such as "Answer", and the problem seems to have been fixed. In any case, report any problem you come across right away.


Proofreading evaluation

To do proofreading, you must first select the task by clicking on "Choose task #...e" where the letter "e" indicates that it is an evaluation task. This selection is necessary to record the name of the person who will do the proofreading, and is only possible if test questions have previously been saved and placed for editing (when someone else has clicked on "Notify work completed").

When the screen comes back a new set of buttons will appear below the white TEXTAREA. Next to the "File proofreading evaluation", four menues have an initial default value of 7 (7 is a passing grade) for the following four criteria
- Reading ease of questions;
- Accuracy of theory, terminology and math as it relates to underlying text;
- Writing quality (grammar, spelling, punctuation),
- Layout (images, audio, video, tables, paragraphs),
Keep in mind that the evaluation of test questions will also serve to assess the quality of the work of authors and/or translators who worked on the page. If you give an evaluation of less than 7 for any criterion, you must enter a short statement of the nature of the problem in the "Problem" item. Enter your evaluation, and click on the "File proofreading evaluation" button. If you make any changes in test questions and click on "Save", you will not be able to do the proofreading.


Color code in "Work status schedule"

In the Work status schedule, the background color of a section is

- white : there is no page of that type in this course;
- light blue : page planned, but not set up;
- light grey or light yellow: the page exists, but no task has been set up for this page;
- green : a task exists for this page, but no volunteer has chosen this page as a task;
- pink : task assigned or page not available;
- lilac : page awaiting translation;
- grey : a volunteer has added, corrected or translated material in the page;
- olive : page finished except for proofreading;
- lime : a volunteer has clicked on the button to "Notify work completed" but page has not been moved to its permanent location for students to see;
- purple : the page has been edited after completion and/or transfer to permanent folder;
- light green : all work on this page is completed and the page has been moved to the permanent folder;
(note that records may not be up-to-date for all the work done, especially for materials inserted prior to the installation of the task monitoring system);

The ID number of the volunteer is shown in a yellow button if he/she is registered as faculty, orange button if he/she is registered as staff, and purple button if it a board member.

The same color code applies for the status of courses, fields (determined most advanced course in field) and tasks: thus

- light blue : potential course;
- pink : course listed but not available;
- lilac : course awaiting translation;
- grey : course under construction;
- light green : course completed and ready for student enrolment.

Selection of task

If a task exists for the currently opened section page, this is indicated in a line just above the yellow shaded box at the top of the screen. If the task has not been selected, a button will appear, that allows to select that task. For then on, the button will no longer appear, and the colour in the "Work schedule" will be pink rather than light green. The deadline is initially set to two weeks from the date selected. The deadline can be changed in "My tasts selection" or "My work".