CMPT 773 (Fall 2006): Grading criteria for draft research plans

Home    Schedule and downloadable readings    Assignments    Course outline    Email-list    Policies   

Revision history

Oct 19: Lingering references to "presentation methods" (last year's topic) removed. Dates added for draft version and final version.

Due date: Tuesday, Oct. 24 (draft), Thursday Oct. 26 (final)

Length: 8-12 double-spaced pages.

This is a group project.

Statement of the project

This project asks you to consider the many different ways to address a single aspect of working with computers. You have already selected a question to consider.

Define three research plans for answering this question. The plans should reflect a wide range of approaches, emphasizing the many facets of HCI. Each plan should have the following parts:

  1. A specific research question contained within your broad, overall question.
  2. A study protocol for answering that question: What kind of participants will you recruit, what will you ask them to do?
  3. A statement of the underlying assumptions of the study and the strengths and weaknesses they impose. What will they help you find? What will they hide from you?
  4. A list of proposed outcomes. What do you expect will happen?
  5. A description of the dependent measures. What will you record and how might you analyze it?
  6. Design outcomes: How might this information inform the design of computer systems?

Grading criteria

The following criteria will be used to grade the assignments. Each criterion is followed by descriptions of several levels of quality, from the highest quality to the lowest quality.

Differentiation and range of the three plans

3. (highest) Plans use 3 substantially different research approaches.

2. Plans use 2 or 3 approaches, but approaches have similar assumptions.

1. (lowest) Plans all use a single research approach and assumptions. For example, all measure time and errors to perform a specified task.

Quality of each research question

3. Question is precise, focused, and answerable with a realistic amount of effort.

2. Question addresses an issue within the broader question.

1. Question is not concerned with the field.

Appropriateness of each research approach

3. Research approach addresses the stated question directly, with key assumptions stated explicitly and minor ones left out.

2. Research approach addresses the stated question in a focused and direct way. However, underlying assumptions are unstated or general.

1. Research approach addresses the stated question in a broad, indirect, or vague way. Connection unclear between approach and question.

Data gathering

3. Dependent measures are realistic, clear, and specific to this question. Have a high chance of answering the research question.

2. Dependent measures are realistic and will measure the claimed construct.

1. Dependent measures are vaguely defined or generic (for example, time and errors only).

General clarity of proposal

3. Concise, yet complete. All key points are described, while small or redundant details are left out. Organization allows an experienced reader to find the key points quickly.

2. All points are included, but there is unnecessary detail and the organization does not guide the reader well.

1. Some key points are missing or organization is weak, radically different from standard scientific reporting.