CMPT 773 (Fall 2006): Review of an HCI paper (First draft)

Home Schedule and downloadable readings Assignments Course outline Email-list Policies

Revision history

Revision 1.1: Dates correcrted to reflect actual schedule. Some typographical errors fixed.

Due date: Tuesday, Sep. 26

In-class exercise: Thursday, Sep. 26.
Bring your draft review to this class. You will trade drafts with another student and comment on each other's work. You will then be able to use those comments to refine your draft.

Proportion of total class grade: 10%

Length: Four double-spaced pages.

This is an individual project.

Introduction

This assignment has several purposes. First, to get you started reading the HCI research literature. Second, to give those of you new to the graduate school practice thinking and writing at the level required for graduate study. Third, to give me a better idea of your writing skills and give you a better idea of what I expect in a written assignment. This description of the assignment may seem long (it's perhaps longer than the paper it describes). Don't be daunted. I'm taking this opportunity to present a structure we'll use throughout the course for thinking about research papers.

In this assignment, you will select one article from a list (given below) and discuss it in four double-spaced pages. Your paper is deliberately short. The main focus will be on clarity of analysis, particularly on locating and describing cause and effect relationships. To get you started, I'm suggesting some structure. I encourage you to modify this as you see fit. Use the following as a starting point for writing. Once you see your own structure emerging, follow that.

Choosing a topic

In this assignment, you will review an article from the HCI literature. I have selected articles from the ACM Transactions on Computer-Human Interaction (TOCHI), the leading journal in HCI:

McGuffin, M. J. & Balakrishnan, R. (2005). Fitts' law and expanding targets: Experimental studies and designs for user interfaces. TOCHI 12(4), pp. 388-422.
Svensson, M., Hook K., & Coster. R. (2005). Designing and evaluating Kalas: A social navigation system for food recipies. TOCHI 12(3), pp. 374-400.
Guimbretiere, F. Martin, A., & Winograd, T. (2005). Benefits of merging command selection and direct manipulation. TOCHI 12(3), pp. 460-476.
Ullmer, B., Ishii, H., & Jacob, R. J. K. (2005). Token+constraints systems for tangible interaction with digital information. TOCHI 12(1), pp. 81-118.
Hinckley, K., Pierce, J., Horvitz, E., & Sinclair, M. (2005). Foreground and background interaction with sensor-enhanced mobile devices. TOCHI 12(1), pp. 31-52.
Whittaker, S., Jones, Q., Nardi, B., Creech, M., Terveen, L., Isaacs, E., & Hainsworth, J. (2004). ContactMap: Organizing communication in a social desktop. TOCHI 11(4), pp. 445-471.
Feng, J., & Sears, A. (2004). Using confidence scores, to improve hands-free speech based navigation in continuous dictation systems. TOCHI 11(4), pp. 329-356.
Ravasio, P., Guttormsen-Schar, & Krueger, H. (2004). In pursuit of desktop evolution: User problems and practices with modern desktop systems. TOCHI 11(2), pp. 156-180.
Brotherton, J. A., & Abowd, G. D. (2004). Lessons learned from eClass: Assessing automated capture and access in the classroom. TOCHI 11(2), pp. 121-155.
Bederson, B. B., Clamage, A., Czerwinski, M. P., & Robertson, G. G. (2004). DateLens: A fisheye calendar interface for PDAs. TOCHI 11(1), pp. 90-119.
Tan, D. S., Gergle, D., Scupelli, P, & Pausch, R. (2006). Physically large displays improve performance on spatial tasks. TOCHI 13(1), pp. 71-99.
Benford, S., Crabtree, A., Flintham, M., Drozd, A. Anastasi, R., Paxton, M., Tandavanitj, N., Adams, M., & Row-Farr, J. (2006). Can you see me now? TOCHI 13(1), pp. 100-133.

Writing the review

A good starting point for your paper is the following structure:

Summarizing the paper

Who wrote it? Not just their names, but what institution are they affiliated with? Which department within that institution? What kind of work is that Department known for? Who were the junior researchers, who the senior? For the senior researcher, where did they do their Ph.D.? What kind of work are they known for? Whom did they study under? What kind of work is their Ph.D. supervisor known for?
You won't be able to put all these details in your article. You may not even be able to find all these details without an outlandish amount of work. Focus on the easily-obtained things and only report the important ones in your paper. Knowing the social context in which the work was done provides important cues for interpreting the paper.
What kind of argument is it? An alternative way of phrasing the same question is, What kind of a contribution does this paper try to make? A paper can aim for one or more of the following contributions:
- Presentation of new empirical data. The classic example would be a paper reporting results of a controlled experiment but this category also includes less-controlled forms such as quasi-experiments, surveys, and certain kinds of historical studies. The main focus of these papers is the results. The method by which the reesults were derived is described in just enough detail to provide a context for interpreting the results and demonstrating their validity.
- Presentation of a new method for gathering data. These papers often include a first set of data gathered using that method. These papers are distinguished from the first kind by a more detailed description of the method and an emphasis on justifying the method rather than simply using it.
- A synthesis of research results. These papers take several papers published earlier and organize their results into a coherent pattern. The synthesis can be narrative or it can be done using sophisticated statistical techniques such as meta-analysis.
- A critique of current research directions or methods. These papers will often have little data and may come close to being rants. The authors will critique currently-fashionable approaches and suggest alternatives.
- A proposal for a new research direction or programme. These papers are close to the critiques described in the above point but focus less on the problems with current methods (although they may describe those briefly) and more on the broad outlines of a new approach.
The above list is neither exhaustive nor exclusive. Some papers may not fit into any of those categories while others may sprawl across three or more. Again, use this list as a starting point.
Who is the audience for this paper? Specialists within one research area such as input device research? A general audience across a research field such as human-computer interaction? The entire discipline (computer scientists)? A multi-disciplinary audience (scientists doing active research)? A lay audience with little direct research experience? The choice of audience has a strong effect on what kinds of knowledge the authors can assume they share with their readers.
What problem are they addressing? Why do they say it is important?
What is their solution to that problem?
How do they support their claims that the solution will work?

In summary, what did they say?

Evaluating the paper

Now that you've summarized the paper, evaluate it:

Do you agree the problem exists? Do you agree it's important?
What does their argument ignore? What doesn't it say? What holes or gaps are there in the argument?
Do you believe their proposed solution will work?

In summary, were you convinced?

Writing the above analysis is easy enough. Fitting it into four pages is the hard part. Save the details up, as you'll get a few more pages in the final version of this paper, anyways. You can't answer every question in full detail. You'll have to focus on only the most important points.

Grading criteria

I will grade this paper on the quality of the analysis. I will also note significant grammatical problems but they will not affect the grade. As the term progresses, I will begin to incorporate grammatical issues into the grade. Note that I will not be concerned with small errors of wording or spelling (unless they pervade the paper). My concern is with constructions that make the meaning obscure, vague, or contradictory.

The assignment will be graded according to how well it meets the following broad goals:

Clarity---Is the main point clearly stated?
Detail---Are the points well-supported by detail?
Depth of theme---Does your paper make a deep point about human-computer interaction (within the obvious limits of four pages)?
Structure---Does the argument flow well and do the points build on one another?

Ultimately, the grade for a writing assignment is based upon a synthesis of all the criteria. Because of this, I do not provide explicit percentages for the above.

General tips

Newcomers to scholarly writing may benefit from the following tips:

Begin the paper with a brief statement of your theme. Then summarize the article in terms of your theme. This will probably require ignoring some important points made in the article. That is fine.
Always cite an article to support your case (even if only to acknowledge that the article contradicts you). You are perfectly welcome to summarize an article in terms of your needs.
Don't tell your reader what you're doing---just do it instead. For example, don't include phrases such as, "This is my introduction" or "Now I will summarize my points". It is not disruptive to use a few marker phrases, in particular introducing your conclusion with, "In summary", but be sparing in such uses.