This page is for workshop-related usages.

Creating Data Comics for Reporting Controlled User Studies

Data Comics for reporting controlled user studies aim to provide easy access to

hypotheses, conditions, and decisions involved in a study
the setup, steps, and procedure,
the results

To that end, data comics draw inspiration from traditional comics such as

a combination of text + picture, i.e., any picture can be accompanied by text to explain additional information about the picture or provide information to resolve ambiguity in the picture.
a sequence of panels, whereby each panel can focus on a single message, i.e., a single piece of information, while the sequence helps to explain more complex information

Especially for data comics, that means that you can

illustrate in pictures what requires visual explanation,
explain through text what is hard to convey through pictures
mix data visualizations (e.g., charts) with schematic representations (e.g., flow-charts), and other illustrations (e.g., study setup)
annotate figures, especially information about your data visualizations
be consistent with your visual choices, e.g., reuse colors and charts as much as possible throughout the comic.
use a variable number of panels: sometimes a single panel is enough, sometimes you may decide on several panels.
experiment with panel size: some panels might be large and contain a lot of information, sometimes a panel might be fairly small with a picture only.

Purpose of This Collaborative Design Session

The purpose of this workshop is

For you: to help you create compelling data comics for your study to be included in your website, a poster, or even a paper.
For us: to understand which concepts you need to illustrate, which visual solutions you prefer, and how we can help you create your comics. This will help us creating more complete guidelines for data comics in HCI.

The collaborate design is structured as follows:

We introduce data comics, including an example,
explain the background of our research,
you prepare a storyboard - i.e., a rough outline - for your own data comic, using the below stages and guidelines. That storyboard, should contain information about

what to show and in which order
textual annotations
visual explanations to explain your study

Once you are happy with your storyboard, you contact us, and we will go with you through your storyboard to discuss any difficulties you found. Together, we will try to find solutions to any issues.
We return you a draft for a polished version, designed by one of our graphic designers/illustrators
you provide feedback for the last polishing.

More information about the study and the ethics procedure can be found here. For any questions, please contact Zezhong Wang.

Report Stages

This section guides you through creating a data comic for your own study.

Keep in mind the following:

you may not need to illustrate each stage in your study. The below list is extensive and designed to capture as many parts of a study as possible. You should chose the stages that are most important and benefit your study most to illustrate.
the list gives examples of what to report. The list may be incomplete, and we are interested in which stages and information your study requires to show.

We do not give visual examples to not bias your decisions. We are interested in which visual solutions you find to report your study. There is no right or wrong as we want to work and discuss with you to find the best possible representation for your study.

1. Context

The objective of this stage is to introduce the research problem to your audience and to provide context and background knowledge to help the audience to understand your research. This includes

The problem you’re studying
the domain
important terms and concepts, and
Any other background information you consider necessary

The context might be best introduced with conventional and schematic drawings, rather than data visualizations.

2. Conditions

Explain the conditions you are testing in your study. This should include

any baseline technique(s),
features to be evaluated,
materials or devices involved in the study.

Similar to Context and Motivation, conditions could be illustrated with conventional drawings such as schematic representations.

3. Hypotheses

Explain the hypotheses of your study.

For predicting effect sizes and quantitative results, for example, you could use the same data visualizations that you would use to report your results. It is important to highlight the specific differences or values you predict in your hypotheses.

You may decide to present the hypotheses before explaining the study conditions if that works better.

4. Tasks

Tasks are designed activities completed by participants in the study to measure performance under different conditions. Illustrating tasks may include

What do participants see or work with during each task?
What do participants have to do in each task, e.g., interactions or other actions?
Which steps are they going through?

5. Stimuli & Materials

Stimuli are selected or generated examples to be used in the study. The stimuli stage should contain

representative examples of stimuli,
parameters and factors that influenced the eventual generation of stimuli (e.g., size, difficulty, …)
generation methods, if appropriate.

6. Participans

Statistical power can be calculated and reported for an experiment to comment on the confidence one might have in the conclusions drawn from the results of the study.

A priori power analysis can be used as a tool to estimate the number of participants required in order to detect an effect in an experiment with a certain probability. The power analysis stage should contain an a priori estimate of the study’s power and the assumptions that were used in the power analysis calculation.

Power analysis is an important step in experimental design, but many studies in HCI currently do not make use of it. If that is the case, skip this stage.

7. Study Setup

The study setup gives an overview of the study process and the logic that went into the study design. It can include

an explanation of the study environment (e.g., in lab, in-the-wild, crowd-sourced, specific devices, interviews, …)
participants and specific selection criteria
the sequence of steps involved in the setup (e.g., instructions, training, repetitions, questionnaires, …)
important information about each state (e.g., length of each stage); sample and task assignment (e.g., by randomization plot)
randomization

Part II: Analysis of Results

8. Data Transformations

This stage presents the transformations and checks done on the collected data, performed before the analysis of results and any significance tests or other inferential statistics. This stage includes explaining

data transformations (e.g., log-transform, …)
outlier removal and/or winsorizing
checks for normality of data distributions
checks for other asssumptions of statistical tests used in analysis

9. Result Presentation

The presentation of results is an essential stage for a study report. This stage can be shown in different levels of detail: it can often use a single chart to summarize a key result, but this visualization may require some explanation to highlight

show significant differences between conditions (e.g., which differences did you find, how large were they, which differences were significant, …)
Show other findings such as trends, extreme values, distributions, outliers, etc.
You can provide different views onto the same chart (e.g., provide overview and detail, or show the same chart twice with different annotations). Furthermore, it may discuss how results reflect different conditions by referring to the stimuli.

10. Hypothesis Evaluation

This stage presents the conclusion by contrasting hypotheses, as stated in the beginning, with the result of the study. You may show

How do your hypotheses compare with your results?
Which hypotheses can be confirmed and which have been ruled out?