Methods Example: Lab Study

How Fast Is Too Fast?

Purpose: As a member of the Open Video research group I helped conduct a series of experiments aimed at gathering empirical evidence about how people interact with preview surrogates (storyboards, slideshows, audio-enhanced previews, keyword-enhanced previews, etc.) for digital video. For example, in one study we investigated the tradeoff between speed and understanding when using fast-forward preview surrogates. Clearly, a faster preview speed will shorten the necessary viewing time, but if the preview surrogate goes by too fast, it will not be useful in supporting accurate relevance judgments of the corresponding video. So, how fast is too fast?

Method: A total of 45 participants took part in the study, in individual sessions lasting about an hour. Each participant viewed four preview surrogates, which varied according to three independent variables: color or black & white media, narrative or documentary content, and preview speed (32, 64, 128, or 256 times normal viewing speed). Both the independent variable combination and the presentation order were counterbalanced. After watching a given fast-forward preview, the participant completed a series of tasks that measured performance on six dependent variables.

Results: As expected, as the speed of the fast-forward surrogate increased, performance decreased, for all tasks. However, the speed at which the performance decrease was statistically significant varied depending on the task. Overall, the fast-forward speed of 1:64 supported good task performance and user satisfaction; this was our recommendation for those considering offering fast-forward video previews, at least until further research sheds more light on the topic.

My Role: I developed the Web-based interface used by participants to view the previews and perform tasks. I designed and developed the database that stored task data. I conducted some of the user participant sessions in the lab, prepared data for statistical analysis, and contributed to analysis discussions and the writing of publications and presentations that described the study.

Related Information

Wildemuth, B.M., Marchionini, G., Yang, M., Geisler, G., Wilkens, T., Hughes, A., & Gruss, R. (2003). How fast is too fast? Evaluating fast forward surrogates for digital video. In Proceedings of the 3rd ACM/IEEE Joint Conference on Digital Libraries (JCDL 2003), 221-230.(Vannevar Bush Award Winner for Best Paper at JCDL 2003)
Geisler, G. (2003). AgileViews: A framework for creating more effective information seeking interfaces.Unpublished Doctoral Dissertation.
Wildemuth, B. M., Yang, M., Geisler, G., Tolleson, T., Elsas, J., Luo, J., & Marchionini, G. (2004). Conceptions of features and semantic clusters as search mechanisms: A pilot study.TREC VID Notebook Paper.
Wildemuth, B.M., Marchionini, G., Wilkens, T., Yang, M., Geisler, G., Fowler, B., Hughes, A., & Mu, X. (2002). Alternative surrogates for video objects in a digital library: Users' perspectives on their relative usability. In Proceedings of the 6th European Conference on Digital Libraries.

Other Lab Studies

How Do People Describe Online Video?
A total of 32 study participants were asked to view four different video resources (from a pool of eight videos in total), answering six questions for each video that elicit descriptions of the video in various ways.
AgileViews Dissertation Study.
Twenty-eight participants completed tasks with two information seeking interfaces providing access to the same resources. Performance with the two interfaces was measured in terms of navigational efficiency, task completion time, navigational backtracking, and task success, as well as several qualitative measures.
Conceptions of features and semantic clusters as search mechanisms: A pilot study.
Four team members completed TREC VID topic searches and responded to measures of their perceptions of the experience of using each search mechanism.
Alternative surrogates for video objects in a digital library: Users' perspectives on their relative usability.
Ten participants viewed several forms of video preview surrogates (storyboards with text or audio keywords, slide shows with text or audio keywords, fast forward) for seven video segments and performed a series of tasks (gist determination, object recognition, action recognition, and visual gist determination) to measure understanding and preferences.