A foundational observation method for studying design situations

Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.