The realities of evaluating educational technology in school settings

Megan Venn-Wycherley; Ahmed Kharrufa; Susan Lechelt; Rebecca Nicholson; Kate Howland; Abrar Almjally; Anthony Trory; Vidya Sarangapani

doi:10.1145/3635146

The realities of evaluating educational technology in school settings

Megan Venn-Wycherley^*, Ahmed Kharrufa, Susan Lechelt, Rebecca Nicholson, Kate Howland, Abrar Almjally, Anthony Trory, Vidya Sarangapani

^*Corresponding author for this work

School of Computer Science

Research output: Contribution to journal › Article › peer-review

10 Citations (Scopus)

53 Downloads (Pure)

Abstract

HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors' contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

Original language	English
Article number	26
Pages (from-to)	1-33
Number of pages	33
Journal	ACM Transactions on Computer-Human Interaction
Volume	31
Issue number	2
Early online date	5 Feb 2024
DOIs	https://doi.org/10.1145/3635146
Publication status	Published - 1 Apr 2024

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 4 Quality Education

Keywords

Additional Key Words and PhrasesEducational technology
classroom evaluation

Access to Document

10.1145/3635146

The realities of evaluating educational technology in school settingsAccepted author manuscript, 634 KB

Cite this

@article{662e54eaa3224ef28e8a4bd20a73edb1,

title = "The realities of evaluating educational technology in school settings",

abstract = "HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors' contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.",

keywords = "Additional Key Words and PhrasesEducational technology, classroom evaluation",

author = "Megan Venn-Wycherley and Ahmed Kharrufa and Susan Lechelt and Rebecca Nicholson and Kate Howland and Abrar Almjally and Anthony Trory and Vidya Sarangapani",

note = "Funding information: We would like to express our sincere thanks to all those who took part in the referenced research, particularly to the staff, pupils and research teams without whom these insights would not have been possible. This work was funded by the EPSRC, including the Centre for Digital Civics award number EP/L016176/1 and EPSRC and BBC iCASE scholarship award number 1623937, and Al Imam Mohammad Ibn Saud Islamic University (IMSIU).",

year = "2024",

month = apr,

day = "1",

doi = "10.1145/3635146",

language = "English",

volume = "31",

pages = "1--33",

journal = "ACM Transactions on Computer-Human Interaction",

issn = "1073-0516",

publisher = "ACM",

number = "2",

}

TY - JOUR

T1 - The realities of evaluating educational technology in school settings

AU - Venn-Wycherley, Megan

AU - Kharrufa, Ahmed

AU - Lechelt, Susan

AU - Nicholson, Rebecca

AU - Howland, Kate

AU - Almjally, Abrar

AU - Trory, Anthony

AU - Sarangapani, Vidya

N1 - Funding information: We would like to express our sincere thanks to all those who took part in the referenced research, particularly to the staff, pupils and research teams without whom these insights would not have been possible. This work was funded by the EPSRC, including the Centre for Digital Civics award number EP/L016176/1 and EPSRC and BBC iCASE scholarship award number 1623937, and Al Imam Mohammad Ibn Saud Islamic University (IMSIU).

PY - 2024/4/1

Y1 - 2024/4/1

N2 - HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors' contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

AB - HCI researchers are increasingly interested in the evaluation of educational technologies in context, yet acknowledge that challenges remain regarding the logistical, material and methodological constraints of this approach to research [18, 53].Through the analysis of the authors' contributed thematic research vignettes, the following article exposes the practical realities of evaluating educational technologies in school settings. This includes insights into the planning stages of evaluation, the relationship between the researcher and the school environment, and the impact of the school context on the data collection process.We conclude by providing an orientation for the design of HCI educational technology research undertaken in school contexts, providing guidance such as considering the role of modular research design, clarifying goals and expectations with school partners, and reporting researcher positionality.

KW - Additional Key Words and PhrasesEducational technology

KW - classroom evaluation

UR - https://www.scopus.com/pages/publications/85184839342

U2 - 10.1145/3635146

DO - 10.1145/3635146

M3 - Article

SN - 1073-0516

VL - 31

SP - 1

EP - 33

JO - ACM Transactions on Computer-Human Interaction

JF - ACM Transactions on Computer-Human Interaction

IS - 2

M1 - 26

ER -

The realities of evaluating educational technology in school settings

Abstract

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this