Novel and innovative modes of interacting with website information have arisen which necessitate methods and tools for their evaluation.  However, it is essential to develop such methods from contexts of use at a macro (i.e. cultural) and micro (individual use contexts) level.  Activity theory has been used extensively in systems evaluation as it bridges these two levels.  This article will extend activity theory by the development of a method, Analytical Activity Method (AAM).  The purpose of this method is as a tool to enable web designers and usability engineers to conduct usability evaluations of web interfaces, particularly the mobile web.  Based on the theoretical underpinnings of activity theory, AAM seeks to extend the activity checklist (Kaptelinin, Nardi, and Macaulay, 1999) for evaluation of websites on computers or mobile devices.  The application of this method contains the possibility to make visible that which was not obviously apparent from the theoretical tropes of Activity theory at system, historical and cultural level.  Finally, this paper concludes that the AAM has provided a comprehensive framework within which to conduct website evaluations and informs future designs.  It proposes some exciting new applications based on the AAM which not only extend activity theory but also provide a framework for understanding new methods of information access.

">

A Methodology For The Evaluation Of Online Learning Resources

Tom Page*, Gisli Thorsteinsson**, Lorna Uden***, Milka Lehtonen****
* Lecturer, Dept.of Design & Technology, Loughborough University, UK.
** Assistant Professor, Dept. of Design and Craft, Iceland University of Education.
*** Staffordshire University, UK.
**** Researcher, MOMENTS and Associate Professor, Media Education, University of Finland.
Periodicity:January - March'2008
DOI : https://doi.org/10.26634/jet.4.4.574

Abstract

Novel and innovative modes of interacting with website information have arisen which necessitate methods and tools for their evaluation.  However, it is essential to develop such methods from contexts of use at a macro (i.e. cultural) and micro (individual use contexts) level.  Activity theory has been used extensively in systems evaluation as it bridges these two levels.  This article will extend activity theory by the development of a method, Analytical Activity Method (AAM).  The purpose of this method is as a tool to enable web designers and usability engineers to conduct usability evaluations of web interfaces, particularly the mobile web.  Based on the theoretical underpinnings of activity theory, AAM seeks to extend the activity checklist (Kaptelinin, Nardi, and Macaulay, 1999) for evaluation of websites on computers or mobile devices.  The application of this method contains the possibility to make visible that which was not obviously apparent from the theoretical tropes of Activity theory at system, historical and cultural level.  Finally, this paper concludes that the AAM has provided a comprehensive framework within which to conduct website evaluations and informs future designs.  It proposes some exciting new applications based on the AAM which not only extend activity theory but also provide a framework for understanding new methods of information access.

Keywords

Evaluation, Learning Resources, Activity Theory

How to Cite this Article?

Tom Page, Gisli Thorsteinsson, Lorna Uden and Milka Lehtonen (2008). A Methodology For The Evaluation Of Online Learning Resources. i-manager’s Journal of Educational Technology, 4(4), 16-27. https://doi.org/10.26634/jet.4.4.574

References

[1]. Bærentsen, K. B. & J. Trettvik (2002). An Activity Theory Approach to Affordance. Proceedings of the second Nordic conference on Human-computer interaction. ACM Press, pp. 51-60.
[2]. Benyon, D., Crerar, A., & Wlkinson, S. (2001). Inclusive differences and Inclusive Design. In C. Stephanidis (Eds.), User Interfaces for All – Concepts, Methods and Tools (Vol. 1, pp. 21-46). Mahwah, NJ: Lawrence Erlbaum Associates
[3]. Bertelsen, O.W., & Bodker, S. (2003). Activity Theory. In Carroll, J.M. (ed.). HCI Models, Theories, and Frameworks. Morgan Kaufman Publishers, pp. 291-324.
[5]. Bodker, S. (1991). Through the interface: A Human Activity Approach to User Interface Design. Hillsdale, NJ: Lawrence Erlbaum Associates.
[6]. Brooks, R. (1991), Comparative task analysis: an alternative direction for Human-computer interaction science, Designing interaction: psychology at the human-computer interface, Cambridge University Press, New York, NY.
[7]. Buyukkokten, O., Garcia-Molina, H. & Paepcke A. (2001). Seeing the Whole in Parts: Text Summarization for Web Browsing on Handheld Devices. Proceedings of the Tenth International World Wide Web Conference, New York, NY: ACM,.
[8]. Chernobilsky, E., Nagarajan, A., Hmelo-Silver, C. E. (2005). Problem-based learning online: Multiple perspectives on collaborative knowledge construction. In T. Koschmann, D. Suthers, & T-W. Chan (Eds.), Proceedings of CSCL (pp. 63-52).
[9]. Cold, J (2006). Using Really Simple Syndication (RSS) to enhance student research. ACM SIGITE Newsletter. Vol. 3 Issue 1, pp.6 -9.
[10]. Dey, A and Abowd G. (2001). Towards a Better Understanding of Context and Context Awareness. GVU Technical Report GIT-GVU-99-22. College of Computing, Georgia Institute of Technology.
[11]. Engeström, Y. (1990). Activity theory and individual nd and social transformation. Opening address at 2 InternationalCongress for Research on Activity Theory, Lahti, Finland, May, pp. 21–25.
[12]. Hertzum, M., & Jacobsen, N. E. (2001). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 13(4), pp. 421-443.
[13]. Kaptelinin V. (1996). Computer-Mediated Activity: Functional Organs in Social and Developmental Contexts. In B.A. Nardi (Ed.), Context and consciousness: Activity Theory and Human–Computer Interaction, (pp.17–44). Cambridge, MA: MIT press.
[14]. Kaptelinin, V., Nardi, B. A. & Macaulay, C. (1997). The Activity Checklist: a Tool for Representing the Space of Context. Interaction, pp.27–39.
[15]. Kuutti, K. (1996). Activity theory as a Potential Framework for Human– Computer Interaction Research. In B.A. Nardi (Ed.), Context and consciousness: Activity Theory and Human–Computer Interaction, (pp.17–44). Cambridge, MA: MIT press.
[16]. Leontjev, A. (1978): Activity, Consciousness and Personality. Englewood Cliffs: Prentice-Hall
[17]. Lewis, C. (1997). Cognitive Walkthrough. In M.G. Helander, et al., (eds.) Handbook of Human Computer Interaction (pp. 717-732). Elsevier.
[19]. Monk, A. F. & Wright, P. C. (1991) Observations and Inventions: New Approaches to the Study of Human- Computer Interaction. Interacting with Computers, No.3, pp.204-216.
[20]. Nardi, B. A., (ed). (1996). Context and Consciousness: Activity Theory and Human-Computer Interaction. Cambridge, MA: MIT Press.
[21]. Nardi, B. (1997). Studying context: A comparison of activity theory, situated action models, and distributed cognition. In B. Nardi (Ed.), Context and consciousness: Activity theory and human computer interaction. Cambridge, MA: MIT Press.
[22]. Norman, D. (2006). Words Matter. Talk about People: Not customers, not consumers, not users. Interactions Vol. 13, No. 5, pp. 49 - 63.
[23]. Nielson, J. & Molich, R. (1990). Heuristic Evaluation of User Interfaces. In Proc. of CHI 90: Human Factors in Computer Systems. Seattle, WA: April 1-5, pp 249-256. New York, ACM/SIUGCHI.
[24]. Page, T., Lehtonen, M. and Thorsteinsson, G. (2006) 'The web-orientation agent (WOA) for simulated learning in technology education. International Journal of Learning Technology, 2(1), pp 62-76.
[25]. Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Addison-Wesley, Wokingham, UK.
[26]. Quek, A. and Shah, H. (2004) The Activity Theoretical Iterative Evaluation Method. In Proceedings of 1st International Workshop on Activity Theory Based Practical Methods for IT Design, part of the 3rd Nordic Conference on Cultural and Activity Research, Copenhagen, Denmark.
[27]. Ryu, T., (2006). Walkthroughs in Web Usability in Zaphiris and Kurnaiwan (eds), Human Computer Interaction Research in Web Design and Evaluation pp. 229 -257.
[28]. Suchman, L. (1987). Plans and Situated Actions. Cambridge University Press.
[29]. Uden, L. (2007). Activity theory for designing mobile learning. International Journal of Mobile Learning and Organisation. Vol. 1, Number 1, 29 October 2006, pp. 81- 102(22).
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Online 15 15

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.