Tag & track. Analytics as a development and evaluation tool

van der Sluis, Hendrik and Loughlin, Colin (2013) Tag & track. Analytics as a development and evaluation tool. In: altc2013 Building New Cultures of Learning; 10–12 Sep 2013, Nottingham, U.K.. (Unpublished)

Abstract

Most instructional design (ID) models, of which arguably the most prominent are ADDIE and the Dick & Carey System Approach, envisage an iterative process in the development of Learning Objects (LOs) between the objective, context, design, development and evaluation, to ensure that the LOs are fit for purpose. However, the literature suggests that instructional designers give less attention to the evaluation of usage than is desired to generate meaningful improvements in content delivery (e.g. Ozdileka & Robeck, 2009). The intention with Kingston University’s Plagiarism Tutorial was to develop the LO in an iterative way, incorporating learning from a combined evaluation process and assess its impact on instances of plagiarism within specific cohorts of students who had used the resource. The resource was developed to enhance existing material on plagiarism and referencing; one aim of which was to improve academic skills in first year students. The content of the Plagiarism Tutorial was redesigned to take advantage of HTML5 mobile technology and was accompanied by examples, tips, links and FAQ’s to provide a pertinent, concise and attractive LO. In order to enhance engagement, short videos in which students explain elements of plagiarism in their own words were included for each topic. The resource contains an online test which provides students with a personalised certificate once a minimum score is attained; this is also used as part of a compliance procedure, providing proof of completion by students who have been accused of plagiarism. The Plagiarism Tutorial was therefore developed with evaluation in mind and includes applied client side data collection methodology as well as other forms of evaluation; such as, an online survey and qualitative interviews which were used to obtain the student perspective. Web-analytics to improve user engagement with websites sites have existed within industry for many years and consist of two main methods: server-side or, client-side data collection and analysis. The latter has, in recent years, become more sophisticated, easier to use and affordable (e.g. Ledford et al., 2010), providing a feasible alternative as an evaluation tool in an educational setting. After a moderate investment of time spent in applying page-tags and event-tracking etc., data for user interaction with the learning object and its elements are instantly available. The results offer a rich insight into the visitor’s behaviour, with engagement indicators such as; time spent on the site, page visits, learning elements used, and the pathway of pages visited. This presentation will explore the results of the evaluation of the Plagiarism Tutorial and show how the page-tagging and event-tracking were useful to determine which pages, objects and links were most effective and, how this information has/will inform further development. References Ledford, J., Teixeira, J. & Tyler, M.E. (2010). Google Analytics (3rd Ed). Indianapolis (USA): Wiley Publishing. Ozdileka, Z. & Robeck, E. (2009), Operational priorities of instructional designers analyzed within the steps of the Addie instructional design model, Procedia Social and Behavioral Sciences 1: 2046–2050.

Actions (Repository Editors)

Item Control Page Item Control Page