The Praxis of Learning Analytics for a Conceptual Open Textbooks System

Table of contents

1. Introduction

t is no longer a secret-if, indeed, it ever was-that escalating textbook costs are putting them beyond the affordability of many students. Senack (2014) in a survey of 2,039 university students, reported that 65% of students had no other choice than opting out of buying a textbook due to expense, and of those students, 94% admitted that doing so would negatively affect their grade in that course. These findings are representative of several other studies (see, for example, Acker, 2011; N. Allen, 2011; Florida Virtual Campus, 2012; Graydon, Urbach-Buholz, & Kohen, 2011; Morris-Babb & Henderson, 2012; Prasad & Usagawa, 2014), showing that affordability of traditional textbooks has become more difficult for many students and thus, in some cases, a barrier to learning.

Despite the problems outlined above, there is no indication that textbook prices will decrease in the foreseeable future; on the contrary, trends point to further increases. However, fortunately, open textbooks hold promise to provide a solution. Weller (2014) appraises open textbooks, a type of open educational resource, as one most amenable to the concept of open education, a concept essentially about elimination of barriers to learning (Bates, 2015). The phrase open educational resource (OER) is an umbrella term used to collectively describe those teaching, learning, or research materials that can be used without charge to support access to knowledge (Hewlett, 2013a). Within the OER context, "freely" means both that the material is openly available to anyone free of charge, either in the public domain or released with an open license such as a Creative Commons license; and that is made available with implicit permission, allowing anyone to retain, reuse, revise, remix, and redistribute the resource (Center for Education Attainment and Innovation, 2015). Conversely, traditional textbooks are extremely expensive and are published under an All Rights Reserved model that restricts their use (Wiley, 2015).

Within the past few years a growing body of literature has examined the potential cost savings and learning impacts of open textbooks. Senack (2014), for example, in a survey of 2,039 university students indicated that open textbooks could save students an average of $100 per course. Similarly, Wiley, Hilton, Ellington, and Hall (2012) in a study of open textbook adoption in three high school science courses found that open textbooks cost over 50% less than traditional textbooks and that there were no apparent differences (neither increase nor decrease) in test scores of students who used open versus traditional textbooks, a finding replicated by Allen, Guzman-Alvarez, Molinaro, and Larsen (2015). This latter finding is in contrast with the findings of Hilton and Laman (2012), who reported that students who used open textbooks instead of traditional textbooks scored better on final examinations, achieved better grades in their courses, and had higher retention rates. A study by Robinson, Fischer, Wiley, and Hilton (2014) (Hewlett, 2013b). The amount of money injected into such projects is significant, and therefore funders, besides requiring usual information about impacts on cost and learning outcomes, are now also increasingly asking for more rigorous information regarding ways-whether, when, how often, and to what degree-in which learners actually engage with their open textbooks. More specifically, as stressed by Stacey (2013) The above discussion points to the need for more sophisticated methods of monitoring open textbook utilization in order to meet these information needs. New analytical methodologies-particularly learning analytics-have made fulfilling this requirement possible. Compared with more subjective research methods such as surveys and questionnaires, learning analytics can capture learners' authentic interactions with their open textbooks in real time. This may improve understanding of textbook usage influences on actual usage behavior, which in turn may help improve efficiency and effectiveness of open textbooks. The method can be used either as a standalone method or to support other traditional research methods. Moreover, learning analytics for open textbooks can provide new insights into important questions such as how to assess learning outcomes based on textbook impact; whether student behavior, content composition, and learning design principles produce intended learning outcomes; and the level of association between amount of markups done and the relevance and difficulty level of the book content areas.

Despite these great potential benefits, so far there exist no studies published to date on systems developed for open textbooks learning analytics. Thus, the main aim of this paper is to close this gap by presenting developmental work and functionalities of a conceptual open textbooks learning analytics system. A distinctive feature of this proposed system is its ability to synchronize online and offline interactional data on a central database, allowing both instructors and designers to generate analysis in dashboard-style displays.

The remainder of this paper is organized in the following manner. It starts with a brief review of literature related to learning analytics, followed by a summary of the framework that guided our development. Next, it describes techniques and tools applied in development of the conceptual learning analytics system for open textbooks, which is the main focus of this paper. The final section concludes the paper and talks about future work.

2. II. Literature Review on Learning Analytics

The concept of learning analytics has been making headlines for some years now, firing up interest amongst the higher educational community worldwide, but its definition remains unified. One frequently cited definition is "the measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs" (Siemens & Long, 2011, p. 34). In other words, learning analytics applies different analytical methods (e.g., descriptive, inferential, and predictive statistics) to data that students leave as they interact with and within networked technology-enhanced learning environments so as to inform decisions about how to improve student learning. A survey of published research shows that learning analytics tactics have been applied in a variety of ways and found useful, some of which include identifying struggling students in need of academic support (Arnold & Pistilli, 2012 (Fritz, 2013).

Given the benefits and opportunities offered by learning analytics, researchers and practitioners have expressed concern about the importance of maintaining the privacy of student data. As Scheffel, Drachsler, Stoyanov, and Specht (2014) emphasize, the nascent state of learning analytics has rendered "a number of legal, risk and ethical issues that should be taken into account when implementing LA at educational institutions" (p. 128). It is common to hear that such considerations are lagging behind the practice, which indeed is true. As such, many individual researchers, as well as research groups, have proposed ethical and privacy guidelines to guide and direct the practice of learning analytics. In June 2014, the Asilomar Convention for Learning Research in Higher Education outlined the following six principles (based on the 1973 Code of Fair Information Practices and the Belmont Report of 1979) to inform decisions about how to comply with privacy-related matters on the use of digital learning data. 1) respect for the rights and dignity of learners, 2) beneficence, 3) justice, 4) openness, 5) the humanity of learning, and 6) the need for continuous consideration of research ethics in the context of rapidly changing technology. Similarly, Pardo and Siemens (2014) in the same year identified the following four principles: 1) transparency, 2) student control over data, 3) security, and 4) accountability and assessment.

Furthermore, in a literature review of 86 articles (including the preceding two publications) dealing withethical and privacy concepts for learning analytics, Sclater (2014) found that the key principles which their authors aspired to encapsulate were "transparency, clarity, respect, user control, consent, access and accountability" (p. 3).

In this context, it is worth noting that "a unified definition of privacy is elusive" (Pardo& Siemens, 2014, p. 442), just like the definition of learning analytics as noted earlier. While there is no unified definition of learning analytics and its privacy practices, there is general agreement that it is crucial for higher educational institutions to embrace learning analytics strategies as a way to improve student learning, but without violating students' legal and moral rights.

3. III.

4. Conceptual Framework

Developmental work was guided by our earlier work proposed in (Prasad, Totaram, &Usagawa, 2016) describing a framework for development of an open textbooks analytics system, as shown in Figure 1. This framework supports textbooks in the EPUB format, a format that has become the international standard for digital books. EPUB file formats are actually advanced html text pages and image files that are compressed and then use a file extension of .epub. Notably, this framework is not specific to open textbooks but equally applicable to other EPUB digital books.

5. System Development : Techniques and Tools

Based on the suggested framework presented above, this section describes methods applied and technologies used in the development of learning analytics system for open textbooks in EPUB format, and is divided into two subsections: data collection, and data analysis and presentation.

6. IV.

7. Data Collection Data Recording

Reading books in EPUB format requires an EPUB reader application. In line with the suggestions of the framework, EPUB.js (https:// github.com/ futurepress /epub.js), an open source web-based EPUB reader application, can be adopted and customized as a central tool to aid datacollection. EPUB.js previously possessed capabilities to record user clicks and annotation data in the local storage of the web browser used by the user to access the EPUB.js application. These capabilities were expanded to record and track a variety of other data, such as user's IP address, web browser type and version, and the type of device used. Following these modifications to the EPUB.js reader application, the EPUB file of the book was embedded into the reader application for standard data collection. Figure 2 represents the customized EPUB.js reader application's user interface. This customized version was used for both online and offline delivery. For online use, the customized EPUB.js reader application was hosted on a web server accessible via the Internet from any web browser. To facilitate offline access, an application installer was created for the Windows platform as the majority of users used Windows-based computers. This installer conveniently installed the customized EPUB.js reader application to the users' computers (Figure 3). However, offline access was limited to a particular web browser: Mozilla Firefox. This is because the customized EPUB reader application used Javascript to send user data to an external data storage server, which most web browsers blocked as a potential security risk. Thus, for offline access, the user interaction recording features were incompatible with most web browsers. Consequently, offline access of the EPUB reader application required the use of Mozilla Firefox (Figure 4). Further development is required to make the code compatible with other web browsers. application is initiated by the user, requiring the user to enter their name and student ID number. These credentials are stored in the local storage of the user'sbrowser, and all user-interaction data sent to the server (and subsequently processed) is tagged with the user's authentication details. The Praxis of Learning Analytics for a Conceptual Open Textbooks System V.

8. Data Synchronization

Data generated during offline usage are stored in the browser's local storage. For the purpose of synchronizing data from web browser's local storage to server, a network-sensing feature was integrated into the EPUB.js reader. This feature checks for an Internet connection at regular intervals (in our case, every 60 seconds) to determine if the user's device is connected to the Internet, and whenever an Internet connection is detected, data from the browser's local storage are sent to the central database server, where the data is used for analytics. However, when used in online mode, the interactional data is directly sent in a database.

9. VI.

10. Data Storage

A central database server, a combination of a PHP script and MySQL database, can be used for data storage. The MySQL database is used to store data, while the PHP script waits for the interaction data to be received from the EPUB reader application. Once receiving new interaction data, the PHP script can validate and records it to the MySQL database. Table 1 shows sample data types recorded for each user interaction.

11. Data Analysis and Presentation

Analysis can be performed on both individual and aggregate (whole class) data, and can be analyzed with regard to various factors as listed below.

12. ?

Links clicked per chapter, per student, or for whole class: This is the count of the links clicked in each chapter.

? Popular web browser used by the users to access the ebook: This is the count of each web browser used.

13. ?

Popular type of device used by the users to access the ebook: This is the count of each device type used.

? Online versus offline usage: This is the count of the all user interaction for online access and offline access.

? Number of students versus chapters viewed: This is the count of the number of students who viewed each chapter.

14. ?

Total views per chapter, per student, or for whole class: This is the count of the number of page views for each chapter.

15. ?

Total bookmarks per chapter, per student, or for whole class: This is the count of the number of bookmarks made in each chapter.

16. ?

User annotations/notes made per chapter, per student, or for whole class: This is a list of all the annotations/notes made for each chapter. viewed: This is the count of the number of students and the count of the number of chapters viewed by each student.

17. ?

? Weekly user interaction: This is the count of the number of interactions by all users grouped by weeks.

The analysis and data presentation (graphic visualizations in dashboard format) can be done using PHP and a Javascript charting library. Computation of interaction data can be done using SQL queries, while the rendering (in dashboard display format) can be done using PHP with the help of a Javascript charting library. Figure 6 shows snapshot of the sample learning analytics dashboard.

Figure 1. Figure 1 :
1Figure 1: Open textbooks analytics system framework. Adapted from "A Framework for Open Textbooks Analytics System," by D. Prasad, R. Totaram, and T. Usagawa, 2016, Tech Trends 1-6. doi:10.1007/s11528-016-0070-3. As illustrated by Figure 1, the framework consists of a nine-step approach beginning with students' initial contact with the text. The figure also illustrates two separate branches of process flow, one for onlineaccess and the other for offline access. All nine steps depicted within the framework, including certain stage-specific mandatory technical requirements, are summarized stepwise as follows: ? A student may access an open textbook, which technically is the synthesis of web-based EPUB reader application, and .epub file of the book to ensure standardized data recording, in online mode, offline mode, or both ways. ? Authentication is optional. It may be useful for organizations that want to identify the learners, for early intervention or to gauge their performance or study their usage patterns, etc. ? Every interaction between a student and an open textbook produces data. These interactional data are records of students' actions with the textbook. Students' actions such as page navigation, jumping to a particular chapter, bookmarks, and annotation notes may be recorded in real-time with a time stamp and user device used. ? When textbooks are used in offline mode, interaction data are temporarily stored in the local storage of the web browser that runs the EPUB reader application. ? As soon as the EPUB reader application detects an Internet connection, it sends all local offline interaction data to the central database. ? Analyses are done on the aggregated interaction data stored in the central database. ? Results of analyses are rendered to the stakeholders for consumption. ? Each stakeholder may take appropriate action on the basis of results. ? The textbook may be revised if required before being made available to the next batch of students.
Figure 2. Figure 2 :
2Figure 2: Customized EPUB.js reader user interface.
Figure 3. Figure 3 :
3Figure 3: Textbook short cut from start menu.
Figure 4. Figure 4 :
4Figure 4: Required browser for offline access.
Figure 5. Figure 5 :
5Figure 5: Sample Authentication interface
Figure 6. Figure 6 :
6Figure 6: Snapshot of the learning analytics dashboard
Figure 7.
Figure 8.
these studies have put forth evidence showing that
replacing traditional textbooks with open textbooks
substantially reduces textbook costs without negatively
affecting student learning. Consequently, demand for
open textbooks is increasing.
As demand has grown, so too have efforts to
develop and distribute open textbooks. Many of these
development and distribution practices are
accomplished through a combination of government,
private, and philanthropic funding
Figure 9. Table 1 :
1
1

Appendix A

  1. , 10.1108/17415651111125513. Interactive Technology and Smart Education 8 (1) p. .
  2. 2012 Florida student textbook survey, http://www.openaccesstextbooks.org/pdf/2012_Florida_Student_Textbook_Survey.pdf 2012.
  3. Learning analytics for online discussions: Embedded and extracted approaches. A F Wise , Y Zhao , S N Hausknecht . Journal of Learning Analytics 2014. 1 (2) p. .
  4. Ethical and privacy principles for learning analytics. A Pardo , G Siemens . 10.1111/bjet.12152. British Journal of Educational Technology 2014. 45 (3) p. .
  5. A study of four textbook distribution models. B Graydon , B Urbach-Buholz , C &kohen . http://www.educause.edu/ero/article/study-four-tex-tbook-distribution-models Educause Quarterly 2011. (4) p. 34.
  6. The effect of recommendation systems on Internetbased learning for different learners: A data mining analysis. C.-C Liu , C.-J Chang , J.-M Tseng . 10.1111/j.1467-8535.2012.01376.x. British Journal of Educational Technology 2013. 44 (5) p. .
  7. The Dataassisted approach to building intelligent technologyenhanced learning environments. C Brooks , J Greer , C &gutwin . Learning analytics: From research to practice, J A Larusson&, B White (ed.) (New York
    ) 2014. Springer. p. .
  8. Collaborative spatial classification. Fourth International Conference on Learning Analytics And Knowledge, p. .
  9. Tap into visual analysis of customization of grouping of activities in eLearning. D A Gómez-Aguilar , Á Hernández-García , F J García-Peñalvo , R &therón . 10.1016/j.chb.2014.11.001. Computers in Human Behavior 2015. 47 p. .
  10. An overview of learning analytics. Teaching in Higher Education, D Clow . 10.1080/13562517.2013.827653. 2013. 18 p. .
  11. Let's not forget: Learning analytics are about learning, D Ga?evic , S Dawson , G Siemens . doi:10. 2015. 59 p. . (Tech Trends)
  12. Scoping the possibilities: Student preferences towards open textbooks adoption for e-learning. D Prasad , T Usagawa . doi:10.4236/ ce.2014.524227. Creative Education 2014. 5 (24) p. .
  13. A framework for open textbooks analytics system, D Prasad , R Totaram , T Usagawa . 2016.
  14. A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. The International Review of Research in Open and Distributed Learning, D Wiley , J Hilton , S Ellington , T Hall . http://www.irrodl.org/index.php/irrodl/article/view/1153/2256 2012. 13 p. .
  15. On the relationship between OER adoption initiatives and libraries, D Wiley . http://opencontent.org/blog/archives/3883 2015. May 20.
  16. , E Coopey , R B Shapiro , E &danahy . 2014.
  17. How some colleges are _campaign=Feed%3A+rss%2Fcnn_latest+%28RS, E Grinberg . 2014. April 21.
  18. , Final_0 , Pdf .
  19. Assessing the impact and efficacy of the open-access chemwiki textbook project, G Allen , A Guzman-Alvarez , M Molinaro , D Larsen . https://net.educause.edu/ir/library/pdf/elib1501.pdf 2015.
  20. Penetrating the fog: Analytics in learning and education. G Siemens , P Long . EDUCAUSE Review 2011. 46 (5) p. .
  21. White paper: Open educational resources, Hewlett . fromhttp://www.hewlett 2013b.
  22. Improving students' help-seeking skills using metacognitive feedback in an intelligent tutoring system. I Roll , V Aleven , B M Mclaren , K R Koedinger . 10.1016/j.learninstruc.2010.07.004. Learning and Instruction 2011. 21 (2) p. .
  23. Discovering usage behaviors and engagement in an Educational Virtual World. J Cruz-Benito , R Therón , F J García-Peñalvo , E P Lucas . 10.1016/j.chb.2014.11.028. Computers in Human Behavior 2015. 47 p. .
  24. Learning analytics and ethics: A framework beyond utilitarianism, J E Willis . http://er.educause.edu/articles/2014/8/learning-analytics-and-ethics-a-framework-beyond-utilitarianism 2014.
  25. Using analytics at UMBC: encouraging student responsibility and identifying effective course designs, J Fritz . https://net.educause.edu/ir/library/pdf/ERB1304.pdf 2013. April.
  26. One college's use of an open psychology textbook. J Hilton , C Laman . doi:10.1080/ 02680513.2012.716657. Open Learning: The Journal of Open, Distance and E-Learning, 2012. 27 p. .
  27. How does the "Open" in OER improve student learning?, J Hilton . http://www.johnhiltoniii.org/how-does-the-open-in-oer-improve-student-learning/ 2016. February. (Web log post)
  28. Course signals at Purdue: Using learning analytics to increase student success, K E Arnold , M D &pistilli . S. B. Shum, D. Ga?evic (ed.) 2012.
  29. Mining LMS data to develop an "early warning system" for educators: A proof of concept. L P Macfadyen , S Dawson . Computers & Education 2010. 54 (2) p. .
  30. L Ricci , B Rokach , Shapira . Recommender systems handbook, & P B Kantor (ed.) p. .
  31. Tracking Student Behavior, Persistence, and Achievement in Online Courses. L V Morris , C Finnegan , S.-S Wu . Internet and Higher Education 2005. 8 (3) p. .
  32. Usage pattern recognition in student activities. M Friedrich , K Schmidt , . . Kloos , C ; M Niemann , K Pardo , A Leony , D , C Kloos , D Gillet , R Crespo García , F Wild . 43. Scheffel, 2011.
  33. The open learning initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. M Lovett , O Meyer , C &thille . 10.5334/2008-14. http://doi.org/10.5334/2008-14 Journal of Interactive Media in Education 2008. 2008. (1) . (Art.13)
  34. TrAVis to enhance online tutoring and learning activities: Real--time visualization of students tracking data, M May , S George , P &prévôt . 2011.
  35. An experiment in open-access textbook publishing: Changing the world one textbook at a time. M Morris-Babb , S Henderson . 10.3138/jsp.43.2.148. Journal of Scholarly Publishing 2012. 43 (2) p. .
  36. M Scheffel , H Drachsler , S Stoyanov , M &specht . Quality indicators for learning analytics, 2014.
  37. Battle for open: How openness won and why it doesn't feel like victory, M Weller . 10.5334/bam. http://dx.doi.org/10.5334/bam 2014. London: Ubiquity Press.
  38. High prices prevent college students from buying assigned textbooks. Student PIRGs, N Allen . http://www.studentpirgs.org/news/ap/high-prices-prevent-college-students-buying-assigned-textbooks 2011.
  39. Why open education demands open analytics, N Bier , C Green . http://openeducation2015.sched.org/event/49J5/why-open-education-demands-open-analytic-models 2015.
  40. Recommender systems in technology enhanced learning, N Manouselis , H Drachsler , R Vuorikari , H Hummel , R &koper . 2011.
  41. Finding student-centered open learning environments on the internet: Automated dialogue assessment in academic virtual communities of practice. N Nistor , S Tra?san-Matud , M Dascalu , H Duttweiler , C Chiru , B Baltes , G Smeaton . doi:10.1016/ j.chb.2014. 07.029. Computers in Human Behavior 2015. 47 p. .
  42. Fixing the broken textbook market: how students respond to high textbook costs and demand alternatives. N Sclater . http://www.washpirg.org/sites/pirg/files/reports/1.27.14FixingBrokenTextbooksReport.pdf Senack, E 2014. 2014. (Code of practice for learning analytics: A literature review of the ethical and legal issues)
  43. , N Y York , Usa: Acm . 10.1145/2567574.2567611.
  44. Open textbooks: The current state of play, http://www.acenet.edu/news-room/Documents/Quick-Hits-Open-Textbooks.pdf 2015.
  45. Assessment analytics using turnitin & grademark in an undergraduate medical curriculum. P Reed , S Watmough , P Duvall . 10.14297/jpaap.v3i2.159. Journal of Perspectives in Applied Academic Practice 2015. 3 (2) p. .
  46. Government support for open educational resources: Policy, funding, and strategies. The International Review of Research in Open and Distributed Learning, P Stacey . http://www.irrodl.org/index.php/irrodl/article/view/1537/2481 2013. 14 p. .
  47. Developing an early-alert system to promote student visits to tutor center. Q Cai , C L Lewis , J Higdon . The Learning Assistance Review 2015. 20 (1) p. .
  48. References Références Referencias offering free textbooks, http://edition.cnn.com/2014/04/18/living/opentextbooks-online-education-resources/index.html?utm
  49. 10.1145/2330601.2330666. 2ndInternational Conference on Learning Analytics and Knowledge, R Ferguson (ed.) (New York, NY, USA
    ) ACM. p. .
  50. 10.1145/2330601.2330627. 2ndInternational Conference on Learning Analytics and Knowledge, R Ferguson (ed.) (New York, NY, USA
    ) ACM. p. .
  51. Learning analytics to identify exploratory dialogue within synchronous text chat. R Ferguson , S B Shum . 10.1145/2090116.2090130. 1st International Conference on Learning Analytics and Knowledge, (New York, NY, USA
    ) 2011. ACM. p. .
  52. An evaluation of learning analytics to identify exploratory dialogue in online discussions. R Ferguson , Z Wei , Y He , S B Shum . 10.1145/2460296.2460313. Third International Conference on Learning Analytics and Knowledge, (New York, NY, USA
    ) 2013. ACM. p. .
  53. S%3A+Most+Recent% 29&utm_ medium=feed & utm_source=feedburner 21. http://www.hewlett.org/programs/education/open-educational-resources Hewlett 2013a. (Open educational resources)
  54. Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. S Lonn , S J Aguilar , S D &teasley . 10.1016/j.chb.2014.07.013. Computers in Human Behavior 2015. 47 p. .
  55. Early alert of academically at-risk students: An open source analytics initiative. S M Jayaprakash , E W Moody , E J M Lauría , J R Regan , J D Baron . Journal of Learning Analytics 2014. 1 (1) p. .
  56. Digital textbooks: A state-level perspective on affordability and improved learning outcomes. S R Acker . Library Technology Reports 2011. 47 (8) p. .
  57. What do we mean by "open, T Bates . http://www.tonybates.ca/2015/02/16/what-do-we-mean-by-open-in-education/ 2015. February. (in education? [Web log post)
  58. , Techtrends . 10.1007/s11528-016-0070-3. p. .
  59. The impact of open textbooks on secondary science learning outcomes. T J Robinson , L Fischer , D Wiley , J Hilton . 10.3102/001. Educational Researcher 2014. 43 (7) p. .
  60. What to do with actionable intelligence: E2Coach as an intervention engine, T Mckay , K Miller , J &tritz . S. B. Shum, D. Ga?evic (ed.) 2012.
  61. , Us: Springer . 10.1007/978-0-387-85820-3_12.
  62. Wolpers . 10.1007/978-3-642-23985-4_27. Towards ubiquitous learning, (Berlin Heidelberg
    ) Springer. 6964 p. .
  63. Using Signals for appropriate feedback: Perceptions and practices. Z Tanes , K E Arnold , A S King , M A Remnet . doi:10.1016 /j.compedu. Computers & Education 2011. 57 (4) p. .
Notes
1
© 2016 Global Journals Inc. (US)The Praxis of Learning Analytics for a Conceptual Open Textbooks System
Date: 2016-01-15