Due to rapid advancements in our ability to collect, process, and analyze massive amounts of data, it is now possible for educators at all levels to gain new insights into how people learn. According to Bainbridge, et. al. (2015), using simple learning analytics models, educators now have the tools to identify, with up to 80% accuracy, which students are at the greatest risk of failure before classes even begin. As we consider the enormous potential of data analytics and data mining in education, we must also recognize a myriad of emerging issues and potential consequences—intentional and unintentional—to implement them responsibly. For example:
- Who collects and controls the data?
- Is it accessible to all stakeholders?
- How are the data being used, and is there a possibility for abuse?
- How do we assess data quality?
- Who determines which data to trust and use?
- What happens when the data analysis yields flawed results?
- How do we ensure due process when data-driven errors are uncovered?
- What policies are in place to address errors?
- Is there a plan for handling data breaches?
The book will provide insights and support for policy makers, administrators, faculty, and IT personnel on issues pertaining to: how data analytics can be used to improve the quality of courses and programs; technological and resource support, including networking capacity and infrastructure; accessibility and use; institutional and management related to ownership of data and decisions on how the data and its interpretations may inadvertently impact students, faculty, and the institution; the quality and reliability of data, as well as accuracy of data-based decisions; and, ethical implications surrounding the collection, distribution, and use of student-generated data.
Due to the complexity of the issues related to the responsible implementation of data analytics in education, it is helpful to examine them through a systematic structure. Khan’s Learning Framework provides such a structure. Each dimension of the framework represents a variety of issues that need to be considered before implementation of a new initiative or program.
The framework, which was initially developed in 1997 to address issues pertaining to the successful implementation of e-learning in education and training, has evolved to encompass the design and delivery of effective, efficient and engaging learning across multiple learning environments and contexts. The framework has been adopted by academics and researchers from around the world and applied to multiple modes of instructional delivery, including distance learning, mobile-learning, blended learning, computer-based training, and technology supported traditional instruction. Almost 20 years old, Khan’s framework remains a valuable tool for evaluating an organization’s educational technology readiness and opportunities for growth. It helps stakeholders think through every phase of a new initiative to ensure that desired learning outcomes are achieved. What follows is a brief description of the eight dimensions:
- The pedagogical dimension addresses issues pertaining to how instructional content is designed, delivered, and implemented, with a strong emphasis on the identification of learners’ needs and how the learning objectives will be achieved. This dimension also addresses the delivery method for the course activities and the appropriateness of the learning environment for achieving the goals of its intended audience. The pedagogical dimension also addresses issues pertaining to the role of data analytics and data mining in learning, their possibilities and limits, and how educators can mine legal and meaningful data sources.
- The technological dimension is concerned with the learning environment, its creation, and the tools required to deliver the learning program. This dimension also addresses hardware and software requirements, as well as infrastructure planning. Technical requirements such as server capacity, bandwidth, security, backups, and other infrastructure issues are also addressed. This is a key consideration in regards to data mining and analytics, as the technological dimension also addresses networking infrastructure issues relating to data volume and transmission.
- The interface design dimension is concerned with factors related to maximizing usability and the user experience. Factors such as web design, content design, navigation, accessibility, and usability testing are addressed in this dimension. The interface design dimension also addresses accessibility and usability issues pertaining to data portals by helping decision makers respond to the question, “Are they usable and accessible by all stakeholders?”
- The evaluation dimension addresses the assessment of learners; evaluation of the instruction and learning environment; assessment of content development processes and of the persons involved in the design process (i.e., the planning, design, production, and evaluation teams); review of instructional design processes (i.e., planning, design, development, and evaluation); and evaluation of learning at the program and institutional levels. The evaluation dimension thereby addresses the quality of data, as well as procedures used in the analysis and mining of data.
- The management dimension deals with issues related to the management of the new initiative or program, such as the continuation, updating, and upkeep. This dimension is used to determine whether the new initiative or program is performing adequately and whether it is meeting its intent. Therefore, the management dimension pertains to data protection and security, scheduling, as well as budgeting for maintenance and upgrades of technology and equipment to support data storage.
- The resource support dimension considers all of the technical and human resource support required to create meaningful and successful environments for the learners. The resource support dimension can address human and technical support issues related to the collection, storage, analysis, and mining of data.
- The ethical dimension identifies the ethical issues that need to be addressed in the design, development, and implementation of courses, new initiatives, and programs. Issues pertaining to social and political influence; diversity; bias; the digital divide; information accessibility; etiquette; and legal issues, such as privacy, plagiarism, and copyright, are also addressed. The ethical dimension then, addresses issues related to the responsible and ethical use of mined data, as well as the protection and anonymity of human subjects.
- The institutional dimension addresses issues pertaining to administrative and academic affairs, as well as student services. Consequently, the institutional dimension addresses issues pertaining to data ownership, how the data are used, and the impact of data-based decisions on students and programs.
Before a new educational initiative is implemented, including the adoption of data analytics and educational data mining for institutional planning and decision-making, each of the aforementioned dimensions should be analyzed to ensure a smooth implementation. Only after thoughtful consideration of the benefits and risks across all affected institutional dimensions can an organization make an informed decision on how to move forward.
The chapters in this book are structured around the eight dimensions of Khan’s Learning Framework.
Section I: Introduction
- Chapter 1: What are Data Analytics and Educational Data Mining? In recent years, learning analytics and educational data mining have emerged to benefit education and the science of learning (Baker & Inventado, 2014). Due to rapid advancements in our ability to collect, process, and analyze massive amounts of data, it is now possible for educators to gain new insights into how people learn. This chapter defines data analytics and data mining and explains how they can be used in educational settings to analyze information, draw conclusions, and make better institutional and program decisions for improving education.
- Chapter 2: A Framework for Implementing Data Analytics in Education. Initially developed in 1997 to address issues pertaining to the successful implementation of e-learning in education and training, Khan’s Learning Framework (2001) has evolved to encompass the design and delivery of effective, efficient, and engaging learning across multiple learning environments and contexts. This chapter introduces the eight dimensions of Khan’s framework and provides a justification for its use as a structure for the organization of the book.
- Chapter 3: Historical and Theoretical Perspectives of Data Analytics and Data Mining in Education. Data mining and data analytics have been used for years by forward-thinking businesses to generate business intelligence for improving decision making (Bienkowski, Feng & Means, 2012). More recently, educators at all levels have begun to use analytics and data mining to improve services and increase student success and retention. This chapter provides a synopsis of significant historical and theoretical perspectives of data analytics and data mining in education. Key theories, innovators, researchers, and events in the development and growth of data analytics in education will be highlighted.
Section II: Pedagogical and Interface Design Issues
- Chapter 4: The Role of Data Analytics in Education: Possibilities and Limitations. While educational data mining has generated numerous innovations for improving student learning, the mining of student data is being increasingly scrutinized by multiple stakeholder groups (Sabourin, Kosturko, FitzGerald, & McQuiggan, 2015). This chapter looks at how K-12 schools and universities are turning to data analytics to predict and monitor student performance to improve learning. It also explores the advantages and risks of using predictive analytics to make decisions about the future of students and programs.
- Chapter 5: Data Sources for Educators: Mining Legal and Meaningful Data for Class and Program Decision Making. While data mining algorithms work best with large data sets, pedagogically meaningful information can be extracted from LMS-generated student tracking data (Macfadyen, 2010). According to Gudivada (2017), there is equal value in collecting and analyzing unstructured data generated by e-learning course communications, such as blogs, discussions, and course messages. This chapter identifies an array of easily accessible data sources available to teachers and instructors to aid them in class and program decision making.
- Chapter 6: Interface Design Issues: Accessibility and Usability of Data Portals in Education. For information to be useful, it needs to be accessible to stakeholders. Data also need to be available in formats that are useful and in formats that are effective to users. This chapter examines the usability of data portals and the information they maintain, as well as their accessibility to administrators and educators.
Section III: Technological and Resource Support Issues
- Chapter 7: Technical Requirements for Data Analytics in Education: Managing Server Capacities, Bandwidth, Security, and Backups. The volumes of data generated through data analytics require special tools and infrastructure to facilitate efficient data management, analysis, validation, visualization, and dissemination (Daniel, 2015). Yet, according to Siemens (as cited in Waters, 2011, para. 20), “in order for learning analytics to have a broad impact in education, the focus needs to move well beyond basic analytics techniques such as those found in Google Analytics.” This chapter addresses the technical requirements needed for the development and implementation of data analytics in educational institutions, including issues concerning managing server capacities, bandwidth, security, and backups. The chapter also addresses cost effective solutions and alternatives.
- Chapter 8: Supporting Data Analytics in Education: Human and Technical Resources Needed for Collecting, Storing, Analyzing, and Mining Data. To capitalize on the power of data analytics and data mining in educational settings, school districts and universities need to build up their analytics capability. In addition to enhancing the technological infrastructure needed to support data management and storage, institutions also need to invest in data-savvy people and processes needed for data preparation, processing, and analysis (Wegener & Sinha, 2013). This chapter addresses the human and technical infrastructure needed to support and maintain the safe and careful collection, storage, analysis, and mining of data in educational environments.
- Chapter 9: Moving Towards Semantic Interoperability through the Adoption of Open Standards. Interoperability is the ability of two or more systems to exchange and interpret shared data. For two systems to be interoperable, they must be able to exchange and present data in such a way that they can be understood and used. According to Miller (2000), this will require drastic changes to how organizations work, as well as their attitudes toward information sharing. To minimize barriers to interoperability, this chapter makes the case for school districts and educational institutions to move towards semantic interoperability through the adoption of open standards.
Section IV: Evaluation and Ethical Issues
- Chapter 10: Assessing Data Quality: Determining What Data to Trust and Use. High-quality data are the precondition for analyzing, using, and guaranteeing the value of the data. Unfortunately, currently available quality standards and quality assessment methods are inadequate (Cai & Zhu, 2015). This is especially true in education, where funding for assessing educational quality is limited. This chapter addresses the factors that determine data quality and how educators and school administrators can determine what data to trust and use.
- Chapter 11: Ethical Issues and Potential Unintended Consequences of Databased-Decision Making. Educational decision makers need access to high quality data that communicate and promote understanding of complex issues (Stiles, 2012). Yet, even with quality data, decisions based on faulty analysis or interpretation may yield unexpected consequences. This chapter addresses some of the ethical issues and potential unintended consequences of data-based decision making in education.
Section V: Management and Institutional Issues
- Chapter 12: Data Cybersecurity Issues: Contingency Plans for Handling Data Breaches. Cybercrime is on the rise and no institution is immune. According to a 2015 Symantec Internet security threat report (Symantec Corporation), 10% of all data breaches in 2013 involved educational institutions. The report added that in 31 identified incidents over 1.3 million identities of students and school workers were exposed. This chapter will address the threats and challenges to data security in educational environments. It will also address tools and best practices institutions can adopt to attain a balance between data protection and responsible data usage.
- Chapter 13: Ensuring Due Process for Data-Driven Errors. Establishing Policies to Address Errors. Although data-driven decision making has the potential to improve learning and increase educational access and retention, stakeholders must be careful not to put too much trust in predictive analytics. According to Alarcon, et al., (n.d.), data-driven education has the potential to perpetuate persistent, and potentially inaccurate, labeling of students. In such cases, what avenues do students have to question the accuracy of data-driven decisions affecting their educational future? This chapter will address best practices for ensuring due process for data-driven errors and recommended policies for addressing such errors.
Objective of the Book
As we consider the enormous potential of data analytics and data mining in education, we must also recognize many emerging issues in order to implement them responsibly. While the subject areas addressed in the proposed book may have been addressed individually through multiple channels, no current textbook brings all the major issues and challenges related to the responsible use of data analytics and mining in education into a single volume. This book is unique in that it will specifically address the promises and challenges of data analytics and data mining in education, as well as the issues pertaining to the thoughtful and purposeful implementation of data analytics and data mining in education. While the focus of the book will have a greater appeal at the graduate level, it may also be appropriate for some undergraduate courses.
This book will be of great interest to educators interested in using learning analytics and educational data mining for targeted recruitment, data-based decision making, program improvement, assessment, and accreditation reporting. The book will provide a comprehensive framework for policy-makers, administrators, faculty, grant writers, researchers, and IT support staff to identify and address issues pertaining to how data analytics can be used to improve the quality of courses and programs; technological and resource support issues pertaining to networking capacity and infrastructure, as well as issues pertaining to accessibility and use; institutional and management issues related to ownership of data and decisions on how the data and its interpretations may inadvertently impact students, faculty, and the institution; issues pertaining to quality and reliability of data and accuracy of data based decisions; and, ethical implications surrounding the collection, distribution, and use of student-generated data.
This book is expected to include 15 chapters, making it convenient for use in traditional semester-long university courses. The topics addressed in the book are of great interest for graduate, doctoral, and professional courses in Educational/Instructional Technology, Educational Leadership, Educational Assessment and Data-Driven Decision Making, School Law, and Educational Research. This book would also be an excellent choice for a Selected Topics course addressing data analytics and data mining in education.
Taylor & Francis Group
711 Third Avenue
New York, NY 10017
Alarcon, A., Zeide, E., Rosenblat, A., Wikelius, K., Boyd, D., Pena Garngadharan, S., & Yu, C. (2014). Data & Society Research Institute, 1-8. Retrieved from http://www.datacivilrights.org/pubs/2014-1030/Education.pdf
Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning Analytics (pp. 61-75). Springer New York.
Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief, 1-57. U.S. Department of Education, Office of Educational Technology.
Bainbridge, J., Melitski, J., Zahradnik, A., Lauria, E., Jayaprakash, S., Baron, J. (2015). Using learning analytics to predict at-risk students in online graduate public affairs and administration education. Journal of Public Affairs Education, 21(2), 247-262.
Cai, L., & Zhu, Y. (2015). The challenges of data quality and data quality assessment in the big data era. Data Science Journal, 14, 2.
Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904-920.
Khan, B. H. (2001). A framework for web-based learning. In B. H. Khan (Ed.), Web-based Training, Englewood Cliffs, New Jersey: Educational Technology Publications.
Khan, B. H. (2015). A Framework for Massive Open Online Courses. In J. R. Corbeil, M. E. Corbeil, & B. H. Khan (Eds.). The MOOC case book: Case studies in MOOC design, development and implementation. 3-16. Ronkonkoma, NY: Linus Books.
Gudivada, V. (2017). Cognitive analytics driven personalized learning. In Corbeil, J. R., Corbeil, M. E. & Khan, B. H. (Eds). Educational Technology Magazine: The Magazine for Managers of Change in Education, 57(1). Englewood Cliffs, NJ: Educational Techn. Publ.
Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588- 599.
Miller, P. (2000). Interoperability: What is it and why should I want it? Ariadne, (24).
Sabourin, J., Kosturko, L., FitzGerald, C., & McQuiggan, S. (2015). Student privacy and educational data mining: Perspectives from industry. International Educational Data Mining Society.
Symantec Corporation. (2015, April). Internet security threat report, Volume 20. Retrieved January 2, 2017, from https://www.symantec.com/content/en/us/enterprise/other_resources/21347933_GA_RPT-internet-security-threat-report-volume-20- 2015.pdf
Stiles, R. J. (2012). Understanding and managing the risks of analytics in higher education: A guide. Educause Review. Available at: http://net.educause. edu/ir/library/pdf/EPUB1201.pdf
Watters, A. (2011). How data and analytics can improve education. O’Reilly Radar.
Wegener, R., & Sinha, V. (2013). The value of big data: How analytics differentiates winners. Bain & Company.