by TED Mining · 2012 · Cited by 752 — Through Educational Data Mining and Learning Analytics: Teachers gain views into students’ performance behavior data exposed a prototypical.

547 KB – 77 Pages

PAGE – 2 ============
Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief U.S. Department of Education Office of Educational Technology Prepared by: Marie Bienkowski Mingyu Feng Barbara Means Center for Technology in Learning SRI International October 2012

PAGE – 3 ============
This report was prepared for the U.S. Department of Education under Contract Number ED-04- CO-0040, Task 0010, with SRI International. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education is intended or should be inferred. U.S. Department of Education Arne Duncan Secretary Office of Educational Technology Karen Cator Director October 2012 This report is in the public domain. Authorization to reproduce this report in whole or in part is granted. While permission to reprint this publication is not necessary, the suggested citation is: U.S. Department of Education, Office of Educational Technology, Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief, Washington, D.C., 2012. This report is available on the Department™s Web site at http://www.ed.gov/technology . On r equest, this publication is available in alternate formats, such as Braille, large print, or compact disc . For more information, please contact the Department™s Alternate Format Center at (202) 260-0852 or (202) 260-0818. Technical Contact Bernadette Ada ms bernadette.adams@ed.gov

PAGE – 5 ============
iii Contents List of Exhibits iv Acknowledgments . v Executive Summary vii Introduction .. 1 Personalized Learning Scenarios . 5 Data Mining and Analytics: The Research Base 7 Educational Data Mining .. 9 Learning Analytics . 13 Visual Data Analytics .. 15 Data Use in Adaptive Learning Systems 17 Educational Data Mining and Learning Analytics Applications 25 User Knowledge Modeling 28 User Behavior Modeling . 29 User Experience Modeling 30 User Profiling .. 32 Domain Modeling .. 33 Learning System Components and Instructional Principle Analysis 34 Trend Analysis 35 Adaptation and Personalization . 35 Implementation Challenges and Considerations 37 Technical Challenges .. 38 Limitations in Institutional Capacity .. 40 Privacy and Ethics Issues . 41 Recommendations . 45 Educators .. 46 Researchers and Developers . 49 Collaborations Across Sectors 50 Conclusion . 51 References . 53 Selected Reading 59 Selected Websites . 63

PAGE – 6 ============
iv Exhibits Exhibit 1. The Components and Data Flow Through a Typical Adaptive Learning System .. 18 Exhibit 2. Student Dashboard Showing Recommended Next Activities . 19 Exhibit 3. Teacher Dashboard With Skill Meter for Math Class .. 20 Exhibit 4. Administrator Dashboard Showing Concept Proficiency for a Grade Leve l . 21 Exhibit 5 Application Areas for Educational Data Mining and Learning Analytics . 26

PAGE – 9 ============
vii Executive Summary In data mining and data analytics, tools and techniques once confined to research laboratories are being adopted by forward- looking industries to generate business intelligence for improving decision making. Higher education institutions are beginning to use analytics for improving the services they provide and for increasin g student grades and retention. The U.S. Department of Education™s National Education Technology Plan, as one part of its model for 21 st-century learning powered by technology, envisions ways of using data from online learning systems to improve instruction. With a nalytics and data mining experiments in education starting to proliferate, sorting out fact from fiction and identifying research possibilities and practical applications are not easy. This issue brief is intended to help policymakers and administrators understand how analytics and data mining have beenŠand can beŠapplied for educational improvement. At present, educational data mining tends to focus on developing new tools for discovering patterns in data. These patterns are generally about the microconcepts involved in learning: one -digit multiplication, subtraction with carries, and so on. Learning analyticsŠ at least as it is currently contrasted with data mining Šfocuses on applying tools and techniques at larger scales, such as in courses and at schools and postsecondary institutions. But both disciplines work with patterns and prediction: If we can discern the pattern in the data and make sense of what is happening, we can predict what should come next and take the appropriate action. Educational data mining and learning analytics are used to research and build models in several areas that can influence online learning systems. One area is user modeling, which encompasses what a learner knows, what a learner™s behavior and motivation are, what the user experience is like, and how satisfied users are with online learning. At the simplest level, analytics can detect when a student in an online course is going astray and nudge him or her on to a course correction. At the most complex, they hold pro mise of detect ing boredom from patterns of key clicks and redirect ing the student™s attention. Because these data are gathered in real time, there is a real possibility of continuous improvement via multiple feedback loops that operate at different time sc ales Šimmediate to the student for the next problem, daily to the teacher for the

PAGE – 10 ============
viii next day™s teaching , monthly to the principal for judging progress, and annually to the district and state administrators for overall school improvement. The same kind s of data that inform user or learner models can be used to profile users. Profiling as used here means grouping similar users into categories using salient characteristics. These categories then can be used to offer experiences to groups of users or to mak e recommendations to the users and adaptation s to how a system performs. User modeling and profiling are suggestive of real-time adaptations. In contrast, some applications of data mining and analytics are for more experimental purposes. Domain modeling is largely experimental with the goal of understanding how to present a topic and at what level of detail. The study of learning components and instructional principles also uses experimentation to understand what is effective at promoting learning. These examples suggest that the actions from data mining and analytics are always automatic, but that is less often the case. Visual data analytics close ly involve humans to help make sense of data, from initial pattern detection and m odel building to sophisticated data dashboards that present da ta in a way that humans can act upon. KŒ12 schools and school districts are starting to adopt such institution -level analyses for detecting areas for instructional improvement, setting policies, and measuring results. Making visible students™ learning and assessment activities opens up the possibility for students to develop skills in monitoring their own learning and to see directly how their effort improves their success. Teachers gain views into students™ performance that help them adapt their teaching or initiate tutoring, tailored assignments, and the like. Robust applications of educational data mining and learning analytics techniques come with costs and challenges. Information technology (IT) departments will unders tand the costs associated with collecting and storing logged data, while algorithm developers will recognize the computational costs these techniques still require. Another technical challenge is that educational data systems are not interopera ble , so brin ging together administrative data and classroom -level data remains a challenge. Yet combining these data can give algorithms better predictive power. Combining data about student performanceŠ online tracking, standardized tests, teacher -generated test sŠto form one simplified picture of what a student knows can be difficult and must meet acceptable standards for validity. It also requires careful attention to student and teacher privacy and the ethical obligations associated with knowing and acting on student data.

PAGE – 11 ============
ix Educational data mining and learning analytics have the potential to make visible data that have heretofore gone unseen, unnoticed, and therefore unactionable. To help further the fields and gain value from their practical applications, the recom mendations are that educators and administrators : Develop a culture of using data for making instructional decisions. Involve IT departments in planning for data collection and use. Be smart data consumers who ask critical questions about commercial offeri ngs and create demand for the most useful features and uses . Start with focused areas where data will help, show success, and then expand to new areas . Communicate with students and parents about where data come from and how the data are used . Help align state policies with technical requirements for online learning systems . Researchers and software developers are encouraged to : Conduct research on usability and effectiveness of data displays. Help instructors be more effective in the classroom with more real -time and data -based decision support tools, including recommendation services. Continue to research methods for using identified student information where it will help most, anonymizing data when required, and understanding how to align data across different systems . Understand how to repurpose predictive models developed in one context to another. A final recommendation is to create and continue strong collaboration across research, commercial, and educational sectors. Commercial companies operate on fast development cycles and can produce data useful for research. Districts and schools want properly vetted learning environments. Effective partnerships can help these organizations codesign the best tools.

547 KB – 77 Pages