Paper presentation @ PLoP 2015 writer’s workshop

Paul Inventado and Peter Scupelli presented “A Data-driven Methodology for Producing Online Learning System Design Patterns” at the 22nd Conference on Pattern Languages of Programs (PLoP) 2015 in Pittsburgh, Pennsylvania.

Abstract:

Online learning systems are complex systems that are frequently updated with new content, upgraded to support new features and extended to support new technologies. Maintaining the quality of the system as it changes is a challenge that needs to be addressed. Design patterns offer a solution to this challenge by providing guides to stakeholders responsible for making design changes (e.g., system developers, HCI designers, teachers, students) that will help them ensure system quality despite changes. Although design patterns for online learning systems exist, they often focus on one aspect of the system (e.g., pedagogy, learning). The data-driven design pattern production (3D2P) methodology utilizes data for producing design patterns in collaboration with stakeholders, addresses stakeholders’ concerns, and ensures the system’s quality as a whole. The paper presents five patterns produced by applying the methodology on the ASSISTments online learning system namely: all content in one place, just enough practice, personalized problems, worked examples, and consistent language. We made two changes to the pattern format: added in-text references in the forces section, and added an evidence section. The references allow the reader to learn more about the force in question. The evidence section highlights key findings uncovered from the 3D2P methodology. Four sources of evidence were considered in the pattern format: (a) literature – existing research on the problem or solution, (b) discussion – expert opinions about the problem or solution, (c) data – measures of the problem’s recurrence, and the solution’s effectiveness based on collected data; and (d) evaluation – assessment of the pattern’s performance when it was applied on an existing system. The changes to the format highlight linkages between pattern elements, theory, and empirical evidence. We believe that links further justify the design pattern, and make it easier for multiple stakeholders to understand them.