Filter by type:

All types ( 64 ) Book chapters ( 7 ) Conference publications (Peer-Reviewed) ( 21 ) Journal publications (peer-reviewed) ( 36 )
Sort by year:

Eliciting the challenges and opportunities organizations face when delivering open online education: a group-concept mapping study

Journal publications (peer-reviewed)
http://www.ou.nl
Schophuizen, M., Kreijns, K., Stoyanov, S., & Kalz, M. (2018). Eliciting the challenges and opportunities organizations face when delivering open online education: a group-concept mapping study. The Internet and Higher Education. 36, 1-12. doi:10.1016/j.iheduc.2017.08.002

The global attention for open online education (OOE) caused a situation in which higher education institutions (HEIs) reconsider the way they deliver education to the population. With a funding policy, the Dutch Government aims to stimulate OOE in HEIs. The goal is to create more expedient, accessible and personalized learning experiences, that contribute to an improvement of quality of education and study success. However, many projects are failing to embed OOE within the institution. In this study, we elicited the challenges and opportunities of OOE projects within an organizational context of Dutch HEIs by using group concept mapping. Multidimensional scaling and hierarchical clustering resulted in a cluster map and a pattern match graph for interpreting the experts’ ideas and opinions, clarifying and structuring the collective understanding. Core themes that represent the challenges and opportunities with regard to OOE identified in this study were: 1. Online teaching, 2. Supporting mechanisms, 3. Assessment, 4. External target groups, 5. Educational flexibility, 6. Quality of education, 7. Institutional reputation, and 8. Educational efficiency. The results indicated a skills gap among educators and a lack of central support for the development of OOE. Organizational efforts to implement OOE should take educational flexibilityand online teaching into account and support mechanisms for OOE should be provided.

A Framework Towards Educational Scalability of Open Online Courses

Journal publications (peer-reviewed)
Kasch, J., Van Rosmalen, P. & Kalz, M. (2017). A Framework Towards Educational Scalability of Open Online Courses. Journal of Universal Computer Science. 23 (9). 845-867.

Although the terms scale and scalable are often used in the context of Open Online Education (OOE), there is no clear definition about these concepts from an educational perspective on the course level. This paper critically discusses the origins of these concepts and provides a working definition for educational scalability. A heuristic framework, which integrates four common educational design principles, is introduced, in order to study support and formative assessment and feedback at large scale. The proposed framework is presented, discussed and applied to five case studies. First qualitative results of the case studies show that the designs are relatively similar. The detailed study of their units of learning, however, indicates practices which can potentially be interesting for other MOOC developers to enhance their design and their scalability. Further research will apply the framework to zoom in on scalable best practices in MOOCs with a focus on scalable practices of formative assessment and feedback.

A review of the types of mobile activities in mobile inquiry-based learning

Journal publications (peer-reviewed)
Suarez, A., Specht, M., Prinsen, F., Kalz, M., & Ternier, S. (2017). A review of the types of mobile activities in mobile inquiry-based learning. Computers & Education. 118, March 2018, 38-55. doi:10.1016/j.compedu.2017.11.004

Inquiry-based Learning is increasingly suggested as an efficient approach for fostering learners’ curiosity and motivation. It helps learners to develop their ability to work in complex and unpredictable environments making them more critical thinkers and agentic learners. Although mobile technology is a suitable support for this learning process, there is a lack of practical strategies for educational practitioners to enact the right balance between enabling agency and supporting the students through the mobile technology. Thus, we conducted a literature review that analyzed 62 studies on mobile inquiry-based learning. The analysis focused on the level of agency supported by mobile technology. This review study provided two main results. The first result is a two-layer classification –with five types and twelve subtypes– of the most common mobile activities used in inquiry-based learning. The types and subtypes are: 1) Direct instruction formed by 1a) location guidance, 1b) procedural guidance and 1c) metacognitive guidance, 2) Access to content formed by 2a) fixed and 2b) dynamic content, 3) Data collection that consists of 3a) cooperative and 3b) collaborative data collection, 4) Peer-to-peer communication formed by 4a) asynchronous and 4b) synchronous social communications and 5) Contextual support that includes 5a) augmented experience, 5b) immersive experience and 5c) adaptive feedback. The second result consists of an analytical framework –based on six dimensions– to assess the level of agency supported by the different types of mobile activities. The learners’ agency dimensions are: 1) Goals, 2) Content, 3) Actions, 4) Strategies, 5) Reflection and 6) Monitoring. Finally, the review presents insights on how this analytical framework can be used by educational practitioners to identify mobile activities that effectively balance learners’ agency with mobile technology.

Solutions for global marine litter pollution

Journal publications (peer-reviewed)
Lohr, A., Savelli, H., Beunen, R., Kalz, M., Ragas, A., & van Belleghem, F. (2017). Solutions for global marine litter pollution. Current Opinion in Environmental Sustainability. 28, October 2017. 90-99. doi:10.1016/j.cosust.2017.08.009

Since the 1950s the amount of plastics in the marine environment has increased dramatically. Worldwide there is a growing concern about the risks and possible adverse effects of (micro)plastics. This paper reflects on the sources and effects of marine litter and the effects of policies and other actions taken worldwide. Current knowledge offers a solid basis for effective action. Yet, so far the effects of policies and other initiatives are still largely insufficient. The search for appropriate responses could be based on possible interventions and profound understanding of the context specific factors for success. Moreover, the scope, timeframe and dynamics of all initiatives are distinctly different and orchestration at all levels, in close cooperation with one another is currently lacking.

Refining Success and Dropout in Massive Open Online Courses

Journal publications (peer-reviewed)
Henderikx, M., Kreijns, K. ,& Kalz, M. (2017). Refining Success and Dropout in Massive Open Online Courses. Distance Education. 38 (3). 353-368. doi:10.1080/01587919.2017.1369006

In this paper we present an alternative typology for determining success and dropout in massive open online courses (MOOCs). This typology takes the perspectives of MOOC-takers into account and is based on the their intentions and subsequent behaviour. An explorative study using two MOOCs was carried out to test the applicability of the typology. Following the traditional approach based on course completion to identify educational success, success rates were 6.5 and 5.6%. The success rates from the perspectives of the MOOC-taker were 59 and 70%. These findings demonstrate that merely looking at course completion as a measure for success does not suffice in the context of MOOCs. This change in addressing MOOC success and dropout provides an alternative view and demonstrates the importance of MOOC-takers’ perspectives.

MoocCast: Evaluating Mobile Screencast for Online Courses

Journal publications (peer-reviewed)
Tabuenca, B., Kalz, M., & Löhr, A. (2017). MoocCast: Evaluating Mobile Screencast for Online Courses. Universal Access in the Information Society.1-9. doi:10.1007/s10209-017-0528-x

The progressive adoption of smartphones and interconnected devices is inspiring students to redesign their physical spaces towards a seamless shift between daily life and learning activities. In the last years, Massive Open Online Courses (MOOCs) represent one of the key resources facilitating universal access to education as they enable students to learn across formal and informal contexts. However, there is little research exploring these alternative ways to present the content offered to learners in MOOCs for improved accessibility from personal contexts. This work presents the first study evaluating mobile-screencast technology as a means to facilitate learning processes in online courses. The contribution from this manuscript is threefold: first, preferred learning spaces for students enrolled to accomplish the activities in a MOOC are identified; second, mobile-screencast is evaluated as a solution for improved accessibility in online courses; third, an open tool for mobile-screencast and initial results from a formative evaluation are presented. This tool can be reused and adapted in further MOOC implementations. Finally, lessons learned are discussed and cues for future implementations are challenged.

Validation of the self-regulated online learning questionnaire

Journal publications (peer-reviewed)
Jansen, R. S., Van Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2017). Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education. 29 (1). 6 – 27. doi:10.1007/s12528-016-9125-x

The number of students engaged in Massive Open Online Courses (MOOCs) is increasing rapidly. Due to the autonomy of students in this type of education, students in MOOCs are required to regulate their learning to a greater extent than students in traditional, face-to-face education. However, there is no questionnaire available suited for this online context that measures all aspects of self-regulated learning (SRL). In this study, such a questionnaire is developed based on existing SRL questionnaires. This is the self-regulated online learning questionnaire. Exploratory factor analysis (EFA) on the first dataset led to a set of scales differing from those theoretically defined beforehand. Confirmatory factor analysis (CFA) was conducted on a second dataset to compare the fit of the theoretical model and the exploratively obtained model. The exploratively obtained model provided much better fit to the data than the theoretical model. All models under investigation provided better fit when excluding the task strategies scale and when merging the scales measuring metacognitive activities. From the results of the EFA and the CFA it can be concluded that further development of the questionnaire is necessary.

Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey

Journal publications (peer-reviewed)
Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29 (1). 28–46. doi:10.1007/s12528-016-9123-z

While MOOCs are recognized nowadays as a potential format for professional development and lifelong learning, little research has been conducted on the factors that influence MOOC participation of professionals and unemployed in MOOCs. Based on a framework developed earlier, we conducted a study, which focused on the influence of background variables such us digital competence, age, gender and educational level on MOOC participation. Occupational setting was considered as a moderator in the analysis of the impact of digital skills. Results of the study showed that MOOCs were an important tool for unemployed participants who were more likely to enroll in MOOCs than employed learners. MOOCs were also a way for workers who do not received employer support for other training activities to get professional development training. Results of the regression analysis showed that a person’s level of digital competence was an important predictor for enrolment in MOOCs and that specifically interaction skills were more important than information skills for participating in the MOOC context.

Academic domains as political battlegrounds: A global enquiry by 99 academics in the fields of education and technology.

Journal publications (peer-reviewed)
Al Lily…Kalz.. et al. (2017). Academic domains as political battlegrounds: A global enquiry by 99 academics in the fields of education and technology. Information Development. 33 (3). 270 – 288. doi: 10.1177/0266666916646415

This article theorizes the functional relationship between the human components (i.e., scholars) and non-human components (i.e., structural configurations) of academic domains. It is organized around the following question: in what ways have scholars formed and been formed by the structural configurations of their academic domain? The article uses as a case study the academic domain of education and technology to examine this question. Its authorship approach is innovative, with a worldwide collection of academics (99 authors) collaborating to address the proposed question based on their reflections on daily social and academic practices. This collaboration followed a three-round process of contributions via email. Analysis of these scholars’ reflective accounts was carried out, and a theoretical proposition was established from this analysis. The proposition is of a mutual (yet not necessarily balanced) power (and therefore political) relationship between the human and non-human constituents of an academic realm, with the two shaping one another. One implication of this proposition is that these non-human elements exist as political ‘actors’, just like their human counterparts, having ‘agency’ – which they exercise over humans. This turns academic domains into political (functional or dysfunctional) ‘battlefields’ wherein both humans and non-humans engage in political activities and actions that form the identity of the academic domain.

The MOOC and learning analytics innovation cycle (MOLAC): a reflective summary of ongoing research and its challenges

Journal publications (peer-reviewed)
Drachsler, H., & Kalz, M. (2016). The MOOC and learning analytics innovation cycle (MOLAC): a reflective summary of ongoing research and its challenges. Journal of Computer Assisted Learning, 32(3), 281-290. doi: 10.1111/jcal.12135

This article describes the MOOCS and Learning Analytics framework (MOLAC). Based on a brief review of ongoing challenges in the field, the article develops a vision for the future use of MOOCs and Learning Analytics to foster educational innovation.

Time will tell: The role of mobile learning analytics in self-regulated learning

Journal publications (peer-reviewed)
Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education.89, 53-74. doi: 10.1016/j.compedu.2015.08.004

This longitudinal study explores the effects of tracking and monitoring time devoted to learn with a mobile tool, on self-regulated learning. Graduate students (n = 36) from three different online courses used their own mobile devices to track how much time they devoted to learn over a period of four months. Repeated measures of the Online Self-Regulated Learning Questionnaire and Validity and Reliability of Time Management Questionnaire were taken along the course. Our findings reveal positive effects of tracking time on time management skills. Variations in the channel, content and timing of the mobile notifications to foster reflective practice are investigated, and time-logging patterns are described. These results not only provide evidence of the benefits of recording learning time, but also suggest relevant cues on how mobile notifications should be designed and prompted towards self-regulated learning of students in online courses.

Establishing a European cross-provider data collection about open online courses

Journal publications (peer-reviewed)
Kalz, M., Kreijns, K., Walhout, J., Castaño-Munoz, J., Espasa, A., & Tovar, E. (2015) Establishing a European cross-provider data collection about open online courses. The International Review of Research in Open and Distributed Learning (IRRODL). 16 (6), 62-77.

While MOOCS have emerged as a new form of open online education around the world, research is still lagging behind to come up with a sound theoretical basis that can cover the impact of socio- economic background variables, ICT competences, prior experiences and lifelong learning profile, variance in intentions, environmental influences, outcome expectations, learning experience, and economic return on taking and completing Massive Open Online Courses (MOOCs). The potential diversity of participants of MOOCs has been taken as a starting point to develop a theoretical model and survey instrument with the goal to establish a large-scale, cross-provider data collection of participants of (European) MOOCs. This article provides an overview of the theoretical model, the start-phase of the project, and reflects on first experiences with the cross- provider data collection.

Tap it again, Sam: Harmonizing personal environments towards lifelong learning

Journal publications (peer-reviewed)
Tabuenca, B., Kalz, M., & Specht, M. (2015). Tap it again, Sam: Harmonizing personal environments towards lifelong learning. International Journal of Advanced Corporate Learning, 8 (1), 16-23. doi: 10.3991/ijac.v8i1.4383

The increasing number of mobile vendors releasing NFC-enabled devices to the market and their prominent adoption has moved this technology from a niche product to a product with a large market-share. NFC facilitates natural interactions between digital world and physical learning environments. The scaffolding of learning ecologies is a key aspect for lifelong learners in their challenge to integrate learning activities into busy daily life. The contribution of this manuscript is twofold: first, a review of scientific litera- ture in which NFC has been used with a direct or indirect purpose to learn is presented, and potential uses for learners are classified according to their type of interaction; based on these findings the NFC MediaPlayer is presented as an instantiation of an ecology of resources (EoR) in a lifelong learning context. Finally, shortcomings and best practices are highlighted in the conclusions, and future work is discussed.

Stop and think: Exploring mobile notifications to foster reflective practice on meta-learning

Journal publications (peer-reviewed)
Tabuenca, B., Kalz, M., Ternier, S., & Specht, M. (2014). Stop and think: Exploring mobile notifications to foster reflective practice on meta-learning. IEEE Transactions on Learning Technologies. Special Issue on Seamless, Ubiquitous, and Contextual Learning. (pp. 1-13). doi: 10.1109/TLT.2014.2383611.

Nowadays, smartphone users are constantly receiving notifications from applications that provide feedback, as reminders, recommendations or announcements. Nevertheless, there is little research on the effects of mobile notifications to foster meta-learning. This paper explores the effectiveness of mobile notifications to foster reflection on meta-learning by presenting the results of two studies: 1) a formative study with 37 secondary school students offering a daily reflection and reporting exercise about their learning experience during the day; 2) an experiment involving 60 adults to read an eBook on energy-efficient driving for one hour. During that time the participants received mobile notifications inviting them to reflect in-action. On the one hand, the results from the first study show that students do not have a habit to see themselves as learners and to develop a “professional” awareness about their daily activity at work/school. On the other hand, the second study explores the effects of different notification types on knowledge gain and motivation. Results envision a higher knowledge gain and motivation for the group assigned with the least complex interactions with mobile devices during the reflection exercise. Finally, these results are discussed and important research questions for future research on mobile notifications are raised.

Mobile authoring of open educational resources for authentic learning scenarios

Journal publications (peer-reviewed)
Tabuenca, B., Kalz, M., Ternier, S., & Specht, M. (2014). Mobile authoring of open educational resources for authentic learning scenarios. Universal Access in the Information Society. 1-15. 15 (3). 329 – 343. doi: 10.1007/s10209-014-0391-y

The proliferation of smartphones in the last decade and the number of publications in the field of authoring systems for computer-assisted learning depict a scenario that needs to be explored in order to facilitate the scaffolding of learning activities across contexts. Learning resources are traditionally designed in desktop-based authoring systems where the context is mostly restricted to the learning objective, capturing relevant case characteristics, or virtual situation models. Mobile authoring tools enable learners and teachers to foster universal access to educational resources not only providing channels to share, remix or re-contextualize these, but also capturing the context in-situ and in-time. As a further matter, authoring educational resources in a mobile context is an authentic experience where authors can link learning with their own daily life activities and reflections. The contribution of this manuscript is fourfold: first, the main barriers for ubiquitous and mobile authoring of educational resources are identified; second, recent research on mobile authoring tools is reviewed, and 10 key shortcomings of current approaches are identified; third, the design of a mobile environment to author educational resources (MAT for ARLearn) is presented, and the results of an evaluation of usability and hedonic quality are presented; fourth, conclusions and a research agenda for mobile authoring are discussed.

A study about placement support using semantic similarity

Journal publications (peer-reviewed)
Kalz, M, van Bruggen, J., Giesbers, B., Waterink, W., Eshuis, J., & Koper, R. (2014). A study about placement support using semantic similarity. Educational Technology & Society. 17 (3). 54-64. doi: www.jstor.org/stable/jeductechsoci.17.3.54

This paper discusses Latent Semantic Analysis (LSA) as a method for the assessment of prior learning. The Accreditation of Prior Learning (APL) is a procedure to offer learners an individualized curriculum based on their prior experiences and knowledge. The placement decisions in this process are based on the analysis of student material by domain experts, making it a time-consuming and expensive process. In order to reduce the workload of these domain experts we are seeking ways in which the preprocessing and selection of student submitted material can be achieved with technological support. This approach can at the same time stimulate research about assessment in open and networked learning environments. The study was conducted in the context of a Psychology Course of the Open University of the Netherlands. The results of the study confirm our earlier findings regarding the identification of the ideal number of dimensions and the use of stopwords for small-scale corpora. Furthermore the study indicates that the application of the vector space model and dimensionality reduction produces a well performing classification model for deciding about relevant documents for APL procedures. Together we discuss methodological issues and limitations of our study whilst also providing an outlook on future research in this area.

Improving The Learning Design of Massive Open Online Courses

Journal publications (peer-reviewed)
Rubens, W., Kalz, M., & Koper, R. (2014). Improving The Learning Design of Massive Open Online Courses. The Turkish Online Journal of Educational Technology (TOJET), 13(4), 71-80.

Massive Open Online Courses (MOOCs) can be regarded as a promising next step in the evolution of distance education. However, they have been criticised for their poor learning design. This article describes the development of an adequate learning design in a series of nineteen MOOCs (called online master classes). A formative evaluation focuses on participation and user satisfaction. A total amount of 2083 individual learners enrolled in online master classes. Overall the user satisfaction is positive and stable. Thirteen pedagogical requirements for MOOCs form the output of this evaluation. It is concluded that the learning design that has been developed, matches with the pedagogical principles of distance education for adult learners. The format has proven to support more a diverse group of learners than the still dominant MOOC formats.

It Doesn’t Matter, But: Examining the impact of ambient learning displays on energy consumption and conservation at the workplace

Journal publications (peer-reviewed)
Börner, D., Kalz, M., & Specht, M. (2014). It Doesn’t Matter, But: Examining the impact of ambient learning displays on energy consumption and conservation at the workplace. Environmental Education Research. doi: 10.1080/13504622.2014.921804.

This study reports an intervention to initiate environmental learning and facilitate pro-environmental behaviour. The purpose was to examine the impact of ambient learning displays on energy consumption and conservation at the workplace, more specifically the evaluation of learning outcome and behaviour change. Using a quasi-experimental design, the empirical study was conducted among employees working at a university campus. For the experimental treatments, ambient learning display prototypes were varied on two design dimensions, namely representational fidelity and notification level. The results do not provide clear evidence that the design of the displays influences learning outcome or that the displays lead to pro-environmental behaviour change. Nevertheless, the sole deployment of the display prototypes eased the comprehension of the information provided and lowered the need for additional information. Thus, ambient learning displays provide a promising framework in the context of environmental learning and beyond.

Lead me gently: Facilitating knowledge gain through attention-aware ambient learning displays

Journal publications (peer-reviewed)
Börner, D., Kalz, M., & Specht, M. (2014). Lead me gently: Facilitating knowledge gain through attention-aware ambient learning displays. Computers & Education. Volume 78, September 2014. 10-19, doi: 10.1016/j.compedu.2014.04.017.

This empirical study reports an intervention to investigate identified research challenges on the evaluation and use of ambient displays in a learning context with the objective to gain insights into the interplay between display design, user attention, and knowledge acquisition. The main research questions were whether an attention-aware display design can capture the user’s focus of attention and whether this has an influence on the knowledge gain. A display prototype corresponding to the main ambient display characteristics was designed, applied in a controlled authentic setting, and evaluated accordingly. The prototype presented information and guidelines for first responders in emergency situations, especially in cases of cardiac arrest. The prototype was enhanced with a custom-built sensor to measure user attention and trigger interruptive notifications. The study was conducted among 52 employees working at a university campus. Using an experimental research design, a treatment group exposed to an attention-aware display design was compared to a control group. The results provide evidence that such a display design can attract and retain attention in such a way that the acquisition of knowledge (i.e. the comprehension of the presented information) is effectively facilitated.

Assessing the crossdisciplinarity of technology-enhanced learning with science overlay maps and diversity measures

Journal publications (peer-reviewed)
Kalz, M., & Specht, M. (2014). Assessing the crossdisciplinarity of technology-enhanced learning with science overlay maps and diversity measures. British Journal of Educational Technology, 45(3), 415–427. doi:10.1111/bjet.12092

This paper deals with the assessment of the crossdisciplinarity of technology-enhanced learning (TEL). Based on a general discussion of the concept interdisciplinarity and a summary of the discussion in the field two empirical methods from scientometrics are introduced and applied. Science overlay maps and the Rao-Stirling-Diversity index are used to analyze the TEL field with a scientometric analysis. The science overlay maps show that a wide variety of disciplines contribute to research in the field. The analysis reveals that the field has been operating on a relatively high level of crossdisciplinarity in the last 10 years compared to 6 other fields of reference. Only in 2004 a decrease in the level of crossdisciplinarity could be identified.

Toward a learner-centered system for adult learning

Journal publications (peer-reviewed)
Hermans, H., Kalz, M., & Koper, R. (2014). Toward a learner-centered system for adult learning. Campus-Wide Information Systems, 31(1), 2–13.

This paper presents an e-learning system that integrates the use of concepts of virtual learning environments, personal learning environments, and social network sites. The system is based on a learning model which comprises and integrates three learning contexts for the adult learner: the formal, instructional context, the personal context for learning management, and the social peer context. The paper outlines how the Open University of the Netherlands (OUN)implemented this model in the OpenU system and has piloted this system within the field of a MSc Program in the learning sciences.

Closer to you: reviewing the application, design and evaluation of ambient displays

Journal publications (peer-reviewed)
Börner, D., Kalz, M., & Specht, M. (2013). Closer to You: Reviewing the Application, Design, and Evaluation of Ambient Displays. International Journal of Ambient Computing and Intelligence (IJACI), 5(3), 16-31. doi:10.4018/ijaci.2013070102

This paper presents results from a recent literature review on ambient displays. While the main background of the authors is education and technology-enhanced learning, the review starts more generic with a broader view on ambient displays and their interactional, instructional, and informational characteristics. Beside depicting characteristics and classifying prototypical designs, the review also sheds light on the actual use of the covered ambient displays, their application context and addressed domains as well as the type of studies conducted, including the used methodologies and evaluation approaches to measure their effectiveness and impact. The review concludes with a discussion of the presented results emphasising the derived implications for the user when interacting with ambient displays.

se of Mobile Applications for Hospital Discharge Letters – Improving Handover at Point of Practice

Journal publications (peer-reviewed)
Maher, B., Drachsler, H., Kalz, M., Hoare, C., Sorensen, H., Lezcano, L., Henn, P., & Specht, M. (2013). Use of Mobile Applications for Hospital Discharge Letters – Improving Handover at Point of Practice. International Journal of Mobile and Blended Learning, 5(4), 19-42.

Handover of patient care is a time of particular risk and it is important that accurate and relevant information is clearly communicated. The hospital discharge letter is an important part of handover. However, the quality of hospital discharge letters is variable and letters frequently omit important information. The Cork Letter-Writing Assessment Scale (CLAS) is a mobile application based on an itemized checklist and scoring system developed to improve the quality of discharge letters. In a recent study, CLAS improved the quality (content, structure and clarity) of discharge letters written by medical students. Retention of these skills into the work-place and effects on patient safety have yet to be demonstrated. The development of standardized electronic discharge letters allow faster and safer transfer of information between healthcare providers and is a welcome advance. Mobile applications using Near Field Communication to seamlessly transfer discharge letters between devices is another important advance.

Beyond the channel: A literature review on ambient displays for learning

Journal publications (peer-reviewed)
Börner, D., Kalz, M., & Specht, M. (2013). Beyond the channel: A literature review on ambient displays for learning. Computers & Education, 60(1), 426–435.

The review analyses work in the research field of ambient display with a focus on the use of such displays for situational awareness, feedback and learning. The purpose of the review is to assess the state-of-the-art of the use of ambient displays with an explicit or implicit learning purpose and the possible classification of respective prototypes on the basis of a presented framework. This framework is comprised of theories around the educational concepts of situational awareness and feedback as well as design dimensions of ambient displays. The review sheds light on results of recent empirical studies within this field as well as developed prototypes with a focus on their design and instructional capabilities when providing feedback. The results expose that the explicit use of ambient displays for learning is not a prominent research topic, although implicitly ambient displays are already used to support learning activities fostering situational awareness by exploiting feedback. Overall ambient displays represent a technological concept with great potential for learning and the review facilitates a proper foundation and research questions for further research in this direction – towards ambient learning displays.

ARLearn: augmented reality meets augmented virtuality

Journal publications (peer-reviewed)
Ternier, S., Klemke, R., Kalz, M., Van Ulzen, P., & Specht, M. (2012). ARLearn: augmented reality meets augmented virtuality. Journal of Universal Computer Science, 18(15), 2143-2164.

This article deals with educational opportunities for mixed reality games and related scenarios for learning. It discusses several issues and educational challenges to be tackled when linking augmented reality and augmented virtuality. Second, the paper describes the architecture of the ARLearn system which offers highly flexible support for different educational settings. Three prototypical use cases implemented based on the underlying ARLearn framework are discussed, which are a field trip system, an augmented Google StreetView client called StreetLearn, and a real time crisis intervention game. ARLearn combines real time notification and mixed reality games across Mobile Augmented Reality and Virtual Reality and the authors aim to use the underlying (open source) framework for further case studies and mixed reality applications for learning support.

Energy awareness displays: motivating conservation at the workplace through feedback

Journal publications (peer-reviewed)
Börner, D., Storm, J., Kalz, M., & Specht, M. (2012). Energy awareness displays: motivating conservation at the workplace through feedback. International Journal of Mobile Learning and Organisation, 6(3/4), 189–203.

The paper presents a project that sets up to make energy consumption data visible and accessible to employees by providing dynamic situated consumption feedback at the workplace. Therefore a supporting infrastructure as well as two example applications have been implemented and evaluated. The resulting prototype fosters a ubiquitous learning process among the employees with the goal to change their consumption behaviour as well as the attitudes towards energy conservation. The paper presents the approach, the requirements, the infrastructure and applications, as well as the evaluation results of the conducted informative study, comparative study, user evaluation, and design study.

Thinking outside the box – A vision of ambient learning displays

Journal publications (peer-reviewed)

With a focus on the situated support of informal and non-formal learning scenarios in ubiquitous learning environments the presented paper outlines the authors’ vision of ambient learning displays – enabling learners to view, access, and interact with contextualised digital content presented in an ambient way. The vision is based on a detailed exploration of the characteristics of ubiquitous learning and a deduction of informational, interactional, and instructional aspects to focus on. Towards the vision essential research questions and objectives as well as a conceptual framework that acquires, channels, and delivers the information framed in the learning process are presented. To deliver scientific insights into the authentic learning support in informal and non-formal learning situations and to provide suggestions for the future design of ambient systems for learning the paper concludes with a research agenda proposing a research project including a discussion of related issues and challenges.

Language Technologies to Support Formative Feedback

Journal publications (peer-reviewed)
Berlanga, A. J., Kalz, M., Stoyanov, S., Van Rosmalen, P., Smithies, A., & Braidman, I. (2011). Language technologies to support formative feedback. Educational Technology & Society, 14(4), 11–20.

Formative feedback enables comparison to be made between a learner’s current understanding and a desired learning goal. Obtaining this information is a time consuming task that most tutors cannot afford. We therefore wished to develop a support software tool, which provides tutors and learners with information that identifies a learner’s progress, and requires only limited human intervention. The central idea is to use language technologies to create concepts maps automatically from texts, such as students’ essays or Blogs. By comparing maps from students over time, or with maps created from tutor’s materials, or by other students, it should be possible to ascertain learners’ progress and identify remedial actions. We review existing tools for automatic construction of concepts maps and describe our initial explorations of one of these tools. This paper then introduces the theoretical background of the proposed tool, design considerations and requirements. An initial validation, which explored tutors’ perceptions of the tool showed that tutors found the approach relevant, but its implementation in practice requires to consider teachers’ practices, the tools already in use, as well as institutional policies.

Implementing infrastructures for managing learning objects.

Journal publications (peer-reviewed)
Klemke, R., Ternier, S., Kalz, M., & Specht, M. (2010). Implementing infrastructures for managing learning objects. British Journal of Educational Technology, 41(6), 873-882. doi: 10.1111/j.1467-8535.2010.01127.x

Making learning objects available is critical to reuse learning resources. Making content transparently available and providing added value to different stakeholders is among the goals of the European Commission’s eContentplus programme. This paper analyses standards and protocols relevant for making learning objects accessible in distributed data provider networks. Types of metadata associated with learning objects and methods for metadata generation are discussed. Experiences from European projects highlight problems in implementing infrastructures and mapping metadata types into common application profiles. The use of learning contents and its associated metadata in different scenarios is described and concluded with lessons learned and pitfalls to avoid.

Expert concept mapping study on mobile learning

Journal publications (peer-reviewed)
Börner, D., Glahn, C., Stoyanov, S., Kalz, M., & Specht, M. (2010). Expert concept mapping study on mobile learning. Campus-Wide Information Systems, 27(4), 240-253.

The present paper introduces concept mapping as a structured participative conceptualization approach to identify clusters of ideas and opinions generated by experts within the domain of mobile learning. Utilizing this approach, the paper aims to contribute to a definition of key domain characteristics by identifying the main educational concepts related to mobile learning. Design/methodology/approach – A short literature review points out the attempts to find a clear definition for mobile learning as well as the different perspectives taken. Based on this an explorative case study was conducted, focusing on the educational problems that underpin the expectations on mobile learning. Using the concept mapping approach the study identified these educational problems and the related domain concepts. The respective results were then analyzed and discussed. Findings – The chosen approach produced several means to interpret the experts’ ideas and opinions, such as a cluster map illustrating and structuring substantial accordances. These means help to gain new insights on the emphasis and relation of the core educational concepts of mobile learning. The core educational concepts of mobile learning identified are: “access to learning”, “contextual learning”, “orchestrating learning across contexts”, “personalization”, and “collaboration”. Originality/value – The paper is original as it uses a unique conceptualization approach to work out the educational problems that can be addressed by mobile learning and thus contributes to a domain definition based on identified issues, featured concepts, and derived challenges. In contrast to existing approaches for defining mobile learning, the present approach relies completely on the expertise of domain experts.

Infrastrukturen zur Verwaltung von Metadaten von Offenen Bildungsressourcen

Journal publications (peer-reviewed)
Kalz, M., Klemke, R., Ternier, S., & Specht, M. (2010). Infrastrukturen zur Verwaltung von Metadaten von Offenen Bildungsressourcen. Zeitschrift für E-Learning. 3/2010. pp.19-29.

In den letzten Jahren sind weltweit zahlreiche Initiativen gestartet worden, um offene Bildungsressourcen zu speichern und der Öffentlichkeit zugänglich zu machen. Zu diesem Zweck sind an vielen Orten Lernobjektrepositorien eingerichtet worden, die die persistente Speicherung von offenen Bildungsressourcen erlauben. Die dezentrale Einrichtung dieser Repositorien machte es für Endbenutzer schwierig, geeignete Ressourcen in ihrer Domäne zu finden und wiederzuverwenden. Darüberhinaus waren in vielen Fällen keine oder wenig standardisierte Metadaten mit den Lerninhalten gespeichert, was das Finden von passenden Lernresourcen noch mehr erschwerte. Um die Sichtbarkeit und Zugänglichkeit von Offenen Bildungsressourcen zu erhöhen, hat die Europäische Kommission im Rahmen des eContentplus-Programmes Projekte gefördert, die diese Sitution verbessern sollten. In diesem Beitrag sollen Erfahrungen aus einigen Projekten vorgestellt und diskutiert werden. Dazu werden in diesem Artikel das verteilte Management von offenen Bildungsressourcen erläutert, die Erstellung eines Applikationsprofils diskutiert sowie die Rolle von verschiedenen Metadaten und deren Erstellung vorgestellt. Der Beitrag richtet sich an Projektleiter und Entscheidungsträger von Projekten, die sich mit Lernobjekten beschäftigen sowie an Forscher, die die Zugänglichkeit von Lernobjekten verbessern wollen.

A Model for New Linkages for Prior Learning Assessment

Journal publications (peer-reviewed)
Kalz, M., van Bruggen, J., Giesbers, B., Eshuis, J., Waterink, W. & Koper, R. (2008). A Model for New Linkages for Prior Learning Assessment. Campus Wide Information Systems. 25, 4. 233 -243.

The purpose of this paper is twofold: first the paper aims to sketch the theoretical basis for the use of electronic portfolios for prior learning assessment; second it endeavours to introduce latent semantic analysis (LSA) as a powerful method for the computation of semantic similarity between texts and a basis for a new observation link for prior learning assessment.

Wayfinding Services for Open Educational Practices

Journal publications (peer-reviewed)
Kalz, M., Drachsler, H., Van Bruggen, J., Hummel, H. G. K., & Koper, R. (2008). Wayfinding Services for Open Educational Practices. International Journal of Emerging Technologies in Learning, 3(2), 24-28.

To choose suitable resources for personal competence development in the vast amount of open educational resources is a challenging task for a learner. Starting with a needs analysis of lifelong learners and learning designers we introduce two wayfinding services that are currently researched and developed in the framework of the Integrated Project TENCompetence. Then we discuss the role of these services to support learners in finding and selecting open educational resources and finally we give an outlook on future research.

Positioning of Learners in Learning Networks with Content-Analysis, Metadata and Ontologies

Journal publications (peer-reviewed)
Kalz, M, Van Bruggen, J., Rusmann, E., Giesbers, B., & Koper, R. (2007). Positioning of Learners in Learning Networks with Content-Analysis, Metadata and Ontologies. Interactive Learning Environments, 15, 191-200.

Positioning in learning networks is a process that assists learners in finding a starting point and an efficient route through the network that will foster competence building. In the past we explored computational approaches to positioning that are based on the contents of the learning network and the behavior of those participating in it, more or less ignoring different efforts to stimulate positioning and competence development from a top-down-perspective. In this paper we introduce a research agenda for positioning in learning networks, discuss several cases and give an outlook on the development of a positioning service for learning networks.

Mediendidaktik in der Lehrerbildung

Journal publications (peer-reviewed)
Kerres, Michael: Kalz, Marco: Mediendidaktik in der Lehrerbildung. Beiträge zur Lehrerbildung, 21 (3), 2003. S. 410 – 421.

Der Beitrag diskutiert Ziele und Inhalte der Aus- und Weiterbildung von Lehr- Personen im Bereich der Mediendidaktik. Vorliegende Angebote – vor allem in der Weiterbildung – sind oft entweder technisch oder medienerzieherisch ausgerichtet und als solches zu eng ausgeführt. Neben der Auseinandersetzung mit Möglichkeiten digitaler Medien im Unterricht sollte die didaktische Refle- xion über die Bedeutung von Medien in Lehr-Lernprozessen im Mittelpunkt stehen. Dazu sind insbesondere die pädagogischen Ziele des Medieneinsatzes zu benennen, zu deren Analyse und Bewertung Leitfragen vorgestellt werden.

Creating Engaging Experiences in MOOCs through In-Course Redeemable Rewards

Conference publications (Peer-Reviewed)
Ortega-Arranz, A., Kalz, M., Martínez-Monés, A. (2018, 17-20 April). Creating Engaging Experiences in MOOCs through In-Course Redeemable Rewards. In Proceedings of the 2018 IEEE Global Engineering Education Conference, EDUCON, Santa Cruz de Tenerife, Spain.

Gamification strategies have been proposed to mitigate student disengagement and dropouts in massive online environments, due to the positive results shown by these strategies at lower scales. Among various gamification strategies, redeemable rewards have been identified as an effective element to intrinsically motivate students and increase their engagement in educational settings, including MOOCs. Yet, effective design, implementation and enactment of this gamification strategy in MOOC contexts might face new challenges, given the unique characteristics of these learning settings such as massiveness. As an attempt to help teachers use redeemable rewards in MOOCs, this paper analyzes the characteristics of MOOCs that influence its integration and presents a proposal of a system supporting the design, implementation and enactment of such rewards. The envisioned system is illustrated by a scenario that describes the main features of this system for teachers and students.

Predicting learner-centered MOOC outcomes: Satisfaction and intention-fulfillment

Conference publications (Peer-Reviewed)
Rabin, E., Kalman, Y.M., & Kalz, M. (2018). Predicting learner-centered MOOC outcomes: Satisfaction and intention-fulfillment, Proceedings of the 13th Chais Conference for the Study of Innovation and Learning Technologies: Learning in the Technological Era, Raanana: The Open University of Israel.

The goal of this research was to understand the predictors of two important learner-centered outcome measures of success in massive open online courses (MOOCs): learner satisfaction and learner intention-fulfillment. In contrast with previous studies which focused on the fulfillment of the course developers’ intentions and placed retention and completion rates as the ultimate outcome measures, these two outcomes are more appropriate for measuring success in the non-formal life-long learning context. Combining data from self- report surveys and actual behaviour, a total number of 125 MOOC participants answered a pre- and a post-questionnaire and their behavioral measurements were harvested from the log-files of the course. Using structural equation modeling enables to see the effect of the independent variables included in the study – demographic and educational background, outcome beliefs, online self- regulation learning, learners behaviour and perceived course usability. The results suggest that participant gender, his/her number of weekly quizzes taken and the length of participation in the course affect the perception of individual intention-fulfilment, while the number of lectures that the participants took affect the level of course satisfaction. Positive outcome beliefs, the ability to regulate the learning by setting goals and the perceived usability of the course affected the level of intention-fulfilment and course satisfaction.

Towards ‘MOOCs with a Purpose’: Crowdsourcing and analysing scalable design solutions with MOOC learners

Conference publications (Peer-Reviewed)
Van Rosmalen, P., Kasch, J., Kalz, M., Firssova, O., & Brouns, F. (2017, 12-15 September). Towards ‘MOOCs with a Purpose’: Crowdsourcing and analysing scalable design solutions with MOOC learners. In É. Lavoué et al. (Eds.), Data Driven Approaches in Digital Education; 12th European Conference on Technology Enhanced Learning: EC-TEL 2017, LNCS 10474 (pp. 486-491). Tallinn, Estonia: Springer International Publishing.

In this presentation we present a study in which we explored the use of a research assignment on instructional design of MOOCs by MOOC students. The use of a research assignment was expected to be of interest for both students and the designer. The assignment is based on a framework to analyse MOOC designs with the objective to identify best practices. It builds on four principles: constructive alignment, task complexity, interaction and formative feedback. The exploration indicates that students positively appreciate this kind of assignments. Moreover, the crowdsourcing alike approach showed to be a valuable way for MOOC designers to get awarded with data gathered by their participants. The participants, be it a small sample, were able to apply the framework to analyse MOOCs and identify best practices. We will discuss the framework and the results of its application. Finally, we will conclude with the experiences of the users.

To change or not to change? That’s the question… On MOOC-success, barriers and their implications

Conference publications (Peer-Reviewed)
Henderikx, M., Kreijns, K., & Kalz, M. (2017). To change or not to change? That’s the question… On MOOC-success, barriers and their implications. In Delgado Kloos, C., Jermann, P., Pérez-Sanagustin, M., Seaton, D.T., & White, S. (Eds.), Lecture Notes in Computer Science: Vol.10254. Digital Education: Out to the World and Back to the Campus (pp. 210-216). Cham, Switzerland: Springer International Publishing AG. doi:10.1007/978-3-319-59044-8_25

This explorative study aimed to get an understanding of MOOC-success as seen from the perspective of the MOOC-taker and the types of barriers which might stand in the way of this success. Data of two MOOCs was used to illustrate MOOC-success from two perspectives and barriers encountered. Following the currently used approach to identify educational success, the success rate of MOOC-II was 5,6%. The success rates from the perspective of the MOOC-taker was 70%. In addition, data of MOOC-I and II showed that the encountered barriers were mainly non-MOOC-related. Workplace issues and lack of time were most frequently indicated. For MOOC-designers’ decision making regarding redesign of a MOOC after evaluation, it is valuable to have insight in these matters to prevent unnecessary design interventions

Implementation intention and how it affects goal achievement in MOOCs

Conference publications (Peer-Reviewed)
Kreijns, K., Kalz., M., Castano-Munoz, J., & Punie, Y. (2017). Implementation intention and how it affects goal achievement in MOOCs. In Kloos, C., White, S., & Seaton, D. (Eds.). Lecture Notes in Computer Science: Vol.10254. Digital Education: Out to the World and Back to the Campus (pp. 210-216). Cham, Switzerland: Springer International Publishing AG. doi:10.1007/978-3-319-59044-8_25

Implementation intentions are a way for MOOC learners to help act- ing out their goal intentions. Implementation intentions are concerned with planning where, when, and how learning will take place as well as planning how much time will be allocated to the learning and determining how potential problems will be resolved (referred to as shielding behavior). The current study investigates the relationship between the degree to which implementation inten- tions are formed and the degree to which goals are achieved (less than intended, all as intended, more than intended) thereby taking the time spent on learning and the number of barriers encountered into account. The results, based on the current data collection of a single MOOC, revealed that the degree of imple- mentation intentions was completely determined by time planning. Implementa- tion intentions did not affect the degree of achieved goals relative to intended goal achievement nor did the number of barriers encountered. Implementation intentions also did not influence the impact of the number of barriers on the de- gree of achieved goals relative to intended goal achievement (the finding was not significant). Finally, MOOC learners who planned time spent less time on learning than those who did not, which suggests these planning learners were more effective with their learning time. For those time-planning leaners, time spent had a significant positive effect on the degree of achieved goals relative to intended goal achievement.

Designing for Open Learning: Design principles and scalability affordances in practice

Conference publications (Peer-Reviewed)
Firssova, O., Brouns, F., & Kalz, M. (2016, 25-26 April). Designing for Open Learning: Design principles and scalability affordances in practice. In Haywood, J., Aleven, V., Kay, J., & Roll, I. (Eds). Proceedings of the Third (2016) ACM Conference on Learning @ Scale (pp. 249-252). Edinburgh, Scotland, UK. ACM, USA. doi:10.1145/2876034.2893426

This work-in-progress paper elaborates on a gradually evolving approach to design of open learning and the design principles used by the Open University of the Netherlands in short open courses – online masterclasses and in Massive Open Online Courses – delivered in the learning environment of the Open University and in the experimental multilingual MOOC aggregator EMMA as part of a European project. As the paper will demonstrate, these principles can be seen as building blocks of open scalable design of active and engaging learning.

In the eye of the beholder: promoting learner-centric design to develop mobile games for learning

Conference publications (Peer-Reviewed)
Schmitz, B., Kalz, M., Klemke, R., & Specht, M. (2014). In the eye of the beholder: promoting learner-centric design to develop mobile games for learning. In M. Kalz, Y. Bayyurt, & M. Specht (Eds.), Mobile as mainstream – towards future challenges in mobile learning. Proceedings of the 13th world conference on mobile and contextual learning. Communications in Computer and Information Science (CCIS), Vol. 479 (pp. 92-100). Istanbul, Turkey: Springer.

Out of the project EMuRgency a game-based learning environment evolved, which trains school children in providing reanimation and cardiopulmonary resuscitation (CPR). The application gets players to act as if they were in a real case of emergency. This paper reports on a formal usability study conducted with two different groups of learners, regular learners and learners with special educational needs (SEN). With the study we compared the two groups of learners with regard to game usability and effectiveness of the intervention. Our intention was to better understand the different needs and requirements to learning materials that game designer need to take into consideration in order to make the learning experience successful for both groups. A total of 89 children played the game simulation. Results showed differences in perception and effectiveness of individual mechanisms for the two groups with regard to usability or switching between tasks and mobile device.

GPIM: Google Glassware for Inquiry-Based Learning

Conference publications (Peer-Reviewed)
Suarez, A., Ternier, S., Kalz, M., & Specht, M. (2014). GPIM: Google Glassware for Inquiry-Based Learning. In C. Rensing et al (Eds.), Proceedings of the 9th European conference on Technology Enhanced Learning - EC-TEL 2014: Open Learning and Teaching in Educational Communities, LNCS 8719 (pp. 530-533). Graz, Austria.

Over the past few years, the use of mobile personal devices has witnessed a widespread take-up. With wearable technology like head-up- displays a new genre of educational technology is appearing to enhance contextualized learning. This paper reports about a Google Glass prototype for Inquiry-Based Learning (IBL). With the use of Google Glass we aim to bring learning to the next level providing a more seamless experience where Glass supports the learning- and inquiry-process just in time and in an unobtrusive way. This demo paper introduces the design and functionalities of Google Glass for the Personal Inquiry Manager (GPIM). The paper concludes with open issues for future research, especially focused on evaluation and future development.

What happened to the crossdisciplinarity of Technology-Enhanced Learning in 2004?

Conference publications (Peer-Reviewed)
Kalz, M. (2013). What happened to the crossdisciplinarity of Technology-Enhanced Learning in 2004? In: Hernández-Leo, D.; Ley, T.; Klamma, R.; Harrer, A. (Eds.) Scaling up learning for sustained impact. Proceedings of the 8th European Conference on Technology-Enhanced Learning (EC-TEL 2013) (pp. 472–477). September 17 – 21, 2013. Paphos, Cyprus. Lecture Notes in Computer Science 8095. Berlin: Springer Verlag.

In a recent study the crossdisciplinarity of the field of Technology-Enhanced Learning was analysed with science-overlay-maps and diversity measures. Results reveal that the crossdisciplinarity of the field has constantly increased over the last 10 years. Only in 2004, a significant decrease of interdisciplinary research could be identified. In this paper we take a closer look at the publications of this year and test our hypotheses for the decrease of crossdisciplinarity.

Pervasive Interventions to Increase Pro-environmental Awareness, Consciousness, and Learning at the Workplace

Conference publications (Peer-Reviewed)
Börner, D., Kalz, M., Ternier, S., & Specht, M. (2013). Pervasive Interventions to Increase Pro-environmental Awareness, Consciousness, and Learning at the Workplace. In: Hernández-Leo, D.; Ley, T.; Klamma, R.; Harrer, A. (Eds.) Scaling up learning for sustained impact. Proceedings of the 8th European Conference on Technology-Enhanced Learning (EC-TEL 2013) (pp. 57–70). Paphos, Cyprus. Lecture Notes in Computer Science 8095. Berlin: Springer Verlag.

This paper reports about pervasive interventions at a university campus to increase the pro-environmental awareness, consciousness, and learning of employees. Based on an assessment of the research gaps in this problem area we present results and design implications from three intervention iterations. While in the first intervention the focus was on increasing awareness through information distribution with ambient learning displays on the campus, the second iteration provided personalised feedback to employees with the help of a sensor network and different client applications. The third iteration then implemented a game-based learning concept. Results reveal that these approaches are effective on different levels and that a combination of these elements can lead to increased pro-environmental consciousness, learning and hopefully a sustained behaviour change of employees.

Designing a mobile learning game to investigate the impact of role-playing on helping behavior

Conference publications (Peer-Reviewed)
Schmitz, B., Ternier, S., Klemke, R., Kalz, M., & Specht, M. (2013). Designing a mobile learning game to investigate the impact of role-playing on helping behavior. In D. Hernández-Leo et al. (Eds.), Scaling up Learning for Sustained Impact. Proceedings of European Conference on Technology Enhanced Learning (EC-TEL), LNCS 8095 (pp. 357–370). Berlin Heidelberg, Germany: Springer-Verlag.

Despite research in mobile learning games has intensified over the last decade, there is relatively little research about how individual game mechanisms influence or change behavior. This article aims at understanding the influence of the game mechanism role-playing and investigates how it can be used to alter behavioral intention. In order to do so, we designed a mobile learning game to train Basic Life Support (BLS) and Cardiopulmonary Resuscitation (CPR). With the game we aim at improving laymen’s willingness to help in case of emergency. First, we illuminate the strand of research related to role-playing in the context of BLS and bystander CPR. Second, we describe the pedagogical framework of the mobile learning game that was designed to train BLS and introduce the game engine this development is based on. Third, we present the results from a first prototype testing, which we carried out with medical professionals as well as laymen in order to test game-play usability and interface. This article concludes by outlining the experimental setting of an upcoming study, which will use the mobile learning game to evaluate the influence of the game mechanism role-playing on the willingness to provide bystander CPR in case of emergency.

ARLearn

Conference publications (Peer-Reviewed)
Börner, D., Ternier, S., Klemke, R., Schmitz, B., Kalz, M., Tabuenca, B., & Specht, M. (2013). ARLearn - Open source mobile application platform for learning. In D. Hernández-Leo et al. (Eds.), Scaling up Learning for Sustained Impact. Proceedings of the 8th European Conference on Technology Enhanced Learning (EC-TEL 2013), LNCS 8095 (pp. 536-539). Berlin Heidelberg, Germany: Springer.

The paper presents and outlines the demonstration of an open source mobile application platform for designing, supporting, and evaluating mobile learning scenarios that make use of media artefacts in a specific context. The platform contains a web-based authoring environment, cross-platform mobile applications to run the scenarios, as well as tools to monitor progress and results. Besides exploring the pedagogical background, the paper describes the conceptual implementation as well as the technical infrastructure and lists the requirements for demonstrating the platform and all its components.

f MOOCS are the answer, did we ask the right questions? Implications for the design of large-scale online-courses

Conference publications (Peer-Reviewed)
Kalz, M., & Specht, M. (2013). If MOOCS are the answer, did we ask the right questions? Implications for the design of large-scale online courses. Working paper 2013/25. Maastricht School of Management, Maastricht.

In this conference contribution we deal with the phenomenon of Massive Open Online Courses (MOOCs). Based on summary of the state of the art we discuss aspects of the learning design of MOOCs that have not been sufficiently studied. More specifically we discuss the issue of diversity and support facilities in MOOCs. We introduce the concept of learning networks and learner support services that have been developed to enable personalized learning scenarios in large-scale online environments. We report about a learning design for large-scale open online courses that has been evaluated and further developed in the last year. Last but not least we discuss future research.

Design of a game-based pre-hospital resuscitation training for first responders

Conference publications (Peer-Reviewed)
Kalz, M., Schmitz, B., Biermann, H., Klemke, R., Ternier, S., & Specht, M. (2013). Design of a game-based pre-hospital resuscitation training for first responders. In A. Holzinger, M. Ziefle, & V. Glavinić (Eds.), SouthCHI 2013, LNCS 7946 (pp. 363-372). Germany: Springer, Heidelberg.

This paper reports about the design of a game-based training intervention for pre-hospital resuscitation training. Our underlying assumption is, that survival chances in cardiac arrest situations could be significantly improved, if bystanders would be better educated and prepared to help. Based on a discussion of problems of current training concepts and related educational the-ories a game-based learning intervention is proposed. The focus of the interven-tion is the improvement of procedural knowledge and self-efficacy of partici-pants. The game is designed on the basis of the ARLearn platform. The game context and game-design is discussed. Last but not least we discuss short-term and long-term evaluation scenarios.

The CLAS App - A training app to improve medical handovers

Conference publications (Peer-Reviewed)
Maher, B., Drachsler, H., Kalz, M., & Specht, M. (2012, 16-18 October). The CLAS App - A training app to improve medical handovers. Presentation at the 11th World Conference on Mobile and Contextual Learning, Helsinki, Finland.

There is a high potential for mobile learning and support applications in the health domain. In this paper we introduce the CLAS App, a mobile application to support handover procedures based on the improvement of writing skills. Handover of patient care is a time of particular risk and it is important that accurate, reliable and relevant information is clearly communicated between one caregiver to another. Improperly conducted handovers lead to wrong treatment, delays in medical diagnosis, life threatening adverse events, patient complaints, medical litigation, increased health care expenditure, increased hospital length of stay and a range of other effects that impact on the health system. The CLAS App helps standardise and improve handover communication between hospital and community healthcare.

Impact of tablet computers and eBooks on learning practices of law students

Conference publications (Peer-Reviewed)
Kalz, M., Specht, M., & Van Oosterzee, M. (2012). Impact of tablet computers and eBooks on learning practices of law students. In M. Specht, J. Multisilta, & M. Sharples (Eds.), Proceedings of the 11th World Conference on Mobile and Contextual Learning 2012 (pp. 203-208). October, 16-18, 2012, Helsinki, Finland.

In this paper we present first results from an ongoing pilot study at the law faculty of the Open University of the Netherlands. Students participating in a bridging course have been given the choice between traditional study methods and the use of a tablet-computer equipped with digital learning resources and electronic textbooks. In this paper we report first qualitative findings from this pilot study. A monthly questionnaire has been administered to let students rate statements on a 7-point-likert-scale. These findings are enriched with results from a focus group session in January 2012. The results are discussed and an outlook on future work is provided.

Ambient Learning Displays: lecture series and results from a participatory design study

Conference publications (Peer-Reviewed)
Börner, D., Kalz, M., & Specht, M. (2012). Ambient Learning Displays: lecture series and results from a participatory design study. In M. Specht, J. Multisilta, & M. Sharples (Eds.), Proceedings of the 11th World Conference on Mobile and Contextual Learning 2012 (pp. 232-235). October, 16-18, 2012, Helsinki, Finland.

Emerging from pervasive and mobile technologies, ambient displays present information and media in the periphery of the user. Thereby the displays situated and interacting in the close proximity are an addition to existing personal interfaces in the foreground, while the user attention can always move from one to the other and back. Especially the ability to deliver contextualised and personalised information in authentic situations fosters ambient displays as an instrument for learning. However the actual design of ambient displays for learning proves to be difficult, as the technical implementations as well as the underlying instructional principles are still immature. The paper presents the main constituents of a lecture series on the use of ambient displays for learning and a first participatory design study conducted during two consecutive lecture sessions. The results show a variety of usable ambient display types, possible learning scenarios, and specific design proposals towards ambient learning displays.

Energy Awareness Displays – Prototype for personalised energy consumption feedback

Conference publications (Peer-Reviewed)
Börner, D., Kalz, M., & Specht, M. (2012). Energy Awareness Displays: designing a prototype for personalised energy consumption feedback at the workplace. 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education (pp. 211-213). March, 27-30, 2012, Takamatsu, Japan: IEEE Computer Society.

The paper describes work-in-progress on a prototype providing personalised energy consumption feedback at the workplace. Based on a provisional framework developed in the context of an ongoing research project the prototype and the supporting infrastructure are conceptually outlined.

A novel approach towards skill-based search and services of Open Educational Resources

Conference publications (Peer-Reviewed)
Ha, K.-H., Niemann, K., Schwertel, U., Holtkamp, P., Pirkkalainen, H., Börner, D. et al (2011). A novel approach towards skill-based search and services of Open Educational Resources. In E. Garcia-Barriocanal, A. Öztürk, & M. C. Okur (Eds.), Metadata and Semantics Research: 5th International Conference MTSR 2011 (pp. 312-323), Izmir, Turkey, October 12-14, 2011. Springer.

Open educational resources (OER) have a high potential to address the growing need for training materials in management education and training. Today, a high number of OER in management are already available in a large number of repositories. However, users face barriers as they have to search repository by repository with different interfaces to retrieve the appropriate learning content. In addition, the use of search criteria related to skills, such as learning objectives and skill-levels is not generally supported. The European co-funded project OpenScout addresses these barriers by intelligently connecting leading European OER repositories and providing federated, skillbased search and retrieval web services. On top of this content federation the project supports users with easy-to-apply tools that will accelerate the (re-) use of open content.

Supporting the Reuse of Open Educational Resources through Open Standards

Conference publications (Peer-Reviewed)
Glahn, C., Kalz, M., Gruber, M., & Specht, M. (2010). Supporting the Reuse of Open Educational Resources through Open Standards. In T. Hirashima, A. F. Mohd Ayub, L. F. Kwok, S. L. Wong, S. C. Kong, & F. Y. Yu (Eds.), Workshop Proceedings of the 18th International Conference on Computers in Education: ICCE2010 (pp. 308-315). November, 29 - December, 3, 2010, Putrajaya, Malaysia: Asia-Pacific Society for Computers in Education.

In this paper we analyse open standards for supporting the reuse of OER in different knowledge domains based on a generic architecture for content federation and higher-order services. Plenty OER are available at different institutions. We face the problem that the mere availability of these resources does not directly lead to their reuse. To increase the accessibility we integrated existing resource repositories to allow educational practitioners to discover appropriate resources. On top of this content federation we build higher order services to allow re-authoring and sharing of resources. Open standards play an important role in this process for developing high-level services for lowering the thresholds for the creation, distribution and reuse of OER in higher education.

OpenScout: Competence based management education with community-improved open educational resources

Conference publications (Peer-Reviewed)
Kalz, M., Specht, M., Nadolski, R., Bastiaens, Y., Leirs, N., & Pawlowski, J. (2010). OpenScout: Competence based management education with community-improved open educational resources. In Halley et al. (Eds.), Proceedings of the 17th EDINEB Conference. Crossing Borders in Education and work-based learning (pp. 137-146). Maastricht, The Netherlands: FEBA ERD Press.

This contribution introduces the European funded OpenScout project. A basic infrastructure to find and reuse open educational resources (OER) in the field of business and management education is introduced. Based on a discussion of competence based education in the domain of business and management education some selected competence models are summarized. An example of a future user and evaluation group of the OpenScout platform is provided with the PLATO network. Two alternative methods to enrich learning resources with competence related information are discussed, namely purpose tagging and problem collections.

The promise and potential of e-assessment for learning

Book chapters
Ras, E., Whitelock, D., & Kalz, M. (2016). The promise and potential of e-assessment for learning. In P. Reimann, S. Bull, M. Kickmeier-Rust, R. Vatrapu, & B. Wasson (Eds.), Measuring and Visualizing Learning in the Information-Rich Classroom (pp. 21-40). Oxford: Routledge.

Formative assessment has gained substantial ground in the last ten years, together with a number of considerable promises that have been made about its potential to promote student learning. The recent drive towards Assessment for Learning and assessment for 21st Century skills raises a set of new challenges for both teachers and students alike. These challenges are related, on the one hand, to progress monitoring that results in responsive teaching or support activities, and on the other hand to the development and implementation of technologies that will allow (semi-) automated and personalised assessment systems. New data sources collected from such software will open new doors for formative assessment practices and related feedback types.

Lifelong Learning and Its Support with New Technologies

Book chapters
Kalz, M. (2015). Lifelong Learning and its support with new technologies. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences: Vol. 14 (2nd ed.) (pp. 93-99). Oxford: Elsevier.

This chapter provides an overview about the use of new technologies for lifelong learning. While in the past learning technologies were mostly provided by educational institutions to support a specific lifetime or shorter learning episodes nowadays more personal technologies are used for lifelong learning to support self-organized learning. Four important developments are introduced in this chapter, namely open learner models and learning analytics, learning networks and networked learning, open educational resources and practices and last but not least mobile and contextualized learning. The state-of-the-art in these research fields is summarized and future potential and requirements for lifelong learning are highlighted.

PLE in formal education: challenges for openness and control

Book chapters
Kalz, M. (2015). PLE in formal education: challenges for openness and control. In S. Kroop, A. Mikroyannidis, & M. Wolpers (Eds.), Responsive Open Learning Environments (pp. 226-228). Heidelberg, Germany: Springer.

This short comment reflects on a critical account of educational technology and makes reference to the chapter by Vieritz et al. about the use of widget bundles for formal learning in higher education.

Mindergie: A pervasive learning game for pro environmental behaviour at the workplace

Book chapters
Kalz, M., Börner, D., Ternier, S., & Specht, M. (2015). Mindergie: A pervasive learning game for pro environmental behaviour at the workplace. In L.-H. Wong, M. Milrad, & M. Specht (Eds.), Seamless Learning in the Age of Mobile Connectivity (397-417). Singapore: Springer.

This chapter reports about a pervasive learning game to increase the environmental awareness and pro-environmental behaviour at the workplace. Based on a discussion of the theoretical background and related work we introduce the game design and game elements. Results of a formative evaluation study are presented and discussed. Results show that incentive mechanisms are less important than challenging game components that involve employees in proposing solutions for energy conservation at the workplace. Conclusions are drawn for future games and energy conservation activities at the workplace.

Innovation und Trends für mobiles Lernen

Book chapters
Specht, M., Kalz, M., & Börner, D. (2013). Innovation und Trends für Mobiles Lernen. In C. de Witt, & A. Sieber (Eds.), Mobile Learning. Potenziale, Einsatzszenarien und Perspektiven des Lernens mit mobilen Endgeräten (pp. 55-74). Springer Fachmedien, Wiesbaden.

Der Beitrag zeigt aktuelle Trends im Bereich der mobilen und ubiquitären Lerntechnologien auf, welche die klassischen Konzepte von Mobilem Lernen erweitern: a) Mobiler und allgegenwärtiger Zugang zu Lerninhalten b) unterbrechungsfreie Lernunterstützung oder “Seamless Learning Support”, die nahtlose Integration von Lernunterstützung in gemischten Lernszenarien, c) Smartphones und Sensoren im Mobilen Lernen, d) Mobile Gaming und mobile Augmented Reality und e) situierte eingebettete Displays. Anhand dieser Trends werden die Konsequenzen für das didaktische Design und darunter liegende Lernkonzepte diskutiert.

Systeme im Einsatz. Lernmanagement, Kompetenzmanagement und PLE

Book chapters
Kalz, M., Schön, S., Lindner, M., Roth, D., & Baumgartner, P. (2011). Systeme im Einsatz. Lernmanagement, Kompetenzmanagement und PLE. In M. Ebner, & S. Schön (Eds.), L3T - Lerhbuch für Lernen und Lehren mit Technologie (pp. 111-118). Graz, Austria: Uni Graz.

Dieser Beitrag stellt drei Formen von technologischen Systemen vor, die derzeit im Bereich des Lernens und Lehrens eingesetzt bzw. diskutiert werden. Weit verbreitet sind Lernmanagementsysteme (LMS), die zur Verwaltung von Lernenden und Kursabwicklung in (Hoch-)Schulen genutzt werden. Jünger sind Kompetenzmanagementsysteme (KMS), die vor allem in Unternehmen Prozesse der Kompetenzentwicklung unterstützen und dokumentieren sollen. In den letzten Jahren hat schließlich ein neues Konzept des webbasierten persönlichen Informations- und Lernmanagements and Aufmerksamkeit gewonnen, die sog. “Persönlichen Lernumgebungen” (PLE). In diesem Beitrag werden keine technologischen Herausforderungen oder Lösungen beschrieben, sondern die praktischen Anforderungen und Wirkungen der Systeme aus pädagogischer Sicht betrachtet.

Persönliche Lernumgebungen: Grundlagen, Möglichkeiten und Herausforderungen eines neuen Konzeptes

Book chapters
Schaffert, S., & Kalz, M. (2009). Persönliche Lernumgebungen: Grundlagen, Möglichkeiten und Herausforderungen eines neuen Konzepts. In K. Wilbers & A. Hohenstein (Hrsg.), Handbuch E-Learning (Gruppe 5, Nr. 5.16, pp. 1-24). Köln, Germany: Deutscher Wirtschaftsdienst (Wolters Kluwer Deutschland), 27. Erg.-Lfg. Januar 2009.

The concept of Personal Learning Environments (PLE) is a new concept which breaks with a lot of traditions of technology-enhanced learning. In its core the learner can take control over his learning environments and he can conduct a “learning environment design” for his own purposes and needs. The chapter introduces the history of the PLE concept and discusses new opportunities and issues which come with this new concept.