Journal Articles

Permanent URI for this collectionhttps://mro.massey.ac.nz/handle/10179/7915

Browse

Search Results

Now showing 1 - 10 of 10

Automating Systematic Literature Reviews with Retrieval-Augmented Generation: A Comprehensive Overview
(MDPI (Basel, Switzerland), 2024-10-09) Han B; Susnjak T; Mathrani A; Garcia Villalba LJ
This study examines Retrieval-Augmented Generation (RAG) in large language models (LLMs) and their significant application for undertaking systematic literature reviews (SLRs). RAG-based LLMs can potentially automate tasks like data extraction, summarization, and trend identification. However, while LLMs are exceptionally proficient in generating human-like text and interpreting complex linguistic nuances, their dependence on static, pre-trained knowledge can result in inaccuracies and hallucinations. RAG mitigates these limitations by integrating LLMs’ generative capabilities with the precision of real-time information retrieval. We review in detail the three key processes of the RAG framework—retrieval, augmentation, and generation. We then discuss applications of RAG-based LLMs to SLR automation and highlight future research topics, including integration of domain-specific LLMs, multimodal data processing and generation, and utilization of multiple retrieval sources. We propose a framework of RAG-based LLMs for automating SRLs, which covers four stages of SLR process: literature search, literature screening, data extraction, and information synthesis. Future research aims to optimize the interaction between LLM selection, training strategies, RAG techniques, and prompt engineering to implement the proposed framework, with particular emphasis on the retrieval of information from individual scientific papers and the integration of these data to produce outputs addressing various aspects such as current status, existing gaps, and emerging trends.
Effectiveness of a Learning Analytics Dashboard for Increasing Student Engagement Levels
(Society for Learning Analytics Research (SoLAR), 2023-12-22) Ramaswami G; Susnjak T; Mathrani A
Learning Analytics Dashboards (LADs) are gaining popularity as a platform for providing students with insights into their learning behaviour patterns in online environments. Existing LAD studies are mainly centred on displaying students’ online behaviours with simplistic descriptive insights. Only a few studies have integrated predictive components, while none possess the ability to explain how the predictive models work and how they have arrived at specific conclusions for a given student. A further gap exists within existing LADs with respect to prescriptive analytics that generate data-driven feedback to students on how to adjust their learning behaviour. The LAD in this study attempts to address this gap and integrates a full spectrum of current analytics technologies for sense-making while anchoring them within theoretical educational frameworks. This study’s LAD (SensEnablr) was evaluated for its effectiveness in impacting learning in a student cohort at a tertiary institution. Our findings demonstrate that student engagement with learning technologies and course resources increased significantly immediately following interactions with the dashboard. Meanwhile, results showed that the dashboard boosted the respondents’ learning motivation levels and that the novel analytics insights drawn from predictive and prescriptive analytics were beneficial to their learning. This study, therefore, has implications for future research when investigating student outcomes and optimizing student learning using LAD technologies.
Data Quality Challenges in Educational Process Mining: Building Process-Oriented Event Logs from Process-Unaware Online Learning Systems
(Inderscience, 2022-05-04) Umer R; Susnjak T; Mathrani A; Suriadi S
Educational process mining utilizes process-oriented event logs to enable discovery of learning practices that can be used for the learner’s advantage. However, learning platforms are often process-unaware, therefore do not accurately reflect ongoing learner interactions. We demonstrate how contextually relevant process models can be constructed from process-unaware systems. Using a popular learning management system (Moodle), we have extracted stand-alone activities from the underlying database and formatted it to link the learners’ data explicitly to process instances (cases). With a running example that describes quiz-taking activities undertaken by students, we describe how learner interactions can be captured to build process-oriented event logs. This article contributes to the fields of learning analytics and education process mining by providing lessons learned on the extraction and conversion of process-unaware data to event logs for the purpose of analysing online education data.
Online Tracking: When Does it Become Stalking?
(World Scientific Publishing, 2021-05-25) Amarasekara B; Mathrani A; Scogings C
Online user activities are tracked for many purposes. In e-commerce, cross-domain tracking is used to quantify and pay for web-tra±c generation. Our previous research studies have shown that HTTP cookie-based tracking process, though reliable, can fail due to technical reasons, as well as through fraudulent manipulation by tra±c generators. In this research study, we evaluate which of the previously published tracking mechanisms are still functional. We assess the e±cacy and utility of those methods to create a robust tracking mechanism for e-commerce. A failsafe and robust tracking mechanism does not need to translate into further privacy intrusions. Many countries are rushing to introduce new regulations, which can have a negative impact on the development of robust technologies in an inherently stateless eco-system. We used a multi-domain, purpose-built simulation environment to experiment common tracking scenarios, and to describe the parameters that de¯ne the minimum tracking requirement use-cases, and practices that result in invading privacy of users. This study will help practitioners in their implementations, and policy developers and regulators to draw up policies that would not curtail the development of robust tracking technologies that are needed in e-commerce activities, while safeguarding the privacy of internet users.
Perspectives on the challenges of generalizability, transparency and ethics in predictive learning analytics
(Elsevier Ltd, 2021-11-20) Mathrani A; Susnjak T; Ramaswami G; Barczak A
Educational institutions need to formulate a well-established data-driven plan to get long-term value from their learning analytics (LA) strategy. By tracking learners’ digital traces and measuring learners’ performance, institutions can discern consequential learning trends via use of predictive models to enhance their instructional services. However, questions remain on how the proposed LA system is suitable, meaningful, and justifiable. In this concept paper, we examine generalizability and transparency of the internals of predictive models, alongside the ethical challenges in using learners’ data for building predictive capabilities. Model generalizability or transferability is hindered by inadequate feature representation, small and imbalanced datasets, concept drift, and contextually un-related domains. Additional challenges relate to trustworthiness and social acceptance of these models since algorithmic-driven models are difficult to interpret by themselves. Further, ethical dilemmas are faced in engaging with learners’ data while developing and deploying LA systems at an institutional level. We propose methodologies for apprehending these challenges by establishing efforts for managing transferability and transparency, and further assessing the ethical standing on justifiable use of the LA strategy. This study showcases underlying relationships that exist between constructs pertaining to learners’ data and the predictive model. We suggest the use of appropriate evaluation techniques and setting up research ethics protocols, since without proper controls in place, the model outcome would not be portable, transferable, trustworthy, or admissible as a responsible outcome. This concept paper has theoretical and practical implications for future inquiry in the burgeoning field of learning analytics.
On Developing Generic Models for Predicting Student Outcomes in Educational Data Mining
(MDPI (Basel, Switzerland), 2022-01-07) Ramaswami G; Susnjak T; Mathrani A; Cowling, M; Jha, M
Poor academic performance of students is a concern in the educational sector, especially if it leads to students being unable to meet minimum course requirements. However, with timely prediction of students’ performance, educators can detect at-risk students, thereby enabling early interventions for supporting these students in overcoming their learning difficulties. However, the majority of studies have taken the approach of developing individual models that target a single course while developing prediction models. These models are tailored to specific attributes of each course amongst a very diverse set of possibilities. While this approach can yield accurate models in some instances, this strategy is associated with limitations. In many cases, overfitting can take place when course data is small or when new courses are devised. Additionally, maintaining a large suite of models per course is a significant overhead. This issue can be tackled by developing a generic and course-agnostic predictive model that captures more abstract patterns and is able to operate across all courses, irrespective of their differences. This study demonstrates how a generic predictive model can be developed that identifies at-risk students across a wide variety of courses. Experiments were conducted using a range of algorithms, with the generic model producing an effective accuracy. The findings showed that the CatBoost algorithm performed the best on our dataset across the F-measure, ROC (receiver operating characteristic) curve and AUC scores; therefore, it is an excellent candidate algorithm for providing solutions on this domain given its capabilities to seamlessly handle categorical and missing data, which is frequently a feature in educational datasets.
Digital divide framework: online learning in developing countries during the COVID-19 lockdown
(Taylor and Francis Group, 2022) Mathrani A; Sarvesh T; Umer R
This article showcases digital inequalities that came to the forefront for online learning during the COVID-19 lockdown across five developing countries, India, Pakistan, Bangladesh, Nepal and Afghanistan. Large sections of population in developing economies have limited access to basic digital services; this, in turn, restricts how digital media are being used in everyday lives. A digital divide framework encompassing three analytical perspectives, structure, cultural practices and agency, has been developed. Each perspective is influenced by five constructs, communities, time, location, social context and sites of practice. Community relates to gendered expectations, time refers to the lockdown period while locations are interleaved online classrooms and home spaces. Societal contexts influence aspects of online learning and how students engage within practice sites. We find structural issues are due to lack of digital media access and supporting services; further that female students are more often placed lower in the digital divide access scale. Cultural practices indicate gendered discriminatory rules, with female students reporting more stress due to added household responsibilities. This impacts learner agency and poses challenges for students in meaningfully maximising their learning outcomes. Our framework can inform policy-makers to plan initiatives for bridging digital divide and set up equitable gendered learning policies.
The perceived benefits of apps by construction professionals in New Zealand
(MDPI AG, 1/12/2017) Liu T; Mbachu J; Mathrani A; Jones B; McDonald B
The construction sector is a key driver of economic growth in New Zealand; however, its productivity is still considered to be low. Prior research has suggested that information and communication technology (ICT) can help enhance efficiency and productivity. However, there is little research on the use of mobile technologies by New Zealand construction workforce. This paper reports findings of an exploratory study with the objective of examining the perceived benefits regarding uptake of apps in New Zealand construction sector. Using self-administered questionnaire survey, feedback was received from the major construction trade and professional organisations in New Zealand. Survey data was analyzed using descriptive, one-sample t-test, Spearman’s rank correlation coefficient and structural equation modeling. Results showed that iPhone and Android phone currently dominate the smartphone market in New Zealand construction industry. The top three application areas are site photos, health and safety reporting and timekeeping. The benefits of mobile apps were widely confirmed by the construction professionals. The benefit of “better client relationship management and satisfaction” has substantial correlation with overall productivity improvement and best predictor of the overall productivity improvement. These findings provide a starting point for further research aimed at improving the uptake and full leveraging of mobile technologies to improve the dwindling productivity trend in New Zealand construction industry.
Exogenous and endogenous knowledge structures in dual-mode course deliveries
(Elsevier Ltd, 5/12/2020) Mathrani S; Mathrani A; Khatun M
Learning Management Systems (LMS) have facilitated any-time any-place learning; thereby enabling educational institutions to embrace dual-mode teaching by engaging with on- and off-campus student cohorts. This study examines how lecturing staff build on existing knowledge structures when tasked with delivery of the same course concurrently over dual modes. Exogeneous knowledge structures are constructed from one's environment, while endogenous structures are rooted in how lecturing staff assimilate their teaching and learning environments to meet the needs of dual-mode learners. Activity Theory provided the theoretical lens and has revealed how exogeneous structures are influenced by the outer triangle elements, that in turn inform on interactions occurring among the inner triangle elements to construct endogenous knowledge. Seventeen lecturing staff participated via open-ended surveys to share their pedagogic approaches for maintaining equable learning experiences across both student cohorts. Findings show that staff can be constrained by LMS functionalities, course curriculum, course structure and lack of technical support. Staff employed additional online tools, changed assessment methods and encouraged online discussions to bring more parity across both learner groups. Our study provides new insights on dual-mode teaching deliveries and shares how lecturing staff develop new knowledge structures from their teaching practice. It will further help in development of instructional strategies especially post Covid-19, when it is likely that teaching modes will have more online component compared to current times.
Gender Diversity Population Simulations in an Extended Game of Life Context
(IEEE, 20/06/2019) Mathrani A; Scogings C; Mathrani S
Cellular automata studies have been instrumental in computational and biological studies for simulating life contours based on simple rule-based strategies. Game of Life (GoL) presented us with one of the earliest automata studies that led the way in exemplifying non-linear spatial representations, such as large-scale population evolution scenarios depicting species dominance, species equilibrium, and species extinction. However, the GoL was driven by interactions among vegetative entities comprising live and die states only. This paper extends GoL to gendered-GoL (g-GoL) in which male phenotypes and female phenotypes interact in an extended world to procreate. Using the g-GoL, we have demonstrated many evolution contours by applying gender-based dependence rules. Evolution scenarios have been simulated with skewed gender ratios that favor the birth of male offspring. Preference for a male child is common in certain cultures; therefore, empirical data realized with skewed gender settings in g-GoL can reveal the long-term impact of non-egalitarian gender societal structures. Our model provides a tool for the study of emergent life contours and brings awareness on current gender imbalances to strengthen multi-disciplinary research inquiry in the areas of social practices, mathematical modeling, and use of computational technologies.

Journal Articles

Browse

Filters

Settings

Sort By

Results per page

Search Results