Journal Articles

Permanent URI for this collectionhttps://mro.massey.ac.nz/handle/10179/7915

Browse

Search Results

Now showing 1 - 10 of 20

Automating Systematic Literature Reviews with Retrieval-Augmented Generation: A Comprehensive Overview
(MDPI (Basel, Switzerland), 2024-10-09) Han B; Susnjak T; Mathrani A; Garcia Villalba LJ
This study examines Retrieval-Augmented Generation (RAG) in large language models (LLMs) and their significant application for undertaking systematic literature reviews (SLRs). RAG-based LLMs can potentially automate tasks like data extraction, summarization, and trend identification. However, while LLMs are exceptionally proficient in generating human-like text and interpreting complex linguistic nuances, their dependence on static, pre-trained knowledge can result in inaccuracies and hallucinations. RAG mitigates these limitations by integrating LLMs’ generative capabilities with the precision of real-time information retrieval. We review in detail the three key processes of the RAG framework—retrieval, augmentation, and generation. We then discuss applications of RAG-based LLMs to SLR automation and highlight future research topics, including integration of domain-specific LLMs, multimodal data processing and generation, and utilization of multiple retrieval sources. We propose a framework of RAG-based LLMs for automating SRLs, which covers four stages of SLR process: literature search, literature screening, data extraction, and information synthesis. Future research aims to optimize the interaction between LLM selection, training strategies, RAG techniques, and prompt engineering to implement the proposed framework, with particular emphasis on the retrieval of information from individual scientific papers and the integration of these data to produce outputs addressing various aspects such as current status, existing gaps, and emerging trends.
Effectiveness of a Learning Analytics Dashboard for Increasing Student Engagement Levels
(Society for Learning Analytics Research (SoLAR), 2023-12-22) Ramaswami G; Susnjak T; Mathrani A
Learning Analytics Dashboards (LADs) are gaining popularity as a platform for providing students with insights into their learning behaviour patterns in online environments. Existing LAD studies are mainly centred on displaying students’ online behaviours with simplistic descriptive insights. Only a few studies have integrated predictive components, while none possess the ability to explain how the predictive models work and how they have arrived at specific conclusions for a given student. A further gap exists within existing LADs with respect to prescriptive analytics that generate data-driven feedback to students on how to adjust their learning behaviour. The LAD in this study attempts to address this gap and integrates a full spectrum of current analytics technologies for sense-making while anchoring them within theoretical educational frameworks. This study’s LAD (SensEnablr) was evaluated for its effectiveness in impacting learning in a student cohort at a tertiary institution. Our findings demonstrate that student engagement with learning technologies and course resources increased significantly immediately following interactions with the dashboard. Meanwhile, results showed that the dashboard boosted the respondents’ learning motivation levels and that the novel analytics insights drawn from predictive and prescriptive analytics were beneficial to their learning. This study, therefore, has implications for future research when investigating student outcomes and optimizing student learning using LAD technologies.
Consumer behavior in immersive virtual reality retail environments: A systematic literature review using the stimuli-organisms-responses (S-O-R) model
(Wiley, 2024-07-10) Erensoy A; Mathrani A; Schnack A; Elms J; Baghaei N
With the rising popularity of immersive virtual reality (iVR) technologies, retailers are increasingly seeking innovative ways to create unique digital shopping experiences for their consumers. However, existing literature lacks a unified and comprehensive review that examines the interplay between virtual stimuli and consumer behavior in iVR shopping environments. To fill this gap, we conducted a systematic literature review, employing the Stimulus-Organisms-Responses (S-O-R) model as the underlying theoretical framework. This review analyzed empirical research on consumer behavior in iVR retail environments by focusing on experimental studies. Following the thematic analysis, we categorized the outcomes into descriptive themes to better comprehend consumer behavior within each theme. Our findings provide valuable insights for retailers and marketers aiming to enhance the consumer shopping experience using iVR technologies and suggest directions for future research.
Use of Predictive Analytics within Learning Analytics Dashboards: A Review of Case Studies
(Springer Nature BV, 2023-09-01) Ramaswami G; Susnjak T; Mathrani A; Umer R
Learning analytics dashboards (LADs) provide educators and students with a comprehensive snapshot of the learning domain. Visualizations showcasing student learning behavioral patterns can help students gain greater self-awareness of their learning progression, and at the same time assist educators in identifying those students who may be facing learning difficulties. While LADs have gained popularity, existing LADs are still far behind when it comes to employing predictive analytics into their designs. Our systematic literature review has revealed limitations in the utilization of predictive analytics tools among existing LADs. We find that studies leveraging predictive analytics only go as far as identifying the at-risk students and do not employ model interpretation or explainability capabilities. This limits the ability of LADs to offer data-driven prescriptive advice to students that can offer them guidance on appropriate learning adjustments. Further, published studies have mostly described LADs that are still at prototype stages; hence, robust evaluations of how LADs affect student outcomes have not yet been conducted. The evaluations until now are limited to LAD functionalities and usability rather than their effectiveness as a pedagogical treatment. We conclude by making recommendations for the design of advanced dashboards that more fully take advantage of machine learning technologies, while using suitable visualizations to project only relevant information. Finally, we stress the importance of developing dashboards that are ultimately evaluated for their effectiveness.
Methodological Aspects in Study of Fat Stigma in Social Media Contexts: A Systematic Literature Review
(MDPI (Basel, Switzerland), 2022-05-17) Wanniarachchi V; Mathrani A; Susnjak T; Scogings C; Moreno, A
With increased obesity rates worldwide and the rising popularity in social media usage, we have witnessed a growth in hate speech towards fat/obese people. The severity of hate content has prompted researchers to study public perceptions that give rise to fat stigma from social media discourses. This article presents a systematic literature review of recent literature published in this domain to gauge the current state of research and identify possible research gaps. We have examined existing research (i.e., peer-reviewed articles that were systematically included using the EBSCO discovery service) to study their methodological aspects by reviewing their context, domain, analytical methods, techniques, tools, features and limitations. Our findings reveal that while recent studies have explored fat stigma content in social media, these mostly acquired manual analytical methods regardless of the evolved machine learning, natural language processing and deep learning methods. Although fat stigma in social media has gained enormous attention in current socio-psychological research, there exists a gap between how such research is conducted and what technologies are being applied, which limits in-depth investigations of fat stigma discussions.
Data Quality Challenges in Educational Process Mining: Building Process-Oriented Event Logs from Process-Unaware Online Learning Systems
(Inderscience, 2022-05-04) Umer R; Susnjak T; Mathrani A; Suriadi S
Educational process mining utilizes process-oriented event logs to enable discovery of learning practices that can be used for the learner’s advantage. However, learning platforms are often process-unaware, therefore do not accurately reflect ongoing learner interactions. We demonstrate how contextually relevant process models can be constructed from process-unaware systems. Using a popular learning management system (Moodle), we have extracted stand-alone activities from the underlying database and formatted it to link the learners’ data explicitly to process instances (cases). With a running example that describes quiz-taking activities undertaken by students, we describe how learner interactions can be captured to build process-oriented event logs. This article contributes to the fields of learning analytics and education process mining by providing lessons learned on the extraction and conversion of process-unaware data to event logs for the purpose of analysing online education data.
Online Tracking: When Does it Become Stalking?
(World Scientific Publishing, 2021-05-25) Amarasekara B; Mathrani A; Scogings C
Online user activities are tracked for many purposes. In e-commerce, cross-domain tracking is used to quantify and pay for web-tra±c generation. Our previous research studies have shown that HTTP cookie-based tracking process, though reliable, can fail due to technical reasons, as well as through fraudulent manipulation by tra±c generators. In this research study, we evaluate which of the previously published tracking mechanisms are still functional. We assess the e±cacy and utility of those methods to create a robust tracking mechanism for e-commerce. A failsafe and robust tracking mechanism does not need to translate into further privacy intrusions. Many countries are rushing to introduce new regulations, which can have a negative impact on the development of robust technologies in an inherently stateless eco-system. We used a multi-domain, purpose-built simulation environment to experiment common tracking scenarios, and to describe the parameters that de¯ne the minimum tracking requirement use-cases, and practices that result in invading privacy of users. This study will help practitioners in their implementations, and policy developers and regulators to draw up policies that would not curtail the development of robust tracking technologies that are needed in e-commerce activities, while safeguarding the privacy of internet users.
Supporting Students’ Academic Performance Using Explainable Machine Learning with Automated Prescriptive Analytics
(MDPI (Basel, Switzerland), 2022-12) Ramaswami G; Susnjak T; Mathrani A
Learning Analytics (LA) refers to the use of students’ interaction data within educational environments for enhancing teaching and learning environments. To date, the major focus in LA has been on descriptive and predictive analytics. Nevertheless, prescriptive analytics is now seen as a future area of development. Prescriptive analytics is the next step towards increasing LA maturity, leading to proactive decision-making for improving students’ performance. This aims to provide data-driven suggestions to students who are at risk of non-completions or other sub-optimal outcomes. These suggestions are based on what-if modeling, which leverages machine learning to model what the minimal changes to the students’ behavioral and performance patterns would be required to realize a more desirable outcome. The results of the what-if modeling lead to precise suggestions that can be converted into evidence-based advice to students. All existing studies in the educational domain have, until now, predicted students’ performance and have not undertaken further steps that either explain the predictive decisions or explore the generation of prescriptive modeling. Our proposed method extends much of the work performed in this field to date. Firstly, we demonstrate the use of model explainability using anchors to provide reasons and reasoning behind predictive models to enable the transparency of predictive models. Secondly, we show how prescriptive analytics based on what-if counterfactuals can be used to automate student feedback through prescriptive analytics.
Perspectives on the challenges of generalizability, transparency and ethics in predictive learning analytics
(Elsevier Ltd, 2021-11-20) Mathrani A; Susnjak T; Ramaswami G; Barczak A
Educational institutions need to formulate a well-established data-driven plan to get long-term value from their learning analytics (LA) strategy. By tracking learners’ digital traces and measuring learners’ performance, institutions can discern consequential learning trends via use of predictive models to enhance their instructional services. However, questions remain on how the proposed LA system is suitable, meaningful, and justifiable. In this concept paper, we examine generalizability and transparency of the internals of predictive models, alongside the ethical challenges in using learners’ data for building predictive capabilities. Model generalizability or transferability is hindered by inadequate feature representation, small and imbalanced datasets, concept drift, and contextually un-related domains. Additional challenges relate to trustworthiness and social acceptance of these models since algorithmic-driven models are difficult to interpret by themselves. Further, ethical dilemmas are faced in engaging with learners’ data while developing and deploying LA systems at an institutional level. We propose methodologies for apprehending these challenges by establishing efforts for managing transferability and transparency, and further assessing the ethical standing on justifiable use of the LA strategy. This study showcases underlying relationships that exist between constructs pertaining to learners’ data and the predictive model. We suggest the use of appropriate evaluation techniques and setting up research ethics protocols, since without proper controls in place, the model outcome would not be portable, transferable, trustworthy, or admissible as a responsible outcome. This concept paper has theoretical and practical implications for future inquiry in the burgeoning field of learning analytics.
On Developing Generic Models for Predicting Student Outcomes in Educational Data Mining
(MDPI (Basel, Switzerland), 2022-01-07) Ramaswami G; Susnjak T; Mathrani A; Cowling, M; Jha, M
Poor academic performance of students is a concern in the educational sector, especially if it leads to students being unable to meet minimum course requirements. However, with timely prediction of students’ performance, educators can detect at-risk students, thereby enabling early interventions for supporting these students in overcoming their learning difficulties. However, the majority of studies have taken the approach of developing individual models that target a single course while developing prediction models. These models are tailored to specific attributes of each course amongst a very diverse set of possibilities. While this approach can yield accurate models in some instances, this strategy is associated with limitations. In many cases, overfitting can take place when course data is small or when new courses are devised. Additionally, maintaining a large suite of models per course is a significant overhead. This issue can be tackled by developing a generic and course-agnostic predictive model that captures more abstract patterns and is able to operate across all courses, irrespective of their differences. This study demonstrates how a generic predictive model can be developed that identifies at-risk students across a wide variety of courses. Experiments were conducted using a range of algorithms, with the generic model producing an effective accuracy. The findings showed that the CatBoost algorithm performed the best on our dataset across the F-measure, ROC (receiver operating characteristic) curve and AUC scores; therefore, it is an excellent candidate algorithm for providing solutions on this domain given its capabilities to seamlessly handle categorical and missing data, which is frequently a feature in educational datasets.

Journal Articles

Browse

Filters

Settings

Sort By

Results per page

Search Results