Massey Documents by Type

Permanent URI for this communityhttps://mro.massey.ac.nz/handle/10179/294

Browse

Subject: search.filters.subject.Software testing

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Test flakiness’ causes, detection, impact and responses: A multivocal review
    (Elsevier Inc, 2023-12) Tahir A; Rasheed S; Dietrich J; Hashemi N; Zhang L
    Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software testing. They are known to cause significant issues, such as reducing the effectiveness and efficiency of testing and delaying software releases. In recent years, there has been an increased interest in flaky tests, with research focusing on different aspects of flakiness, such as identifying causes, detection methods and mitigation strategies. Test flakiness has also become a key discussion point for practitioners (in blog posts, technical magazines, etc.) as the impact of flaky tests is felt across the industry. This paper presents a multivocal review that investigates how flaky tests, as a topic, have been addressed in both research and practice. Out of 560 articles we reviewed, we identified and analysed a total of 200 articles that are focused on flaky tests (composed of 109 academic and 91 grey literature articles/posts) and structured the body of relevant research and knowledge using four different dimensions: causes, detection, impact and responses. For each of those dimensions, we provide categorization and classify existing research, discussions, methods and tools With this, we provide a comprehensive and current snapshot of existing thinking on test flakiness, covering both academic views and industrial practices, and identify limitations and opportunities for future research.
  • Thumbnail Image
    Item
    A novel tool for resource utilisation regression testing of JVM-based applications : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University, Manawatu, New Zealand
    (Massey University, 2013) Hewson, Fergus
    In recent years, automated regression testing with tools, like JUnit, has become a cornerstone of modern software engineering practice. Automated testing focuses on the functional aspects of software. Automated performance testing is less popular. Where used, it is based on the principle of comparing measurements against static thresholds representing parameters such as system runtime or memory usage. This thesis presents an alternative approach to automatically generate test oracles from system calibration runs. This approach is particularly useful to safeguard the investment made when software is manually fine-tuned. A proof-ofconcept tool was developed that works for all applications that can be deployed on the Java Virtual Machine (JVM), and is capable of testing all properties that can be accessed through Java Management Extensions (JMX) technology.