Massey Documents by Type

Permanent URI for this communityhttps://mro.massey.ac.nz/handle/10179/294

Browse

Filters

Settings

Subject: search.filters.subject.Parallel processing (Electronic computers)

Search Results

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    Performance modelling, analysis and prediction of Spark jobs in Hadoop cluster : a thesis by publications presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science, School of Mathematical & Computational Sciences, Massey University, Auckland, New Zealand
    (Massey University, 2022) Ahmed, Nasim
    Big Data frameworks have received tremendous attention from the industry and from academic research over the past decade. The advent of distributed computing frameworks such as Hadoop MapReduce and Spark are powerful frameworks that offer an efficient solution for analysing large-scale datasets running under the Hadoop cluster. Spark has been established as one of the most popular large-scale data processing engines because of its speed, low latency in-memory computation, and advanced analytics. Spark computational performance heavily depends on the selection of suitable parameters, and the configuration of these parameters is a challenging task. Although Spark has default parameters and can deploy applications without much effort, a significant drawback of default parameter selection is that it is not always the best for cluster performance. A major limitation for Spark performance prediction using existing models is that it requires either large input data or system configuration that is time-consuming. Therefore, an analytical model could be a better solution for performance prediction and for establishing appropriate job configurations. This thesis proposes two distinct parallelisation models for performance prediction: the 2D-Plate model and the Fully-Connected Node model. Both models were constructed based on serial boundaries for a certain arrangement of executors and size of the data. In order to evaluate the cluster performance, various HiBench workloads were used, and workload’s empirical data were fitted with the models for performance prediction analysis. The developed models were benchmarked with the existing models such as Amdahl’s, Gustafson, ERNEST, and machine learning. Our experimental results show that the two proposed models can quickly and accurately predict performance in terms of runtime, and they can outperform the accuracy of machine learning models when extrapolating predictions.
  • Thumbnail Image
    Item
    Parallel simulation methods for large-scale agent-based predator-prey systems : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science at Massey University, Albany, New Zealand
    (Massey University, 2019) Quach, Dara (Minh) Quang
    The Animat is an agent-based artificial-life model that is suitable for gaining insight into the interactions of autonomous individuals in complex predator-prey systems and the emergent phenomena they may exhibit. Certain dynamics of the model may only be present in large systems, and a large number of agents may be required to compare with macroscopic models. Large systems can be infeasible to simulate on single-core machines due to processing time required. The model can be parallelised to improve the performance; however, reproducing the original model behaviour and retaining the performance gain is not straightforward. Parallel update strategies and data structures for multi-core CPU and graphical processing units (GPUs) are developed to simulate a typical predator-prey Animat model with improved perfor- mance while reproducing the behaviour of the original model. An analysis is presented of the model to identify dependencies and conditions the parallel update strategy must satisfy to retain original model behaviour. The parallel update strategy for multi-core CPUs is constructed using a spatial domain decomposition approach and supporting data structure. The GPU implementation is developed with a new update strategy that consists of an iterative conflict resolution method and priority number system to simultaneously update many agents with thousands of GPU cores. This update method is supported by a compressed sparse data structure developed to allow for efficient memory transactions. The performance of the Animat simulation is improved with parallelism and without a change in model behaviour. The simulation usability is considered, and an internal agent definition system using a CUDA device Lambda feature is developed to improve the ease of configuring agents without significant changes to the program and loss of performance.
  • Thumbnail Image
    Item
    Parallel processor implementation in computerized tomography using transputers : a thesis presented in partial fulfilment of the requirement for the degree of Master of Technology in Production Technology at Massey University
    (Massey University, 1993) Wu, Wenju
    Image reconstruction by computerized tomography provides a nonintrusive method of imaging the internal structure of objects. From measurements of radiation (e.g. X-rays or gamma rays) passed through an object, it is possible to observe the internal structure. The reconstruction process is computationally intensive and requires imaginative parallel processing algorithms to attain 'real­ time' performance. The Inmos transputer makes parallel processing algorithms both feasible and relatively straight forward. In this thesis, a modification to the backprojection algorithm is introduced in order to improve the speed of the implementation. Work carried out has involved evaluating how these algorithms ( convolution, backprojection and interpolation ) can be used in multiprocessor concurrent architecture to obtain rapid image reconstruction. Several suitable transputer network structures have been advanced to simulate the image reconstruction. The reconstruction time is decreased very greatly and the image reconstruction result is good.
  • Thumbnail Image
    Item
    A peer-to-peer message passing system for parallel computing : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University, Albany, New Zealand
    (Massey University, 2004) de Klerk
    This thesis presents an implementation of a computational grid system that utilises network-enabled computers to execute parallel applications. The system is fully decentralised and self-configuring and handles the joining and departure of nodes transparently to the user. The system allows users to specify the resources such as operating system, network connection speed and system memory their applications require. It then searches for these resources and executes the applications on appropriate peers. It uses checkpointing of application processes to allow a running process to vacate a host and migrate to another peer when the host leaves the network. Process migration is transparent to the user and the processes automatically find the new address of the migrated process when migration is completed.
  • Thumbnail Image
    Item
    Concurrent Viola Jones classifiers on a portable Beowulf cluster : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Mechatronics at Massey University
    (Massey University, 2008) Chemudugunta, Ravi Kiran
    Real-time Computer Vision is an interesting application for supercomputing, real-time applications (vision processing in particular) employ special purpose hardware such as DSPs to achieve high performance. This thesis explores parallel computers particularly commodity general purpose hardware. We also build a prototype to better understand the economics of supercomputing, specifically related to mobile computing - low power, rugged design by building a mobile computer. A new communication layer is built, where by the nature of the locality of the nodes allows one to optimise the protocols to reduce the latency comparably. Finally a study and in depth results of the algorithm, the Viola Jones Object detector in parallel are presented followed by reflection and future work based on the current results and platform.