Security weaknesses in AI-generated code : a comparative analysis across models and programming languages : a thesis presented in partial fulfilment of the requirements for the degree of Master of Information Sciences in Software Engineering at Massey University, Palmerston North, New Zealand

dc.contributor.authorSoltanian Fard Johromi, Ali
dc.date.accessioned2026-11-17T20:03:47Z
dc.date.issued2026
dc.description.abstractThe security of AI-generated code remains a major challenge for its widespread adoption. Code generation models have demonstrated strong performance on functional benchmarks. However, their outputs frequently contain bugs and security weaknesses that reduce their reliability in real-world settings. Prior research has explored a range of approaches to mitigate these issues, including static analysis-guided generation and prompt engineering techniques. However, the effectiveness of these methods varies significantly across models, programming languages, and settings. This thesis presents a systematic investigation of strategies for improving the security of model-generated code with respect to Common Weakness Enumeration (CWE) weaknesses. Both prompting-based techniques and fine-tuning approaches for model output refinement are evaluated across multiple large language models and programming languages. In addition to measuring the frequency of security weaknesses, this research analyses their severity, co-occurrence patterns, and whether remediation strategies introduce new weaknesses while attempting to fix existing ones. This research is based on 9,600 code samples generated from scenario-based prompts derived from the MITRE CWE Top 25, using five large language models and four programming languages. All outputs are analysed using static analysis tools, supported by custom detection scripts and manual validation. The baseline analysis of AI-generated code found that security weaknesses were present across all evaluated models, programming languages, and scenarios, with multiple security weaknesses occurring within the same code samples. The results show that security improvements are highly dependent on both the model and refinement strategy used. Although fine-tuning consistently achieves the strongest reductions in security weaknesses, prompting techniques also provide measurable improvements, with Meta Prompting performing best among them. However, all approaches exhibit limitations, and in some cases, introduce new weaknesses as a side effect of remediation. These introduced weaknesses are generally less severe than those present in the original outputs, indicating an overall reduction in security risk despite their occurrence. Furthermore, no single method consistently eliminates security weaknesses across all models and scenarios, highlighting the lack of a universal solution for secure AI code generation. The results highlight the need to integrate security analysis into LLM-based development workflows and to further research more robust methods for secure code generation.
dc.identifier.urihttps://mro.massey.ac.nz/handle/10179/100626
dc.identifier.urihttps://doi.org/10.82921/mro-161
dc.language.isoen
dc.publisherMassey University
dc.rightsThe authoren
dc.subject.anzsrc460202 Autonomous agents and multiagent systems
dc.titleSecurity weaknesses in AI-generated code : a comparative analysis across models and programming languages : a thesis presented in partial fulfilment of the requirements for the degree of Master of Information Sciences in Software Engineering at Massey University, Palmerston North, New Zealand
dc.typeThesis

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
SoltanianFardJahromiMInfScThesis.pdf
Size:
14.29 MB
Format:
Adobe Portable Document Format

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
9.89 KB
Format:
Item-specific license agreed upon to submission
Description: