ADVANTAGES AND DISADVANTAGES OF GENERATING CODE USING LARGE LANGUAGE MODELS IN MODERN IDE
DOI:
https://doi.org/10.35546/kntu2078-4481.2025.2.2.36Keywords:
AI-assistant, code generation, IDE, LLMAbstract
In this paper we discuss ways to use large language models to generate program code. The focus is on IDEs and their built-in artificial intelligence for such tasks as creating projects, building software application architectures, and prototyping applications based on prompts. AI-generated code lacks traceability and often requires iterative refinement, challenges that existing IDEs are unable to address. Unlike human-created code, generated code does not integrate well with structured development workflows. As a result, developers struggle with AI hallucinations, when models generate plausible but incorrect code that is difficult to verify. This research has identified situations of incorrect output of LLMs as intent misuse, hallucination, missing items misuse, and redundancy misuse. Misuse of third-party APIs in generated code is a serious problem in software development. Although LLMs have demonstrated impressive code generation capabilities, their interaction with complex library APIs remains highly error-prone, potentially leading to software malfunctions and security vulnerabilities. The obtained results show that code autocomplete is more useful, as it makes it possible to generate code on demand in compliance with the specified restrictions and with a lower probability of errors. However, among the disadvantages of this approach is the lack of the ability to generate code in several files at once. The results obtained will contribute to the optimal choice of options for using AI and LLMs in application development. This work is aimed at improving the understanding of scenarios for using LLMs in working with program code.
References
Šavelka, J., Agarwal, A., Bogart, C., Song, Y., & Sakr, M. F. (2023). Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses? Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. DOI: https://doi.org/10.48550/arXiv.2303.09325
Reeves, B., Sarsa, S., Prather, J., Denny, P., Becker, B.A., Hellas, A., Kimmel, B., Powell, G., & Leinonen, J. (2023). Evaluating the performance of code generation models for solving parsons problems with small prompt variations. Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. DOI: https://doi.org/10.1145/3587102.3588805
Denny, P., Kumar, V., & Giacaman, N. (2023). Conversing with Copilot: Exploring prompt engineering for solving CS1 problems using natural language. Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. DOI: https://doi.org/10.1145/3545945.3569823
Finnie-Ansley, J., Denny, P., Becker, B.A., Luxton-Reilly, A., & Prather, J. (2022). The robots are coming: Exploring the implications of openai codex on introductory programming. Proceedings of the 24th Australasian Computing Education Conference. DOI: https://doi.org/10.1145/3511861.3511863
Zhong, L., Wang, Z. (2024). Can LLM replace stack overflow? A study on robustness and reliability of large language model code generation. Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 19. DOI: https://doi.org/10.48550/arXiv.2308.10335
Mousavi, Z., Islam, C., Moore, K., Abuadbba, A. & Babar, M. A. (2024). An investigation into misuse of java security apis by large language models. Proceedings of the 19th ACM Asia Conference on Computer and Communications Security. DOI: https://doi.org/10.48550/arXiv.2404.03823
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022). Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. Chi conference on human factors in computing systems extended abstracts. https://doi.org/10.1145/3491101.3519665
Zhuo, T.Y., He, J., Sun, J., Xing, Z., Lo, D., Grundy, J., & Du, X. (2025). Identifying and Mitigating API Misuse in Large Language Models. DOI: https://doi.org/10.48550/arXiv.2503.22821
