Abstract
The increasing use of Large Language Models (LLMs) in software development has garnered significant attention from researchers evaluating the capabilities and limitations of LLMs for code generation. However, much of the research focuses on controlled datasets such as HumanEval, which do not adequately capture the characteristics of LLM-generated code in real-world development scenarios. To address this gap, our study investigates self-admitted code generated by LLMs on GitHub, specifically focusing on instances where developers in projects with over five stars acknowledge the use of LLMs to generate code through code comments. Our findings reveal several key insights: (1) ChatGPT and Copilot dominate code generation, with minimal contributions from other LLMs. (2) Projects containing ChatGPT/Copilot-generated code appears in small/medium-sized projects led by small teams, which are continuously evolving. (3) ChatGPT/Copilot-generated code generally is a minor project portion, primarily generating short/moderate-length, lowcomplexity snippets (e.g., algorithms and data structures code; text processing code). (4) ChatGPT/Copilot-generated code generally undergoes minimal modifications, with bug-related changes ranging from 4% to 12%. (5) Most code comments only state LLM use, while few include details like prompts, human edits, or code testing status. Based on these findings, we discuss the implications for researchers and practitioners. Index Terms-Empirical study, Large Language Models, Code Generation, GitHub I. I NTRODUCTION Large Language Models (LLMs), trained on massive amounts of text and code, have been widely applied to various software engineering tasks [1]-[6]. Their growing adoption, particularly OpenAI's GPT series models, in software development has attracted considerable interest among researchers in examining the quality of code generated by these models. Some studies focus on proposing benchmarks for code generation, such as the widely used HumanEval [7], MBXP [8], as well as recently proposed CoderEval [9], ClassEval [10], and EvoCodeBench [11], which address more complex real-world programming tasks (i.e., non-standalone function generation