Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer.
Abstract
Keywords: Stack Overflow Title Generation CodeT5 Nucleus Sampling Maximal Marginal Ranking Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems. Nevertheless, if inexperienced developers fail to describe their problems clearly, it is hard for them to attract sufficient attention and get the anticipated answers. To address such a problem, we propose M3 NSCT5, a novel approach to automatically generate multiple post titles from the given code snippets. Developers may take advantage of the generated titles to find closely related posts and complete their problem descriptions. M3 NSCT5 employs the CodeT5 backbone, which is a pre-trained Transformer model having an excellent language understanding and generation ability. To alleviate the ambiguity issue that the same code snippets could be aligned with different titles under varying contexts, we propose the maximal marginal multiple nucleus sampling strategy to generate multiple high-quality and diverse title candidates at a time for the developers to choose from. We build a large-scale dataset with 890,000 question posts covering eight programming languages to validate the effectiveness of M3 NSCT5. The automatic evaluation results on the BLEU and ROUGE metrics demonstrate the superiority of M3 NSCT5 over six state-of-the-art baseline models. Moreover, a human evaluation with trustworthy results also demonstrates the great potential of our approach for real-world application. 1. Introduction Stack Overflow (SO) is one of the most popular Question&Answering websites for developers to seek answers to programming problems. However, it remains a challenge [4, 32, 37] to help developers write high-quality question posts that attract enough attention from potential experts. Especially, non-native English speakers or inexperienced developers may struggle to clearly describe their encountered problems, let alone summarize the problems into informative titles. One way for developers to write better question posts is to first search for related posts with the problematic code snippets and then complete their problem descriptions and post titles. None