A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts.
Abstract
Syntax Tree (AST), and then extracted the semantic information of code from AST. For example, Hu et al. [11] firstly traversed the AST by the (Structure-based Traversal) SBT method to obtain the SBT sequences, then used the plain source code and the SBT sequences as inputs to learn the semantic information of source code. LeClair et al. [12] regarded the AST as a graph, and used the Graph Neural Network (GNN)-based encoder to model the AST, combined with the RNN-based encoder to model the plain source code. However, ASTs can be represented as multiple modalities, such as SBT sequences and graphs, each focusing on a distinct aspect of the semantic information. Therefore, it is not comprehensive to use a single AST modality to represent the semantic information of code. (2) How to capture the long-range dependencies between code tokens. The source code of smart contract methods can be very long. In our experiment dataset, smart contract methods contain 58.72 code tokens on average, and the longest one contains 3,272 code tokens. Previous works often applied the Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), or Gated Recurrent Unit (GRU) models to extract features from their inputs, which have shown to be difficult to capture the long-range dependencies between code tokens [13], [14]. In order to address the above challenges, in this paper, we propose a Multi-Modal Transformer-based (MMTrans) code summarization approach for smart contracts. Firstly, the Abstract-Code comment has been an important part of computer programs, greatly facilitating the understanding and maintenance of source code. However, high-quality code comments are often unavailable in smart contracts, the increasingly popular programs that run on the blockchain. In this paper, we propose a Multi-Modal Transformer-based (MMTrans) code summarization approach for smart contracts. Specifically, the MMTrans learns the representation of source code from the two heterogeneous modalities of the Abstract Syntax Tree (AST), i.e., Structure-based Traversal (SBT) sequences and graphs. The SBT sequence provides the global semantic information of AST, while the graph convolution focuses on the