DI-UMONS : Dépôt institutionnel de l’université de Mons

Recherche transversale
(titres de publication, de périodique et noms de colloque inclus)
2022-03-23 - Colloque/Article dans les actes avec comité de lecture - Anglais - 4 page(s) (A publier)

Golzadeh Mehdi , Decan Alexandre , Chidambaram Natarajan , "On the Accuracy of Bot Detection Techniques" in 4th Workshop on Bots in Software Engineering (BotSE), Pittsburgh, USA, 2022

  • Codes CREF : Informatique appliquée logiciel (DI2570), Informatique générale (DI1162), Analyse de systèmes informatiques (DI2572)
  • Unités de recherche UMONS : Génie Logiciel (S852)
  • Instituts UMONS : Institut de Recherche en Technologies de l’Information et Sciences de l’Informatique (InforTech), Institut de Recherche sur les Systèmes Complexes (Complexys)
Texte intégral :

Abstract(s) :

(Anglais) Development bots are often used to automate a wide variety of repetitive tasks in collaborative software development. Such bots are commonly among the most active project contributors in terms of commit activity. As such, tools that analyse contributor activity (e.g., for recognizing and giving credit to project members for their contributions) need to take into account the bots and exclude their activity. While there are a few techniques to detect bots in software repositories, these techniques are not perfect and may miss some bots or may wrongly identify some human accounts as bots. In this paper, we present an exploratory study on the accuracy of bot detection techniques on a set of 540 accounts from 27 GitHub projects. We show that none of the bot detection techniques are accurate enough to detect bots among the 20 most active contributors of each project. We show that combining these techniques drastically increases the accuracy and recall of bot detection. We also highlight the importance of considering bots when attributing contributions to humans, since bots are prevalent among the top contributors and responsible for large proportions of commits.

Identifiants :
  • FNRS : O.0157.18F-RG43 and T.0017.18