First international Competition on Runtime Verification: rules, benchmarks, tools, and final results of CRV 2014

E. Bartocci (1) , Y. Falcone (2, 3) , B. Bonakdarpour (4) , C. Colombo (5) , N. Decker (6) , K. Havelund (7) , Y. Joshi (8) , F. Klaedtke (9) , R. Milewicz (10) , G. Reger (11) , G. Rosu (3) , J. Signoles (12) , D. Thoma (6) , E. Zalinescu (13) , Y. Zhang (3)

1 TU Wien - Vienna University of Technology
2 LIG - Laboratoire d'Informatique de Grenoble
3 University of Illinois at Urbana-Champaign [Urbana]
4 McMaster University [Hamilton, Ontario]
5 University of Malta [Malta]
6 Lübeck University of Applied Sciences
7 JPL - Jet Propulsion Laboratory
8 University of Waterloo [Waterloo]
9 NEC Europe Ltd., Network Laboratories
10 UAB - University of Alabama at Birmingham [ Birmingham]
11 University of Manchester [Manchester]
12 DILS - Département Ingénierie Logiciels et Systèmes
13 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]

The first international Competition on Runtime Verification (CRV) was held in September 2014, in Toronto, Canada, as a satellite event of the 14th international conference on Runtime Verification (RV’14). The event was organized in three tracks: (1) offline monitoring, (2) online monitoring of C programs, and (3) online monitoring of Java programs. In this paper, we report on the phases and rules, a description of the participating teams and their submitted benchmark, the (full) results, as well as the lessons learned from the competition.

Keywords: Verification; Benchmarking; C (programming language); Competition; Computer software; Monitoring; Partial discharges; C programs; International competitions; Java program; Off-line monitoring; Online monitoring; Participating teams; Run-time verification; Toronto; Canada

Domains: Computer Science [cs]

E. Bartocci, Y. Falcone, B. Bonakdarpour, C. Colombo, N. Decker, et al.. First international Competition on Runtime Verification: rules, benchmarks, tools, and final results of CRV 2014. International Journal on Software Tools for Technology Transfer, 2017, pp.1-40. ⟨10.1007/s10009-017-0454-5⟩. ⟨cea-01845191⟩