PRECRIME

ABOUT

PRECRIME: Self-assessment Oracles for Anticipatory Testing

Project: European Research Council - Advanced Grant (ERC-AdG)
Duration: 2019-2023

Summary:
One of the long-lasting dreams in science fiction is the ability to arrest criminals before they even commit crimes. Software testing researchers have a similar dream: when the context for a bug manifestation occurs in the field, the goal is to discover and fix the bug before it causes any in-field failure. In fact, current practice of pre-release testing is severely limited when dealing with autonomous AI (Artificial Intelligence) systems (such as self-driving cars, robots, automated traders, virtual doctors and customer service chatbots), running in complex, rapidly changing environments, which cause their run-time adaptation, learning and knowledge acquisition, because pre-release testing cannot exhaustively explore all different contexts and states in which the software will be running.

The PRECRIME project introduces a new, disruptive view on testing, called anticipatory testing and aimed at fixing bugs before they even manifest themselves in the field. Anticipatory testing is activated at run-time by a new type of oracles, called self-assessment oracles, which observe and report unexpected execution contexts. A self-assessment oracle is an estimator of the system's confidence in being able to handle a new execution context correctly. The goal of anticipatory testing is to anticipate any failure that might occur in the field due to unexpected execution contexts. Whenever an execution context monitored at runtime by self-assessment oracles is estimated as a low confidence context for the system, anticipatory testing exercises the software automatically and extensively in similar contexts. Timely activation of anticipatory testing by runtime observations results in early, anticipatory fault detection. Combined with automated patch synthesis, anticipatory testing leads to the release of a patch for the fault before any software failure occurs in the field.

TEAM

Project Team

Paolo Tonella
Principal Investigator

Paolo Tonella

Andrea Stocco
Postdoc

Andrea Stocco

Gunel Jahangirova
Postdoc

Gunel Jahangirova

Vincenzo Riccio
Postdoc

Vincenzo Riccio

Matteo Biagiola
Postdoc

Matteo Biagiola

Nargiz Humbatova
PhD student

Nargiz Humbatova

Michael Weiss
PhD student

Michael Weiss

Tahereh Zohdi
PhD student

Tahereh Zohdi

Nelson Brochado
PhD student

Nelson Brochado

MSc students

 Marco Calzana, thesis title: "Misbehaviour Prediction in Autonomous Vehicles using Auto-Encoders", September 2019

UROP Summer students

 Simone Giacomelli, Nelson Brochado, project title: "Extreme Testing of Self-Driving Cars", Summer 2019
 Martin Odermatt, project title: "Developing an Interactive Tool to Allow Human Control in MLS Testing", Summer 2020
PUBLICATIONS

Publications

  Gunel Jahangirova, Andrea Stocco, Paolo Tonella: Quality Metrics and Oracles for Autonomous Vehicles Testing. Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST), 2021. (to appear)
  Michael Weiss, Paolo Tonella: Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring. Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST), 2021. (to appear)
  Antonia Bertolino, Pietro Braione, Guglielmo De Angelis, Luca Gazzola, Fitsum Kifetew, Leonardo Mariani, Matteo Orrù, Mauro Pezzè, Roberto Pietrantuono, Stefano Russo, Paolo Tonella: A Survey of Field-based Testing Techniques. ACM Computing Surveys (CSUR), 2021. (to appear)
  Vincenzo Riccio, Gunel Jahangirova, Andrea Stocco, Nargiz Humbatova, Michael Weiss, Paolo Tonella: Testing Machine Learning based Systems: A Systematic Mapping. Empirical Software Engineering (EMSE), 2020.
  Vincenzo Riccio, Paolo Tonella: Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing. Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2020.
  Valerio Terragni, Gunel Jahangirova, Mauro Pezzè, Paolo Tonella: Evolutionary Improvement of Assertion Oracles. Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2020.
  Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella: Taxonomy of Real Faults in Deep Learning Systems. Proceedings of the 42nd International Conference on Software Engineering (ICSE), 2020.
  Andrea Stocco, Michael Weiss, Paolo Tonella: Misbehaviour Prediction for Autonomous Driving Systems. Proceedings of the 42nd International Conference on Software Engineering (ICSE), 2020.
  Gunel Jahangirova, Paolo Tonella: An Empirical Evaluation of Mutation Operators for Deep Learning Systems. Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST), 2020. Best paper award.
  Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella: An Empirical Validation of Oracle Improvement. IEEE Transactions on Software Engineering (TSE), 2019.
  Mariano Ceccato, Davide Corradini, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orrù, Paolo Tonella: A Framework for In-Vivo Testing of Mobile Applications. Proceedings of the 13th IEEE International Conference on Software Testing, Verification and Validation (ICST), 2020.
  Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella: Run Java Applications and Test Them In-Vivo Meantime. Proceedings of the 13th IEEE International Conference on Software Testing, Verification and Validation, Tool Track (ICST-Tool), 2020.
  Mariano Ceccato, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orrù, Paolo Tonella: Toward In-Vivo Testing of Mobile Applications. Proceedings of the 1st International Workshop on Governing adaptive and Unplanned Systems of Systems (GAUSS), 2019.
TECHNICAL REPORTS

Technical Reports

  Vincenzo Riccio, Paolo Tonella: Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing. TR-Precrime-2020-01
  Valerio Terragni, Gunel Jahangirova, Mauro Pezzè, Paolo Tonella: Evolutionary Improvement of Assertion Oracles. TR-Precrime-2020-02
  Vincenzo Riccio, Gunel Jahangirova, Andrea Stocco, Nargiz Humbatova, Michael Weiss, Paolo Tonella: Testing Machine Learning based Systems: A Systematic Mapping. TR-Precrime-2020-03
  Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella: Taxonomy of Real Faults in Deep Learning Systems. TR-Precrime-2019-01
  Andrea Stocco, Michael Weiss, Paolo Tonella: Misbehaviour Prediction for Autonomous Driving Systems. TR-Precrime-2019-02
  Gunel Jahangirova, Paolo Tonella: An Empirical Evaluation of Mutation Operators for Deep Learning Systems. TR-Precrime-2019-03
  Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella: An Empirical Validation of Oracle Improvement. TR-Precrime-2019-04
  Mariano Ceccato, Davide Corradini, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orrù, Paolo Tonella: A Framework for In-Vivo Testing of Mobile Applications. TR-Precrime-2019-05
  Antonia Bertolino, Guglielmo De Angelis, Breno Miranda, Paolo Tonella: Run Java Applications and Test Them In-Vivo Meantime. TR-Precrime-2019-06
  Mariano Ceccato, Luca Gazzola, Fitsum Meshesha Kifetew, Leonardo Mariani, Matteo Orrù, Paolo Tonella: Toward In-Vivo Testing of Mobile Applications. TR-Precrime-2019-07
EVENTS AND TALKS

Events and Invited Talks


  Paolo Tonella: Ensuring quality and dependability of systems based on deep learning. Keynote at the 8th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE). June 29, 2020.
  Andrea Stocco, Gunel Jahangirova, Vincenzo Riccio (Workshop Co-organizers): DeepTest: 2nd International Workshop on Testing for Deep Learning and Deep Learning for Testing. July 1, 2020.
  Paolo Tonella: How to test a system based on deep learning. Invited talk at the CHOOSE Forum 2019 on Software Testing. Zurich, Switzerland, November 22, 2019.
  Paolo Tonella: What is a bug in deep learning applications? Invited talk at the 61st CREST Open Workshop (COW) on Statistics, Learning, Genetic Improvement and Testing for Programs. London, UK, October 21, 2019.
  Andrea Stocco: Black-box Confidence Estimation for Misbehavior Prediction in Autonomous Driving Systems. Invited talk at the 61st CREST Open Workshop (COW) on Statistics, Learning, Genetic Improvement and Testing for Programs. London, UK, October 21, 2019.
  Paolo Tonella: Can we trust software systems that learn? Invited talk at the Swiss Digital Day. Lugano, Switzerland, September 3, 2019.
  Paolo Tonella: Search Based Test Generation: from Programmed Systems to Systems that Learn. Invited talk at Politecnico di Torino, Italy, June 27, 2019.
  Paolo Tonella: The research challenges of the ERC project PRECRIME. Invited talk at Software Engineering for Machine Learning Applications (SEMLA). Polytechnique Montreal, Canada, June 12, 2018.
SOFTWARE

Software and replication artefacts


  DeepJanus: A Tool for Model-based Exploration of the Frontier of Behaviours for Deep Learning Systems Testing
  SelfOracle: Misbehaviour Prediction for Autonomous Driving Systems
  Taxonomy of Real Faults in Deep Learning Systems
VIDEOS

Conference presentations and invited talks


  Gunel Jahangirova: An Empirical Evaluation of Mutation Operators for Deep Learning Systems
  Andrea Stocco: Misbehaviour Prediction for Autonomous Driving Systems
  Nargiz Humbatova: Taxonomy of Real Faults in Deep Learning Systems
SUPPORTERS

The PRECRIME project is funded by the European Research Council, it is supported by Università della Svizzera italiana (USI), and it was granted 12-months of Amazon AWS cloud credits for research in 2019-2020. In Summer 2019 it was granted two summer student scholarships funded under the program UROP (Undergraduate Research Opportunities Program) of the Faculty of Informatics. The project is hosted inside the Software Institute's labs at USI.

CONTACTS

CONTACTS

We welcome visiting researchers!

Paolo Tonella, Università della Svizzera italiana (USI), Via G. Buffi 13, 6900 Lugano, Switzerland
Phone: +41 58 666 4848
Email: paolo.tonella@usi.ch