What are there risks of hiding the use of formal methods and what are the strategies to mitigate them?

Main -> FAQ -> CIF-PQAM-2


 * Theme: Control Impact of Formalism (CIF)
 * Role: PQAM

Answer
Experienced formal methods users report that reading and understanding formal specifications is not a significant problem [Sno01a]. With suitable training, programmers find formal specifications no more difficult to understand than programs. On the other hand the same users reported that creating formal specifications is very difficult. They said that the main difficulty lies in finding useful abstractions. This indicates that the problem lies not in specifying the detailed semantics but in the preliminary stage of choosing the objects and data structures that make up the model [Sno01b].

Hidden formal methods come with a set of risks that must be understood properly before choosing to use them or not.

Risks:

 * Need to switch to explicit mode for instance due to lack of automation or lack of expressiveness of the hiding layer. The associated penalty is that one will need to understand the mechanics bridging formal methods to the more informal input language. Furthermore, the hiding layer should be able to accommodate with user's intrusive action.
 * low efficiency of underneath model can arise because the user might not have the proper grasp on the assertions that are verified under the hood. For isntance, the user might introduce additional complexity in the model such as symmetry. As the user has no direct vision of the verified assertions, he might be unable to develop a model that is easy to verify.
 * Introduction of extra complexity due to the hiding the hiding implies that some easily understandable model must be translated into formally verifiable model. This translation process might introduce additional complexity in the verification. For isntance, consider a typed user language being translated into a non-typed formally analysable model, the typing information might give rise to parasitic predicates that will require additional computation.
 * Difficulty to understand results of formal analysis the formal analysis happening under the hood can deliver various results. these must be properly translated back to notions that are expressed in terms of the input model.
 * Erroneous estimate of risk or workload hiding formal methods might deliver interesting speed-up in system development. However, as it introduces extra risks, these methods introduce additional uncertainty in the development process.
 * Breach in qualified development process when one requires to rely solely on qualified development tool for critical system development, introducing an additional tool in the chain might introduce additional complexity, opportunity for errors, or non-qualified software tool in the development chain.

The main mitigation is to properly understand that although hidden FM can provide great support for formal method development, they are not the silver bullet of formal methods. They provide some assistance, but they will not enable a novice user to use formal methods without any form of training.

Mitigations:

 * Regarding risks related to tooling, the best mitigation is to properly analyse the tooling before selecting and deploying it.
 * Having a detailed view on what is happening under the hood: hidden formal methods are still useful because it will provide a more global view of the model, allowing the developers to focus on the content, rather than on the syntax and, if the hiding is properly implemented, it will increase the productivity, simply by comparing the size of the model to the size of the hidden formal model.
 * Need to understand the hidden formal model (bits of info from Steve Wright: you need to understand the machine to understand the C language you are typing)