How do organizational procedures used in various system development life cycle processes need to be adapted when formal methods are introduced?

Main -> FAQ -> CIF-HM-2


 * Theme: Control Impact of Formalism (CIF)
 * Role: HM

Each company, and each sector has its own role and task distribution scheme. We therefore present here an agnostic vision of the task repartition, to be adapted to each company. The roles considered in this FAQ are the one defined in Roles.

High-Level Managers
The use of formal method dramatically shifts the workload of a project towards the analysis phase: much more work is required in the analysis phase, to develop formal models, and much less is required at the testing phase. See our dedicated FAQ QI-PQAM-2. As such, managers need to:
 * Adapt their personal workload metrics to accurately estimate the costs and delay for projects that use formal methods
 * Adapt their personal progress monitoring metrics to monitor the proper progress of projects that use formal methods
 * Adapt their go-go-go procedure to decide on the use (or not) of formal method in a project.

Estimating the cost factors in the context of formal methods requires some experience.

The factors involved in such go-no-go procedure are discussed in. Basically,these include:
 * *Some obligation to use formal method*, for instance, because it is requested by the customer (see FAQ about "External Factors Pushing for Formal Method Adoption")
 * *The development is internal, or on a shared-risk contractual vehicle* This is mainly to compensate for the possible low accuracy of cost and delay estimates
 * *The management is ready to face the cost shift* from late phase of the project to early phase. This can lead to an early red flagging of the project as being over schedule, and indirectly make it really of schedule due to increased reporting requests.

Project and QA Managers
Project managers and QA managers need to properly design the development process and select the most adequate formal method to ensure that:
 * *The claims that are proven by the formal method are relevant:* One can prove many things about a model; one should ensure that what is proven is useful to the project, and that all what should be proven is actually proven, or discharged to another validation method
 * *The artefact on which the claim is proven is the most adequate one:* One can prove different things on different artefact. For instance, on can prove some behavioural properties on abstract state machines or on the programmatic source code. It is much easier to prove such claims on abstract state machines. This is a trade-off between cost (the earliest defects are detected,the cheaper they are to correct, checking more abstract artefacts is simpler than more concrete ones) and level of assurance (the later the verification is performed, the fewer opportunities there are to introduce defects in the process)
 * *The abstractions made in the model that is actually verified are valid:* Models introduce abstraction; one should ensure that the established proofs remain valid in the real world although they might be done on an abstract model. This is discussed in the FAQ G-EA-1.
 * *The level of assurance that is delivered by the provers is adequate:* Not all formal method deliver the same level of assurance. This is discussed in details in EM-PQAM-3. One should select the most appropriate formal method according to the current context
 * *The formal method is compliant with the targeted standard* Some sectors require very high assurance, and might require proofs to be cross checked by a redundantly with a different prover, or to rely only on certified provers. Some provers of the Deploy project are certified for some CENELEC level of assurance. This is discussed in ExFac-HM-1.

Engineers and Analysts
Engineers need to:
 * *Develop formal artefacts:* This can be more-less time consuming depending on the considered artefact.
 * *Formally define the claims that need to be proven:* These can typically be derived from requirements documents. Some formal methods natively include these claims, for isntance if they are related to the proper use of programming language constructs like in software code verification tools à la Polyspace.
 * *Prove the claims on the artefact:* depending on the considered formal methods, this can be more-less time consuming, depending on the level of guidance that is required by the tooling, the run time of the tooling, and, possibly, the intertwined development process where the model is gradually proven as it is developed.
 * *Exploit the model in the next step of the development process:* When a formal model has been developed, and some claims proven on it, this artefact should be exploited in the development process. This can be translation process that is best carried by automated tools.

QA and Safety Engineers
QA and safety engineers need to assess that
 * The claims that are proven by the formal method are relevant: One can prove many things about a model; one should ensure that what is proven is useful to the project, and that all what should be proven is actually proven, or discharged to another validation method
 * The abstractions made in the model that is actually verified are valid: Models introduce abstraction; one should ensure that the established proofs remain valid in the real world although they might be done on an abstract model. This is discussed in the FAQ G-EA-1.
 * The formal methods have been used in an adequate way. For instance, one can prove any truth with a theorem prover if one includes an absurdity in the axioms.
 * The artefact on which claims have been proven is properly exploited in the downwards development process
 * The model is easy to prove. Some proof technology reach higher proof automation of performance if the model is developed according to some rules (avoid some type of constructs, avoid symmetries, etc.). Such verification can be performed by QA before the model is actually proven, to spare the time of engineers.

QA then amounts to checking that a formal models complies with the established guidelines. At Siemens transportation, the above tasks are under the responsibility of the safety engineers. QA are in charge of verifying that the developed models match some reference good practice that have been synthesized into a set of guidelines. This verification is performed before proofs are made. Some of these good practices are related to the three aforementioned bullets while some others are aimed at reaching a high level of auto-proving. This is how Siemens reported to be internally organized.