What is the cost or effort needed to train engineers/analysts to use a new formalism?

taking into account their previous experience with formal engineering methods

Main -> FAQ -> TSP-HM-1


 * Theme: Training Scope and Pace (TSP)
 * Role: High-Level Manager (HM)

Influencing Factors
Several factor can influence the necessary amount of training to formal method that is needed in order to become a productive developer of formal models. These include:
 * Previous experience with formal methods Having a previous experience with other formal methods is beneficial, as they all exhibit some notion of declarativeness and mathematical notation.
 * Previous experience with classical development process It seems that switching from a routine where one reasons about he procedural behaviour of software artefacts to the more declarative concepts used in formal methods is a challenge[siemens interview et XMOS interview]. It turns harder with the amount of experience of the engineer with procedural thinking because the trainee tends to identify the concepts used in formal methods to the ones used in software developments although they are semantically different.
 * The considered formal method Some formal methods are inherently easier to learn than others. It is not only the mathematical notation that must be considered, but the general semantics of formal models, including the level of declarativeness, and specific notions such as refinements, which might take various shapes. It seems that the mathematical notation might not be the biggest training issue for computer scientist, and they are used to programming languages, which are a form of mathematical notation. It is also possible to hide the mathematical notation behind a high-level language or a domain-specific, to help formal method adoption by developers . For instance the B method can be made easier to use thanks to the UML-B tool support, which hides part of the B formalism behind well-known UML notation . In the deploy project, SAP has examined how the Event-B method could be used to validate some form business processes, and therefore could partially hide the Event-B notation behind a graphical interface supporting business-process notation. We only consider here non-hidden formal methods.
 * The level of automation of the formal tool The level of automation of the underlying formal analysis algorithms is a factor that influences the training cost [Ar08]. Some formal methods rely of proofs which have to be constructed by the developers. There is an additional training required for proof development. Some other formal methods, although being fully automated, might require the developer to learn on the underlying mechanics in order to fine tune its model for better verification efficiency by e.g. avoiding models exhibiting many symmetries.
 * The resilience of users to the adoption of formal methods at all Some user are simply not interested by formal methods. Very little can be done to address this issue. Formal methods can also be presented to developers as advanced debugging mechanism [Ar08].

Organisation of the training
It has been assessed in the DEPLOY project that initial training is crucial. One of the major problems are not related with the development of tutorial examples, but to the scaling to other models, and to the identification of the proper modelling style which has to be tuned for each application sector.

The process was globally the same in all deployment and is worth being detailed:
 * At the beginning of the DEPLOY project, a 3-day block course has been organized in Zurich for all the industrial partners. The course was judged as useful to learn the basics of the notations and have a first hands-on with the tools, however, it turned out to be largely insufficient.
 * It was followed by what we called mini-pilots, i.e. a number of modelling exercices still of limited scope but already anchored in sector specific problems. Those were useful to transition to the modelling of larger pilot problem.
 * During the pilot phase, industrial partners were coached by experts (the academic partners). The initial people involved were generally a rather small team of in-house highly qualified people who already had a PhD in FM or related field, or had similar industrial experience. We believe that this structure enabled a long-run training to modelling style, and somehow compensated for the lack of extensive training.
 * At the end, internal experts emerged from this training and ensured the internal training of a "second generation" of people, some of them having less need to have a strong background in FM, especially when a more "hidden FM" strategy was followed.

Some training records

 * Siemens trained a graduate in the context of an internship on Event-B. The trainee was already trained about formal methods in general, but not about the Event-B method. She was productive within one week, but was coached during longer time by experts in formal methods at STS. After her internship, she was signed up by STS.
 * Space System Finland, an industrial partner of the Deploy project trained a pool of engineer to the Event B method. They assessed the following :
 * The participants of this study spent 2-4 months on the Event-B training
 * The work effort varied from 25% to 50% depending on the other projects
 * About one man-month of partly tutored study was needed to gain ability for independent modelling work
 * The AES Group is a Brazilian company which has been developing railway sub-systems since 1998. In the context of the Deploy Associate program, 12 people (out of the 30 people of the company) successfully followed a one week training session during spring 2010. This training was organised on-site with the help of the University of Southampton who designed tailored training material. A dead-man case study was used as support. The model now has been passed over to an employee of AESGroup, to pursue its development. This employee, with no previous background in formal methods, was able to pursue the development, base only on the one week lecture given internally. See the AES book chapter of the Deploy Book for more details.
 * Critical Software is active in the development of mission critical and software critical systems for space, telecom or energy applications. The UK branch of the company joined the Deploy Associate program to initiate an internal project involving Event-B. The team of people involved in this project includes three peoples with siz, seven, and two years of experience, respectively, with software and system engineering background. They assessed that the switch to declarative thinking is not easy to do, especially for people having an extensive experience in procedural thinking. People having a lot of experience tend to include too many details that are implementation-specific. For instance, they erroneously tried to identify events to procedure or functions. The mathematical notation in itself is not an issue, as it is only an additional programming language. The issue actually comes from the declarative structure and nature of the model. See the AES book chapter of the Deploy Book for more details.
 * Siemens also reported that training new graduates to the B formal methods is much easier than training engineers with several years of experience who are highly accustomed to the traditional development process.

Surprisingly, it has been reported that training to formal methods is easier if one has less experience in classical development process. The explanation is that those people then to think in terms of their usual procedural coding habits rather than shifting to more abstract and declarative specification.

Reducing the need of training by hidding FM
Most of the effort of transferring a formal method to new staff can be avoided when hiding the formal method behind tools already used by the targeted users. At SAP, the formal aspects are hidden behind the tool already in use by the target team. Users don't really need to know that a formal method is used in the background. Consequently, the effort for training can be saved. The drawback is there is a need to hide the formal methods completely so that no guidance is required from the user or no error related to the formal method is reported at the error level. The effort to hide the formal method can be quite significant. For example, for SAP it is required to achieve high level of proof automation for the checks and to produce user level explanations from the output of the formal tools used. However in the long term if a significant amount of staff members use the tool, the additional effort of hiding the formal method is largely compensated by savings on training. (See also the FAQ on hidding FM)

Hiring people prior experience in FM
XMOS enrolled a PhD who made his thesis on Event-B. At the beginning of his PhD, he had spent one full year in learning formal methods. He did not had any guidance or training, and had no previous background on formal methods. He had an industrial background in imperative programming, embedded systems, and digital system development (VHDL). The major difficulty was to grasp new concepts such as abstraction, which is quite different in the Event-B formal methods from what it is in classical programming, and declarativeness, which is virtually absent from the embedded programming world. The mathematical notation was not an issue. During the Deploy Associate program, little help was required or only for very specific and advanced issues. The achievement of this work are quite impressive and detailed in Deploying Event-B in an Industrial Microprocessor Development. The level of maturity achieved even allowed the production of a model to estimate the effort required to build a system in that specific domain.