Archive Article

EDS-SCICON DEFENCE LTD
FADEC T55 SOFTWARE VERIFICATION
FINAL REPORT
ISSUE 1.0
DOC REF C0179E/R-01/0145
JULY 1993

Author: C Gough
Reviewers: T Jennings and P Keenan
Approved for issue: P Wordsworth

CONTENTS
1. INTRODUCTION
2. PROGRESS TO DATE
3. OVERVIEW OF THE FINDINGS
3.1 NUMBER OF ANOMALIES FOUND
3.2 GENERAL TYPES OF ANOMOLIES
4. DISCUSSIONOF THE ANOMOLIES
4.1 TRACEABILITY AND ANALYSIS OF CODE
4.2 HSDE RESPONSES

1.INTRODUCTION

This final report has been prepared for ADC/HI (Mr B J Barton) in response to his letter H12A/205 dated 1 July 1993, to provide a view on the status of the Verification of the T55 Software at the point of project termination.This work was being conducted by EDS-Scicon under contract H12A/205.

The report contains the following further sections:
Section 2 A statement on progress to date
Section 3 An overview of the findings to date
Section 4 A discussion of the significance of the anomalies identified to date taking into consideration HSDE responses.

2.PROGRESS TO DATE

Full details of all the stages involved in the verification process are documented in EDS-Scicon's "Proposal To MOD (PE) H/ENG 2 For Verification of FADEC Software and Documentation" (Reference: B28536.00.102 Issue 1.0 dated November 1991).For the purposes of this report, the verification process can be viewed as comprising 4 basic stages:
i. Customise the translators and other tools required to convert the code into a format compatible with the proprietary analysis tool SPADE.

ii. Examine the T55 documentation and ensure that the requirements can be fully traced through each level of the documentation.

iii. Perform Static Analysis including semantic analysis on the code using the SPADE tool.

iv. Produce a Final Report documenting the results of the verification.

All the work on stages i and ii has been completed, and the input required for the Final Report (stage iv) on each module is produced as part of the Static Analysis process.

The major part of the verification process relates to the Static Analysis of the code; this can be divided into the following steps for each module:
i. perform analysis,

ii. perform local review to ensure that the analyst has correctly identified all the anomalies with in the module,

iii. perform local review to ensure that the module contains the correct links to other modules with which it has an interface,

iv. perform rework on any areas identified by either of the reviews, and

v. perform final review to ensure that all the rework has been done correctly and that the documentation of the results are to a consistent standard.

The simplest way of assessing progress is to record the number of modules which have completed each of the above steps.

However, the Static Analysis has to be completed on the lowest level modules first and then on the modules which call the lowest level modules and so on up the "call graph" of the code.The early modules tend to be the smallest and simplest, and the last modules tend to be the largest and most complicated.The average effort required to analyse a module will therefore increase as work progresses.Hence an alternative way to review progress is to compare number of lines of code in the modules that have completed each of the steps identified above, to the total number of lines of code.

The code for the T55 is divided into two Lanes, the Primary and the Reversionary.

Table T1 shows the progress to date on the Static Analysis stage of the process.This is measured by the number of modules which have completed each of the steps identified above, and by the percentage of the total number of modules which have completed each stage.(Annex A provides a complete list of all the modules/subprograms by name and provides details of their current status).

Table T1 - Progress To Date Measured By Number Of Modules

Primary Lane Reversionary Lane Both Lanes
StatusNo.%No.%No.%
Not yet started7240.7%199140.2%
Waiting for Local Review147.9%0146.2%
Waiting for Global Review52.8%1262.6%
Waiting for Rework95.1% 1104.4%
Waiting Third Review21.1%131.3%
Modules Delivered7542.4%2710245.3%
Totals 177 49 226

Alternatively, Table T2 provides details of the total number of lines of code in the modules which have been delivered and the percentage of the overall total number of lines this represents.A line of code is considered to be a source line which is not a comment and is not a blank line.

Table T2- Progress To Date Measured By Lines Of Code

Total number of lines of code Total number of lines in modules which have been delivered Percentage of code completed
Primary Lane12349215117.4%
Reversionary Lane391574519%
Both Lanes16264289717.8%

In summary, it can be seen from Tables T1 and T2 that 102 modules (45.2%) have been analysed, revised and delivered representing 17.8% of the total number of lines of code.

3.OVERVIEW OF THE FINDINGS

3.1 NUMBER OF ANOMALIES

Whenever an anomaly is found during any part of the verification process, it is classified into one of four categories depending on its severity and its potential impact on either code.The four categories are described in Section 4 of this report.

Table T3 provides details of the total number of anomalies in each category that have been found.It should be noted that these figures represent EDS-Scicon's initial view of the category of each anomaly which may change as a result of further investigation, or as a result of replies from HSDE.

Table T3- Total Number Of Anomalies In Each Category

Cat 1Cat 2Cat 3Cat 4
Primary Lane131114281
Reversionary Lane8431520
Documentation Traceablilty3539753
TOTAL56193132104485

3.2GENERAL TYPES OF ANOMALIES

A large number of different types of anomalies have been found.Whilst many show no common theme, there have been some general types on anomalies which appear in a variety of similar guises.The 8 general types of anomaly which have been seen most often are listed below.Under each general type are examples of some of the specific anomalies which form part of the general type.The severity and hence the category assigned to a type of anomaly depends on exactly where it appears in the code, but as an indication of the seriousness the different types of anomaly, the typical category assigned to an occurrence of a type of anomaly is given in brackets at the end of the description.

1.FAILURE TO TRACE CODE TO DOCUMENTATION

a. Algorithms that could not be traced to the documentation (Category 2).
b. Individual actions on paths that could not be traced to the documentation (Category 1 or 2).
c. Paths that could not be traced to the documentation (Category 2).
d. Interfaces to hardware that do not appear to be documented (Category 2).
e. Undocumented values for booleans, constants (Category 2 or 3).
f. Overflow of variables (Category 1 or 2).
g. Undocumented use of persistent local variables (Category 2).
h. Parameters which could not be traced to the documentation (Category 2).
i. Extra code not required in documentation (Category 2).
j. No documentation at all (Category 2).
k. The variables in the code do not trace (without considerable analysis) to the variables mentioned in the documentation (Category 2).
l. A requirement in the documentation which does not appear in the code (Category 1)

2.INCORRECT CODE COMMENTS

a. Incorrect code comments (Category 3).
b. Incorrect specification of a parameter (Category 2).

3.REDUNDANT CODE

a. Unreferenced labels (Category 4).
b. Statements which do not perform any documented/required function (Category 2).
c. Addition of an opcode to test a flag that was set correctly by the previous instruction (Category 2).
d. entire subprograms that are not used in the FADEC (Category 1).
e. Redundant data declarations (Category 2).
f. Arrays with unusual elements (Category 2).
g. Literals declared and never used (Category 2).

4.ALIASING

a. Addressing the same location using two different literal names (Category 2).
b. Jumping to an address which may or may not have an associated label (Category 2).
c. More than one label associated with an instruction address (Category 2).
d. Multiple names for a single variable or constant (category 2).
e. Aliasing of a word variable name to the element of an array (Category 2).
f. Aliasing of data variable by allowing the index of one array to be incremented beyond the upper bound of that array into the subsequent variables in memory (Category 2).
g. Using mnemonics and literals to represent the same number (Category 2).
h. Locating differently named segments to the same physical location and using these names arbitrarily in the code to address date (Category 2).

5.UNINSTRUCTED CODE

a. Uninstructed code due to the handling of RAM failure conditions (Category 4).
b. Uninstructed code due to a common block of code being used on both paths of an IF_THEN_ELSE statement (Category 2 or 4).
c. Uninstructed code due to the nature of CLI implementation (Category 4).

6.MISMATCH OF DATA TYPES

a. Indexing simple word variables (Category 2).
b. Applying a byte operation to a word variable and so leaving the upper byte undefined, rather than clearing it (Category 2).
c. Array used as a single word (Category 2).
d. Using a second element of an un-initialised array giving a rounding error (Category 1).

7.INCORRECT CODE

a. Overflow not handled as documented (Category 1).

8.DOCUMENT ANOMALIES

a. Incorrect documentation
b. Ambiguous documentation
c. Typographical errors

4. DISCUSSION OF THE ANOMALIES

Each anomaly is categorised by EDS-Scicon into one of the following:

CATEGORY 1 - EDS-Scicon has a high level of confidence that the anomaly relates to a real error in the code or a discrepancy between the code and the documentation.

CATEGORY 2 - the anomaly relates to poor code or poor correspondence between code and documentation but it is likely that it performs the intended function.

CATEGORY 3 - these anomalies relate purely to obvious documentation errors such as typographical errors and incorrect commenting of code.They have no direct implication for the correct operation.

CATEGORY 4 - these anomalies arise from the "style" of coding, or the way in which the code has been modelled for analysis, they do not relate to any source language use that will cause incorrect operation

In a rigorously developed safety critical system which has been written according to a strict coding standard, the code can be expected to contain none, or very few category 1 anomalies and only a small number (in the order of ten's) category 2 anomalies.

The other types of anomaly are likely to be more common, although in a carefully developed system there ought not to be too many category 3 anomalies.

A category 1 anomaly indicates an actual coding error or non-compliance with the documentation.However this does not mean that it necessarily has any safety implication, it may not even affect the overall operation of the system.Each category 1 anomaly has to be investigated (in this case ultimately by HSDE) to determine its impact on the system behaviour.

When a response from HSDE indicates that an error does not have any safety impact or does not significantly affect the operation of the system, it should be emphasised that latent errors in the code may still have serious implications for future maintenance of the system.

Likewise, category 2 anomalies need to be checked to ensure that the intended function is performed.Category 2 anomalies may not have any immediate impact on safety but they represent poor coding practice or poor correspondence between code and documentation and as such, may have serious implications for any further maintenance.

4.1TRACEABILITY AND ANALYSIS OF CODE

The other major impact of poorly developed code, indicated by a preponderance of category 1 and category 2 anomalies is the difficulty in tracing and analysing such code.

Poorly developed code is significantly more difficult to analyse than code written and documented to a well defined standard.Each category 1 and 2 anomaly has to be carefully traced and analysed to determine its impact on the system.This has been the major factor in the problems in analysing the T55 FADE code.In addition each category 4 caused the analyst a significant amount of extra work, in establishing that it did not "mask" a more serious anomaly.

Another effect of a large number of category 1 and 2 anomalies is to diminish their impact.On well written and documented code with few significant anomalies, each category 1 or 2 anomaly is viewed seriously and causes some concern.When there are hundreds of such anomalies the threshold of what is acceptable tends to be diminished by the presence of so many anomalies due to poor programming style and inadequate documentation.This has to be guarded against very carefully.

4.2HSDE RESPONSES

In general the responses to queries regarding category 1 and category 2 anomalies from HSDE has been adequate although their answers, invariably imply no need for any modifications to the code.Where EDS-Scicon has had concerns as to whether the anomaly has safety critical implications, HSDE has provided more thorough responses.

There were a number o cases where EDS-Scicon did not feel that HSDE's response adequately covered the query or that any action suggested (or lack of any action) to remedy the situation was not sufficient.

The most serious of these is the anomaly in the Reversionary Lane concerning use of a byte address rather than a word address.HSDE has given a response to this query and has assured EDS-Scicon that the code behaves correctly.However, the correct operation depends on an undocumented feature of the Intel ASM96 microcomputer.

EDS-Scicon's concern is that a change in the mask or process of the ASM96 chip at some point in the future may cause incorrect operation of the FADEC.

Table T4 provides details of the responses that have been received from HSDE in reply to formally reported anomalies.In addition to these replies, EDS-Scicon has also received replies to separate urgent request for help on anomalies which might have safety implications.When such anomalies have been encountered, HSDE has been informed immediately and requested for assistance as soon as the anomaly is identified, rather than waiting until the analysis is completed, and the anomaly could be formally reported.

Table T4 - HSDE Responses

Number of Replies Received From HSDE Percentage of Total Number Of Anomalies
Primary Lane208.4%
Reversionary Lane00%
Document traceability11575.6%
TOTAL13528.4%