Here's a cautionary tale about public perception of software safety...

[From Europe Intelligence Wire, Tuesday, February 15, 2000]

Western Daily Press/ MPs call for new Chinook inquiry

Europe Intelligence Wire via NewsEdge Corporation : MPs are demanding a new investigation into the 1994 Chinook helicopter crash after a report revealed an "unquantifiable risk" with the on-board software.

The National Audit Office (NAO) said 21 "anomalies" were found by Wiltshire-based experts in the computer software that governed fuel supply to the engines.

Shadow defence secretary Iain Duncan Smith and Liberal Democrat spokesman Menzies Campbell called for the Ministry of Defence to reopen its inquiry into the crash on the Mull of Kintyre, which killed 29 people.

The NAO report into the RAF's fleet of Chinook twin-rotor helicopters quoted Ministry of Defence experts as saying the safety-critical software was "not fit" for its purpose.

The report said the MoD team found the engine-control software to be "unverifiable". The Chinook was carrying some of Britain's top security experts when it crashed.

An RAF inquiry returned a verdict of "gross negligence" against the two dead pilots, Flight Lieutenants Jonathan Tapper and Rick Cook, who were accused of calculating a wrong rate of climb.

But their families have long pressed for a new inquiry, claiming it would prove the software was to blame for the wrong rate of climb.

Despite claims that the system software was unsafe, the Government has continually refused to reopen the inquiry because of a "lack of new evidence".

Mr Duncan Smith said: "The NAO report clearly highlights the need for an independent inquiry."


[From Ministry of Defence: Accepting Equipment Off-Contract and Into Service, UK National Audit Agency, HC 204, Parliamentary Session 1999-2000, February 11, 2000, ISBN 010 556659 4, pg. 36:]

Chinook MK 2 Mid-Life Update

1. In 1990, the Department placed a £143 million contract with Boeing Helicopters to upgrade 32 Chinook Mk1 helicopters to the Mk2 standard. The conversion involved the replacement of the transmission, hydraulic and electrical systems, various structural modifications and fitting converted modified Textron-Lycoming T-55 engines and a Full Authority Digital Electronic Control system (FADEC). The FADEC maintains the correct balance between the fuel flow, and therefore power output, of the Chinook's two engines which reduces pilot workload and makes the aircraft "easier" to fly. It is made up of a computer system which utilises inputs from various sensors to measure the amount of fuel that the helicopter's engines need, and a mechanical system which delivers the correct amount of fuel.

2. Because of the extent of the upgrade, the Mk2 was designated a new type within the United Kingdom fleet and the aircraft therefore had to receive United Kingdom military airworthiness certification - then called Controller Aircraft Release - before being returned to operational service. This involved the aircraft being put through a series of ground and flying trials costing some £1 million by the Defence Evaluation and Research Agency based at Boscombe Down.

3. As part of these trials, Boscombe Down wished to verify the software in the FADEC system using their preferred method known as Static Code Analysis. This element of the trials programme accounted for roughly ten per cent of the overall costs and a contract was placed on EDS-SCICON, an expert software engineering and testing company, for this purpose. In verifying the software, EDS-SCICON divided the anomalies which they found into four categories, with category 1 being the most significant. In their view, well developed software should contain none or very few category 1 and only a small number of category 2 anomalies. However, by the time EDS-SCICON had tested 18 per cent of the lines of code in the FADEC software they had already identified 21 category 1 and 154 category 2 anomalies in the software structure and documentation. Having discovered such a high incidence of anomalies at such an early stage, EDS-SCICON stopped testing because the way the software had been written made it unsuitable to full verification using the Static Code Analysis technique. The Department chose to terminate the EDS-SCICON contract at this point because the requirement for Static Code Analysis was an internal Boscombe Down policy, not supported by Defence Standards. The anomalies identified by EDS-SCICON were all reviewed by the equipment's Design Authority, who confirmed that whilst undesirable, none of them represented an airworthiness concern.

4. On the basis of EDS-SCICON's work, in October 1993, Boscombe Down advised the Department that they could not recommend Controller Aircraft Release for the Chinook Mk2 because of the "unquantifiable risk associated with the unverifiable nature of the FADEC software," and concluded that rewriting the software was essential.

5. In making their recommendation, Boscombe Down recognised that operational use of the Mk2 might be necessary before re-written software became available and provided a set of recommendations which were intended to allow flying if necessary but which would mitigate any consequent risk associated with their concern over potentially unpredictable FADEC software behaviour. In recognition of Boscombe Down's continued, but unsubstantiated concerns, the Department accepted their advice and restricted the loads carried by the Mk2 to ensure that the helicopter's all-up mass did not exceed 18,000 kilograms - the level at which the helicopter can fly normally with only one engine functioning - and a reduction of 26.5 per cent in load carrying capacity compared with the Staff Requirement. This precautionary limitation had a minimal impact as 90 per cent of all operations are carried out within the 18,000 kilograms limit. In March 1994 the restriction was relaxed to apply only to internal loads since underslung loads to an all-up mass of 22,700 kilograms (as specified in the Staff Requirement) could be jettisoned in an emergency.

6. As a result of the concerns raised about the software, the FADEC manufacturer has addressed some of the anomalies within the software, and produced an improved version which was then subjected to a Sneak and Traceability analysis to gain further confidence in its integrity. In September 1998 the Department, with the Royal Air Force's consent, issued full Controller Aircraft Release for the Chinook Mk2, removed the internal carriage all-up mass restriction for internal loads and allowed the Mk2 fleet to operate to the maximum payloads specified by Boeing which are 21 per cent higher than those specified in the Staff Requirement. In making the decision to issue full Controller Aircraft Release, the Department, as is their right as executive airworthiness authority for the aircraft, weighed the advice of Boscombe Down against other factors, including the equipment's Design Authorities and the consequences of failure, and concluded that the aircraft could be cleared for safe flight within the parameters identified. Notwithstanding the internal differences of opinion over FADEC, there were no significant impacts on acceptance into service and the Chinook Mid-Life Update was delivered on time, below budget and provided a significant increase in payload capability compared to the Staff Requirement.


[From  Europe Intelligence Wire,  Wednesday, June 21, 2000]

The Guardian/Inquiries into the Chinook crash were misled by the Ministry of Defence

Richard Norton-Taylor
richard.norton-taylor@guardian.co.uk

Europe Intelligence Wire via NewsEdge Corporation : A highly unusual meeting will take place this morning in Whitehall. Sir Malcolm Rifkind, a former Tory defence secretary, will urge Geoff Hoon, present holder of the post, to overturn a verdict on the cause of one of the RAF's most hotly disputed accidents.

It is extremely rare for a former cabinet minister publicly to challenge a decision taken during his own term of office. Rifkind's initiative reflects deeply held views shared by a growing number of MPs from all parties, and others with inside knowledge, about an issue which despite the MoD's best efforts will not go away.

On June 2 1994, an RAF Chinook helicopter carrying some of Britain's most experienced intelligence officers in Northern Ireland crashed into the Mull of Kintyre. All 29 on board died, including both pilots. A Scottish fatal accident inquiry concluded that it was impossible to pinpoint the blame. A subsequent RAF board of inquiry concluded that 'the most probable cause' was that the crew selected an 'inappropriate ROC [rate of climb] to safely overfly the Mull. However, it also said there were 'many potential causes of the accident' and it was 'unable to determine a definite cause'.

Two air marshals - Sir William Wratten and Sir John Day, the senior officer responsible for RAF Chinooks - took it upon themselves to overrule the inquiries' verdicts. They accused the two pilots - Flight Lieutenants Jonathan Tapper and Richard Cook, sufficiently skilled to have been cleared for special forces operations - of 'gross negligence' by breaking flying rules in bad weather.

Such a verdict requires an extremely high burden of proof, higher than that used in criminal courts for murder. RAF rules in force at the time, but since abandoned, stated that 'only in cases in which there is absolutely no doubt whatsoever should deceased aircrew be found negligent'.

In a newspaper last weekend, Wratten justified his verdict by insisting that the pilots had failed to exercise skill and judgment by flying into bad weather too low and too fast. Yet he also conceded that since the Chinook was not equipped with a black box or cockpit voice recorder 'there is inevitably a degree of speculation as to the precise detail of the events prior to impact'. Why the pilots 'elected to ignore the safe options open to them we shall never know'. This hardly meets the demanding test laid down by the RAF rules which then existed.

A mountain of evidence has emerged since the crash which makes a mockery of the verdict or, as Rifkind, not known for overstatement, put it yesterday, points to 'massive uncertainty'. Documents obtained by Computer Weekly magazine and seen by the Guardian show that the MoD has repeatedly misled both the official inquiries and the Commons.

Much of the evidence concerns the Chinook's software system called Fadec, short for Full Authority Digital Electronic Control. Last week Scotland's senior law officer, the lord advocate, Colin Boyd QC, said that new information about faults in the Chinook's Fadec software 'may be thought to reinforce' the inconclusive findings of the fatal accident inquiry. He conceded that it 'invites speculation as to another possible cause for the crash'.

In 1997, the MoD told Menzies Campbell, the Liberal Democrat defence spokesman, that the RAF inquiry 'found no evidence of structural or technical malfunction'. What the board of inquiry actually concluded was that 'distraction by a technical malfunction could have been a contributory factor in the accident'. The MoD's own air accident investigation branch reinforced the point, telling the inquiry that the Chinook's 'pre-impact serviceability could not be positively verified' - in other words, there was no forensic evidence.

The inquiry was also told that shortly before the Mull crash, Chinook pilots were confronted with 'unforeseen malfunctions of a flight critical nature [which] have mainly been associated with the engine control system Fadec. They have resulted in undemanded engine shutdown, engine run up .'

The MoD subsequently told the Commons defence committee that Boeing, manufacturer of the Chinook, 'did not consider the Fadec to be flight safety critical'. Yet in evidence in the US, where it was suing the manufacturer of the Fadec system, Textron Lycoming, in relation to an earlier accident, the MoD stated: 'The software was truly criticial in maintaining safe flight'.

Defence ministers told MPs in 1998 that the MoD sued the US company because of negligence in 'testing procedures' and not, they insisted, because of 'a failure of the software'. Yet three years earlier, the MoD told the US authorities that the accident at issue - involving an MoD Chinook in 1989 - was caused by an engine overrun 'to an unprecedented and violent overspeed' during testing. It blamed the problem on the Fadec system's 'faulty design'.

Neither the Scottish fatal accident inquiry nor the RAF board of inquiry were told that at the time of the Mull of Kintyre crash that the MoD was suing Textron Lycoming over failings in the Fadec system.

The Aeroplane and Armament Research Establishment at Boscombe Down, the MoD's own airworthiness assessors, grounded non-operational Chinooks a day before the crash on the Mull because of concerns about the Fadec system. Four days after the crash, on June 6 1994, Wratten drafted an angry memo to one of his senior colleagues complaining about Boscombe Down's decision to stop Chinook flying trials for the second time that year.

After making it clear that Boscombe's decision would be overruled, Wratten added that Boscombe's attitude 'does nothing to engender aircrew confidence in the aircraft'.

Earlier this year a report by fellows of the Royal Aeronautical Society concluded that the verdict of pilot negligence was not sustainable in the light of evidence pointing to other problems which could have caused the crash on the Mull. Wratten and Day were so infuriated by the report that they resigned from the society.

The misleading goes on. Last week John Spellar, the armed forces minister, told the Liberal Democrat MP, Thomas Brake, that the Chinook Mark 2 - of the kind that crashed on the Mull - had a 'complete set of flight reference cards in June 1994'. These, he said, contained all the normal and emergency operating drills in force at the time, including what to do in the event of a possible Fadec malfunction.

His answer directly contradicts evidence given by RAF officers to the inquiries. The RAF inquiry was told that the Chinook flight reference cards were based on earlier models of the helicopter which were not fitted with Fadec, according to a transcript seen by the Guardian. A Chinook pilot told the Scottish fatal accident inquiry that the cards in force in June 1994 'were actually held to be so incorrect as to be contributory to the number of engine malfunctions being observed'.

Against this background, it is scarcely surprising that Rifkind will today ask Hoon either to set aside the verdict on the Mull crash or set up an independent inquiry. 'If the government is not careful, this is going to be a cause celebre,' Menzies Campbell said yesterday.


[From Financial Times, London Ed1, November 30, 2000 pg. 4]

NATIONAL NEWS/ MPs seek to overturn inquiry findings
CHINOOK CRASH REPORT SAYS THERE ARE 'CLEAR GROUNDS FOR DOUBT' OVER RULING THAT PILOT ERROR

The government should overturn an official verdict that pilot error was to blame for the 1994 Chinook helicopter crash that killed 29 people, including some of Northern Ireland's top intelligence experts, the Commons public accounts select committee says today.

The committee says that the finding of a Royal Air Force board inquiry that pilot error was to blame for the crash did not satisfy the burden of proof required in such inquiries.

Negligence on behalf of the pilots should have been the verdict if it had been positively identified to have been the cause of the crash, the MPs say. But according to the report, there were "clear grounds for doubt in a number of areas".

Among these was the fact that there was no conclusive technical data recovered from the wreckage, ruling out a technical malfunction as a potential cause of the crash.

The committee goes on to draw attention to a number of problems experienced during the acceptance process of the Chinook Mark 2, which it says are relevant to the crash of the Chinook helicopter ZD-576 of the same type on the Mull of Kintyre in Scotland. It describes the process as "flawed".

It notes for example that in the lead-up to the crash, the Chinook Mk 2 was experiencing repeated and unexplained technical difficulties caused by the software of its full authority digital electronics control system.

The committee said that it was "unconvinced" by arguments put forward by the Ministry of Defence suggesting that the software had been adequately tested to demonstrate its safety.

It says it cannot understand how, given the absence of an accident data recorder on the helicopter, both the board of inquiry and the MoD could have had "no doubt whatsoever" that crew negligence caused the crash.

David Davis, PAC chairman, said that although the truth as to what really happened on that fateful day in June 1994 may never be known, the evidence pointed clearly to a "miscarriage of justice".

He added that the time had finally come to challenge the "unwarrantable arrogance" of the MoD, which has consistently backed the findings of the board of inquiry.

The MoD, however, is continuing to insist that there is no new evidence that casts doubt on the "integrity" of the RAF finding. MPs have warned that if the MoD persists on this track, they may consider appealing to Tony Blair, the prime minister, to become more directly involved in the issue.


ComputerWeekly also ran a series of articles called "RAF Justice" with a (very overblown) discussion of the Chinook FADEC.