Wednesday, May 5, 2010

Low Level Aerobatic Peer Reviews

Just been reading some articles at Avweb.
The first, "Lucky or Good"
http://www.avweb.com/news/probablecause/probable_cause_62_lucky_or_good_198053-1.html
"There's something about the typical experienced-pilot's personality that is antithetical to safety. I'm not an expert in analyzing personalities -- though I know what I like -- but it seems the very traits that make someone a "good stick" also make that same skilled pilot a safety risk."
and the other, "We Worry About the Wrong Things and It's Killing Us"
http://www.avweb.com/news/pilotlounge/pilots_lounge_122_we_worry_about_the_wrong_things_196933-1.html
"My friend just can't figure out why we Americans so blithely accept the true risks we face while continuing to smoke, over-eat, not wear seatbelts and not raise heck about hospital procedures, yet we get ourselves all in a twitter over the low risk items and take all sorts of expensive and often-redundant precautions that would be better spent on the high-risk stuff.
I didn't have an answer for my friend, but it caused me to look at the same question as applied to flying. While the fatality numbers for general aviation, just under 500 in 2006 -- far less than the number who die each year falling in bathtubs -- are very low, they have to be compared with the very small number of people who get into general aviation airplanes in the first place. With that in mind, our accident rate is far worse than the airlines and somewhat worse than automobiles, about the same as for motorcycles. Therefore, it's worth evaluating: Do we pilots worry about how to deal with the actual risks we face? As an aside, do we flight instructors teach students (and pilots in for recurrent training) how to identify and avoid doing stupid, high-risk things in airplanes?"
I was reminded of a discussion paper that I wrote for one of the aerobatic clubs about a year ago. There was some discussion. Some said that peer reviews are not needed as "we already tell people". Nope, you might tell some-one if they experience a sudden loss of judgement and does something silly but then they already know it themselves.
Some wanted peer reviews as a condition of permanent low level permissions. Nope, then we'll just have people chasing to get the peer review stamps in their book just before contest registration - if that, why would a contest registrar be looking for it? No-one else will be chasing it up so they won't happen.
If people don't do peer reviews for the right reasons then they won't be effective.
That discussion paper has gone nowhere so I thought that I'd share it with you here. I have seen that a few aerobatic pilots are practicing it. At one contest recently, peer reviews were apparently mentioned at the briefing and offered the assistance of a stamp. Unfortunately not many pilots know what a peer review actually was.

When CASA issued CAAP 155-1, Aerobatics, in January 2007 they introduced the concept of peer reviews which has largely been ignored by the aerobatic community since then. We know that CASA has been reconsidering their policy on low level aerobatics so should the club take the initiative to improve safety for aerobatic pilots before CASA imposes any restrictions?
If you consider the details of any accidents, not just those involving aerobatics, do you think that an effective regime of peer reviews would have eliminated any of them? Or even just one – would that make it worthwhile for all to undertake peer reviews?
How safe is a novice safety pilot in a competition? What do they know of their responsibilities and required capability?
Many organisations offer Pilot Proficiency Programs for their members – should the aerobatic club undertake a similar program for the benefit of its members?
This discussion paper is prepared for use by the aerobatic club in consideration of the recent history of a large number of tragic accidents involving low level aerobatics and the potential for CASA to revise their policy with a possible adverse effect on the operations of club members.
BACKGROUND
Some years ago, CASA developed two new draft regulations, Parts 61 and 91, which included some new rules relating to aerobatics.
PART 61 FLIGHT CREW LICENSING Subpart P - Flight Activity & Maintenance Authorisations
"1.1.2 The following flight activity authorisations are specified in this Subpart -
    d) Aerobatics
    e) Aerobatics below 3000 ft AGL
    f) Aerobatics below 1500 ft AGL"
i.e. rules relating to aerobatics were to be split into three different categories as above. Basic aerobatic endorsements would be above 3000 ft as now. Aerobatics down to 1500 ft would be administered by industry i.e. appropriate instructors and others (via the aerobatic club) would have the authority to make logbook entries permitting pilots to do aerobatics down to 1500 ft. Aerobatics below 1500 ft would continue to be administered by CASA.
There was a draft Advisory Circular which had been developed by CASA in broad consultation with industry.
PART 91 GENERAL OPERATING AND FLIGHT RULES
91.075 Aerobatic Flight
"(3) The pilot in command of an aircraft must not perform aerobatic manoeuvres in the following circum stances without CASA's approval:
    (a) below 1500 feet AGL;
    (b) in the vicinity of a public gathering or at an air display;
    (c) at night."
To enable the provisions of Part 61, this rule in Part 91 above had to be altered for it to read 1500 ft instead of 3000 ft.
The aerobatic club objected to this change – they wanted to retain 3000 ft as the minimum height for aerobatics thus requiring a specific delegation for CASA for any aerobatics below 3000 ft. So, as CASA staff had changed, being unware of the broad industry consultation they took the aerobatic club's view and put 3000 ft into the draft regulation. I learnt a lesson there – even though many of us had contributed to the draft AC and rule development none of us respodned to the NPRM – we need to have responded and said it was good. CASA only got negative comments.
The draft AC 91.075(0) has been available for review in its current form since September, 2001. Appendix 1, Para 3.3 is quite clear on the subject of initial approvals for aerobatics:
"Pilots may be certified as safe to conduct all the primary aerobatic manoeuvres above a minimum height of 1500 feet, but must not be cleared for aerobatic manoeuvres below their individual spin recovery certification. A minimum height of 3000 feet is recommended for most initial approvals."
Para 2.3 also refers to limitations for inexperienced pilots. The AC also outlines requirements for aerobatic flight instructors, training and the provision of an Operations Manual which would detail the conduct of aerobatic training. This is the mechanism by which CASA would control the process for training and issue of approvals for aerobatics down to 3000 ft; and down to 1500 ft.
I went to the CASA FLOT2003 conference in Sydney specifically to have that draft changed back to the original 1500 ft and succeeded in that. But, who would've thought that the aerobatic club would oppose the new rules which were aimed at facilitating our sport. The aerobatic club purported to represent its members yet did not tell its members what it intended to do – these days it is so easy to communicate with members with email or online forums!
Six years later the regulatory reform programme (not just aerobatics) has gone nowhere so I wouldn't rely on any changes to the rules in my lifetime. i.e. we must work within the existing CAR 155 and, probably, the existing CAO 40.0.
OTHER COUNTRIES
UK
Aerobatics in the UK is not regulated in that there is no minimum height specified for aerobatics and aerobatic training is not required – there is no aerobatic endorsement (it is likely to be required soon with the new EASA rules).
http://www.caa.co.uk/docs/33/cap393.pdf page 291
The AOPA has an aerobatic training syllabus and a certificate which is generally accepted as the standard.
http://www.aopa.co.uk/scripts/course_a.php
The British Aerobatic Association and the Tiger club have standards to be demonstrated prior to their members performing at low level at their events.
Display pilots require a CAA authorisation http://www.caa.co.uk/docs/33/cap403.pdf
USA
The USA does not require aerobatic training and has no aerobatic endorsement.
http://www.airweb.faa.gov/Regulatory_and_Guidance_Library/rgAdvisoryCircular.nsf/0/74a345ccf8eb48a6862569d900744be1/$FILE/ATTNUI98/ac91-48.pdf
Minimum height for aerobatics is 1500 ft however the IAC has gained a waiver for its members whereby they may practice and compete to competition levels without any further approval subject to that flying being within an area approved by the FAA for low level aerobatics.
http://www.iac.org/
Display flying is largely administered by ICAS with annual renewals of low level waivers which are issed by the FAA. ICAS has a detailed manual.
http://www.airshows.aero/
Canada
From some brief research, Canada has a tighter regulatory regime than Australia.
CASA POLICY - THE CAAP
Back in the '70s low level concessions were renewed annually – testing was by CASA (more correctly CASA's predecessor) or aerobatic club members approved by CASA. CASA also approved aerobatic club members to conduct low level aerobatic coaching.
Flying Operations Instruction No 13-2 Issue 2 about '93 brought some significant changes:
A low level approval will remain current while the pilot holds a valid licence.
AAC members may be approved to conduct aerobatics down to 330ft.
The AAC may nominate members for approval for the testing of AAC members for the issue of low level approvals. So far so good, but to gain approval from the CAA as a low level tester the pilot must undergo a flight test with an examiner of airmen. This flight test must be repeated annually except that it may be conducted biannually in the case of a pilot who has competed at Unlimited level at our Nationals or who has competed in a World aerobatic contest during the preceding year.

Appendix 1 contained the knowledge requirements for low level aerobatics which largely remain today.
The came another policy with durations of two years for low level aerobatic approvals and limitations on aircraft power.
CAAP 155-1 was developed with broad industry involvement and introduced the provision for CASA delegates to issue low level aerobatic permissions. It was issued in January 2007.
http://www.casa.gov.au/download/CAAPs/ops/155_1.pdf
It contained some excellent information, some of which was based on USA AC's but most was developed locally.
It introduced new guidance for threat and error management.
It introduced the recommendation for annual peer reviews.
It introduced new guidelines for low level aerobatic permissions:
It included a low level aerobatic test form with additional guidance on the conduct of the test.
It contained guidance on conditions of low level aerobatic permissions:
“6.15 Conditions On Permissions
6.15.1 A delegate cannot cancel a permission once it is issued. Therefore a delegate needs to carefully consider whether there is a need to issue the permission with conditions. Delegates may issue a low-level aerobatics permission with any conditions they believe necessary in the interests of safety. For the guidance of delegates, it is considered that the following conditions are applicable to all low level permissions to provide an acceptable level of safety:
(1) Other than one-off permissions, the permission can be issued for an indefinite period, but a delegate may include an expiry date if he or she considers this necessary in the interests of safety.
(2) A limitation to single-engine aeroplanes up to 800 hp or to a particular aircraft type or types.
(3) A height limitation specifying a minimum level for the conduct of aerobatics. Delegates may specify any height limitation they believe necessary in the interests of safety, but the following height limitations would probably cover most circumstances:
• Not below 1500' AGL;
• Not below 1000' AGL;
• Not below 500' AGL; or
• Unlimited.

Note: There is no requirement for pilots to hold a permission at each higher level before being issued one at a lower level, although some form of progression would be the normal expectation. The delegate may issue a
permission with any height limitation that is based on safety considerations.”

The sample permission letter includes the following conditions:
“1 The minimum heights and distances to maintain separation from any group of persons must be
those specified for spectators in Civil Aviation Order (CAO) 29.4.
2. Subject to clause 3, the approved person must not conduct flight manoeuvres below the
minimum heights specified under regulation 157 of CAR 1988.
3. If the approved person is permitted under this instrument to conduct flight manoeuvres below
500 feet, the flight may only be conducted over a location approved by the appropriate CASA
office as suitable for the conduct of those manoeuvres.
4. Passengers must not be carried during manoeuvres below 1500 ft, nor during any acrobatic
demonstration, display or competition.
5. The approved person is not allowed to conduct an acrobatic flight over public gatherings.
6. The approved person must not conduct acrobatic manoeuvres within or over:
(a) any location where acrobatic manoeuvres are likely to be a hazard to the navigation of
other aircraft;
(b) any location known or likely to be noise sensitive;
(c) an area where an aircraft malfunction would endanger the lives of persons.”

Most importantly, a delegate cannot withdraw or cancel a low level aerobatic permission.
Issues for the Aerobatic Club
As the club purports to represent its members and to have an interest in the safety of its members it should take a proactive approach in considering the recent history of accidents and what it should do to improve the future safety of aerobatic pilots.

Pilots are encouraged to consider accidents (not just aerobatic accidents) where a change in the way the pilot conducted his/her operations would've have avoided the accident. CAAP 155-1 offers a process for individuals to go to their peers to seek feedback on their own operations and potentially improve the way they do things from a safety point of view. Increased safety can only result from this very simple, but disciplined activity.
Peer Review Process
“The peer review process is intended to provide an independent assessment by a similarly qualified person or persons on the way the pilot conducts the activity and to identify any incorrect techniques or practices that the pilot may have developed over time. It is not intended to be a flight test for the renewal of the permission, but an opportunity for constructive discussion with other practitioners with a view to enhancing the safety of a pilot’s performance.”
As noted above, the provision for indefinite permissions carries some risks in that there is no further assessment and it is extremely difficult to take away a permission.
The peer review process should be promoted by all pilots within the club.
“7.28.3 The following is the recommended procedure for the peer review process:
• The pilot should have had sufficient recent practice and/or training to be able to conduct a sequence of low-level aerobatics safely;
• The pilot should brief the observer(s) on the sequence to be flown;
• The pilot should fly the sequence under observation, either from the ground or the aircraft down to the level of the permission, or the level to which the pilot intends to exercise the permission, if higher;
• After the flight, the pilot and the observer(s) should de-brief the sequence to identify ways in which performance and safety could be improved; and
• The review is entered in the pilot's logbook and signed by the pilot and by the observers as a record to indicate that the observation and discussion has taken place. It could include a disclaimer that the observer is not certifying the pilot's competence.
7.28.4 The observers would need to have proficiency in low-level aerobatics and preferably also in assessing low-level aerobatic performance. Suitable observers would be any one of the following:
• CAR 155 delegate; or
• At least two other low-level permission holders with similar permissions; or
• CASA Flying Operations Inspector (FOI).
7.28.5 During the debriefing process it is important to be objective in identifying items that were done well and those that could have been done better. Emphasis should be on providing input and advice on ways to improve safety and performance rather than on questioning an individual’s ability.
7.28.6 Signing-off as an observer for peer review should not be construed as certifying the competency of the pilot, but that the review has taken place and that any issues of concern have been brought to the pilot’s attention.
7.28.7 The object is not to assess the pilot as suitable or otherwise to continue to hold the permission, but in cases where continued operation by the pilot would constitute a serious risk to air safety there would be some moral responsibility for the participants to counsel the pilot and, if necessary, bring this to the attention of CASA.”

A sample form of the peer review logbook entry:
PEER REVIEW iaw CAAP 155-1 AEROBATICS Section 7.28.
I have observed a low level aerobatic sequence performed by ......................
and debriefed the pilot on opportunities to improve safety & performance.
Note: this is not a certification of competency.
Signed ................................  ARN ...............   Date ..............

The aerobatic club would be neglecting the safety of its members if it did not promote peer reviews as recommended by the CAAP. There may be other actions the aerobatic club could take to improve safety however this is one measure which has clearly been introduced by CASA and which has been ignored.
 
It is important not to let peer reviews become rubber stamping for mates. Should the aerobatic club accredit reviewers? Should the aerobatic club provide training and guidance material for reviewers? (This could include a checklist which the reviewer may chose to retain.)
 
The actual review is verbal at a debriefing session after a performance has been observed. Although the logbook entry records that the review was undertaken there is no statement of competency recorded. However, reviewers may like to keep records of debriefs for their own purposes (perhaps if they were asked by a coroner). Althought the CAAP refers to a debrief of a single performance the reviewer should be encouraged to bring up relevant matters of a broader nature.

i.e. the basic objective is to identify any hazardous attitudes and help the pilot to understand that so as to take appropriate action.

a. Antiauthority (don’t tell me!).
e.g.
I can join the circuit neater than that and save a couple of minutes, some-one will tell me if there is conflicting traffic.
That's not a zoom climb after take-off – my aeroplane can sustain a 60 deg climb angle, some-one will tell me if there is conflicting traffic.
45 minutes fuel reserves - its only going to be a 10 minute flight.
The spectators can't see me unless I'm really close to the fence during my display.

b. Impulsivity (do something quickly!)
e.g. I'm flying three Unknowns in this contest.
Miserable weather and no contest flying so I'll just do a display.

c. Invulnerability (it won’t happen to me).
e.g. All those things happen to other people because ….... but
engine failure at low level
spin just took a little longer to recover than usual
something went wrong with that lomcevak
there was nothing wrong with the aeroplane before the first flight of the day

d. Macho (I can do it).
e.g. Watch me on Utube – I can fly just as well as …......
Difficult unknown but I need to be down where the judges can see me.

e. Resignation (what’s the use?).
e.g. I need to do three monthly checks and a biannual flight review and low level aerobatic renewals so why do I need to do these peer reviews?

Some-one is likely to tell you if you nearly have an accident but then you might've scared yourself anyway. The thing is that the one that will get you will be something different, something that has been brewing for some time as a result of your display of one or more of those hazardous attitudes. No-one has enough lives to learn from their own mistakes – learn from the mistakes of others.
Safety Pilot
The following condition on low level approvals limits the qualifications of a safety pilot in competitions:
“Passengers must not be carried during manoeuvres below 1500 ft, nor during any acrobatic
demonstration, display or competition.”
The delegate may vary this however I note that there is absolutely nought in the test crtieria to assess a pilot's ability to sit in the passenger seat of an aeroplane and act as pilot in command while another pilot (who does not have a low level aerobatic permission) perform low level aerobatics, especially under the stress of a competition. How safe is a new safety pilot?
PILOT PROFICIENCY PROGRAM
Many other local organisations offer Pilot Proficiency Programmes for their members.
eg http://aviationsafety.org.au/files/ASR_GPPP_Course_Introduction_Whats_it_all_about.pdf
http://www.piper.org.au/prof.html
http://www.cessnapilotsassociationofaustralia.org.au/pilotproficiency.html
http://www.mooney.org.au/index.php/mooney-safety-ppp.html
http://www.comancheflyer.com.au/new/join.php
http://www.abs.org.au/uploads/Promo_Brochure.pdf
Perhaps it is easier to list the flying organisations which do not offer Pilot Proficiency Programs to its members?
Did you notice the insurance companies feature as sponsors or offer discounts on insurance for those who attend?
Should an Aerobatic PPP be developed as an improvement and alternative to the peer review process?
FAA AC 60-22, Aeronautical Decision Making is a good reference. Another good reference is the book "A Pilot's Guide to Safe Flying" by local author Sander Vandeth.

CONCLUSIONS
We know that CASA has been reconsidering their policy on low level aerobatics so should the club take the initiative to improve safety for aerobatic pilots before CASA imposes any restrictions?
If you consider the details of any accidents, not just those involving aerobatics, do you think that an effective regime of peer reviews would have eliminated any of them? Or even just one – would that make it worthwhile for all to undertake peer reviews?
How safe is a novice safety pilot in a competition? What do they know of their responsibilities and required capability?
Many organisations offer Pilot Proficiency Programs for their members – should the aerobatic club undertake a similar program for the benefit of its members?