And we have the battle scars to prove it.

For a number of years our company has been AS9100 registered, in recognition of the many unique quality management requirements of the military/aerospace sector, and our company’s willingness to abide by them. Section 7.1.2 of the AS9100C standard specifies that a registered organization “shall establish, implement, and maintain a process for managing risk to the achievement of applicable requirements.” Elaborating, the standard goes on to say that to be compliant, the qualifying organization will assign responsibilities for risk management; define risk criteria, including likelihood and probable consequences of taking certain risks; identify, assess and communicate these risks throughout the process; identify, implement and manage actions to mitigate risks exceeding established risk acceptance criteria, and accept remaining risks after completing any mitigating actions.

Charming. Still with me?

Pointedly, the standard makes no attempt to define, specify or impose what risk is most critical, burdensome, or potentially debilitating to a specific organization, leaving that for each company to define as an integral part of their process. Risk is sort of like McCarthy Era communism: It’s out there, waiting to be rooted out.
Which is why divine providence has graced us with auditors.

What, then, does this mean?

Let us postulate first that, in my role as a business owner, a significant portion of every waking hour is dedicated to some form of risk management. To imagine otherwise would insult the intelligence of any reasonable business owner. The mere reality of owning and operating a firm in 2013 is an ongoing case study in risk management. Further, the small business sector, in which our company operates, owes its existence in part to doing things the Big Guys either can’t do quickly or won’t do without an accompanying mountain of paperwork and procedure. That’s the symbiosis between big and small business. Big business checks up on the well-oiled functioning of that symbiosis by means of accrediting standards and organizations, observed and enforced by our friends in the auditing profession.
Who, ironically, have the final word on risk, without risking a thing themselves. The Auditor gets to decide whether we struggling businesses have properly assessed risk in the evidence documenting the daily conduct of our own operations. So judgmental. The nerve.

Risks for me, and for our business, include the following, in no particular order:

• That the customer is certifiably insane or, worse, bereft of common sense.
• Human stupidity, either self-inflicted or from a qualified external source.
• Armageddon.
• Acts of God.
• Random chance or caprice.
• Failure of the tested product to work as designed.
• Failure of our designated test system to work as intended.
• Sudden illness or plague, famine, war or pestilence.
• Failure of the customer to pay their bill.
• Unplanned, and potentially lethal, outbreaks of incompetence on the part of the customer, the test engineer or both.

We deal with most of these things daily. We have the battle scars to prove it.

Let us further stipulate that as test engineers we face certain unique risks, high among them being:

• That the customer has failed to adequately specify their requirements because of ignorance, spinelessness or an assumption of supernatural intuitive powers on the part of the test services provider. (We offer the latter for an additional charge.)
• The customer has unrealistic expectations regarding (less) time, (lower) cost or worse: both. But in any event, we should just know!
• That test engineering services represent a cost rather than an enduring value and, therefore, are to be avoided, except as a last resort, usually at 4:45 p.m. on a Friday preceding a three-day weekend.

Why does this matter to me? Because I wonder in practice whether risk management, according to the standard, makes a difference. A personal example will elucidate.

Seeking a brief respite from the delights of day-to-day risk management, in November 2009 my wife and I took a Mediterranean cruise. The ship was the Costa Concordia. You may have heard of it. It is the same ship now in the headlines, as it is being raised upright and (hopefully) salvaged from the rocks of Giglio Island in Tuscany, where it has rested on its side for the past 19 months.

It would be an understatement to say that the story of the Costa Concordia’s demise is a monument to poor risk management. Well-deserved opprobrium has been attached to the then-Captain’s reckless disregard for the hazards of sailing in shallow water with 4,200 passengers. However, when we sailed roughly the same course in 2009, the ship had a different captain, but a similar attitude to that portrayed by the media the night the Concordia ran aground. Specifically, to our eyes, the management of the ship barely rose above the level of chaos. Crewmembers on our cruise simply did not know what to do. Lifeboat drills, for example, seemed to be treated as attendance-optional events. (My understanding is that they are required on the first day of a cruise by maritime law, and attendance by all passengers is mandatory.) Baggage was misplaced. Tour excursions were completely disorganized. A culture of indifference to customer comfort, service and safety seemed to permeate the vessel.

But, by God, they passed their audits. They had the signatures on file to prove it.

After that cruise in 2009, my wife and I vowed never to journey on that ship – or cruise line – again. Ever. And when the Concordia hit the rocks in Tuscany in January 2012, we were saddened but hardly surprised. To us it seemed an inevitable consequence.

But they were audited.

The cynical view of risk management taken by many business owners is to game the system and do the minimum necessary to pass a cursory third-party audit, demonstrating sufficient wide-eyed conviction to hoodwink the auditor into thinking they are taking the issue seriously. We, the audited, are further expected to rank these risks for the auditor’s benefit. Less work for them that way. No-risk auditing, if you will.

Auditors are human too. This does not make them bad or inherently evil people. They have to eat like the rest of us. And like the rest of us, they often choose the path of least resistance. Auditors are also seldom confronted with the challenge and pressure of having to pay the bills. There are few documented instances of auditors rising to the top of large organizations, and for good reason. Nevertheless, by some pretzel logic, auditors hold almost arbitrary sway over how our business defines risk, and how we quantify that definition.

So what do we as a testing company do? For us, the crucial moment in risk identification comes during what we call the Contract Review Process, in which we determine the following:

1. Can we do the project as it has been specified?
2. If it has not been completely specified, can we fill the gaps in the customer’s specification, and demonstrate to concerned laypersons (buyers) that we both know what we’re doing and warrant separating their company’s money in our direction?
3. Most important, are there things we absolutely cannot do, and have we made these “NOs” explicit in our communications to the customer?

Following these simple steps methodically ensures negligible risk. Or, as the First Officer of Asiana 214 said to his Captain – busy setting the autopilot the morning of July 6, 2013 (somewhere west of San Francisco) – “What a beautiful morning, and what could possibly go wrong on a day like today?”

Robert Boguski is president of Datest Corp., (datest.com); rboguski@datest.com. His column runs bimonthly.

Testing mission-critical functions in automotive electronics.

Read more ...