+ Sidebar

e-Auditing: A Matter of Context

Picture11.png
by Shauna Wilson In my…

by Shauna Wilson

In my travels, I am learning from others that some third-party registrars are not accepting internal audits that are conducted virtually. This growing concern over auditing methods is the antithesis of modern work environments. Obviously, a virtual audit is conducted for a remote office, in which most of the time, teams are working together online to communicate and resolve issues. Shouldn’t the audit method replicate the actual working environment? In this article, I will examine e-auditing validation criteria, the use of the context of the organization, and conclude by reviewing other opportunities gained using e-auditing methods.

An e-audit is a systematic, independent, and documented process to obtain evidence through electronic means to determine the extent of conformity to the audit criteria.  The use of e-auditing is increasing because so much of the technology we use in our daily lives—connecting with friends on Skype, finding jobs through LinkedIn, or attending online classes—is done over the Internet. These activities become a gateway to enhancing and applying online communication techniques. The more familiar we become with technology, the less anxious we feel about its interactive uses.

Validating an e-audit relies on the technology used and the auditor’s skill to facilitate a virtual meeting while coordinating with the remote location to find nonconforming evidence. This coordination of events is not an easy task without technical grounding in information technology and facilitation skills. Realistically speaking, a fair amount of registration auditors are limited in this area due to their intense travel schedules. At best, they are passive listeners in “all hands” online meetings. This is not a reason to stop conducting internal audits virtually. Note that ISO 9001:2015 itself and its requirement to understand the context of the organization seems to be a tacit endorsement of the e-auditing process.

ISO 9001:2015 provides an illustration of how complex businesses have become to compete in a global market to offer affordable products. For example, products that contain batteries often make headlines. We can no longer carry a Samsung Note 7 on an airplane or fly with a motorized skateboard. Let’s examine a fictional scenario to apply the “context of the organization” requirement with e-auditing methods.

Battery ABC Co. is a research and development laboratory that designs and manufactures lithium batteries in a small but powerful format, enabling longer charges and lighter cell phones, tablets, and watches. Battery ABC Co. relies on external providers to manufacture its batteries.  Based on the following strengths-weaknesses-threats-opportunities (SWOT) analysis, what internal audit plan does the company’s management team need to mitigate both internal and external issues to meet the needs and requirements of all parties?

 

 

In this scenario, management should consider a hybrid audit model. This would incorporate multiple verification methods: onsite audits, e-audits, document information reviews, and product testing to understand the supply chain quality management system. An audit plan should include design and development, the release of product at remote locations, and external provider reviews to ensure manufacturing processes are managed appropriately. External provider reviews could include line yields and defect Pareto charts, process e-audits, and product testing by a third-party lab to ensure the battery meets product specifications and regulatory requirements.

A hybrid audit model approach is necessary for organizations of this nature to completely verify internal and external issues and that interested parties’ needs and requirements are met. The following chart shows an example of a hybrid audit management plan.

e-Auditing: A Matter of Context

e-auditing is an efficient and effective method for risk-based thinking, working with external providers to ensure process controls are in place, reviewing product-related issues real time, and enhancing understanding among all interested parties. Companies that invest in e-auditing allow remote locations to learn from one another. They gain a better understanding of remote processes and can leverage and standardize common processes across distant locations.  Rather than refuse e-auditing methods, training to use technology while facilitating an audit should be a priority of internal and external auditors.

 About the author

Shauna Wilson is president at Amazon Consulting Inc. She is a performance management consultant who designs efficient and effective quality systems. Wilson is an IRCA-certified auditor and leading expert in remote auditing. She holds a Master’s degree in performance management technologies/instructional design.  Wilson wrote InterneTeaming.com: Tools to Create High Performance Remote Teams and co-authored eAuditing Fundamentals: Virtual Communication and Remote Auditing and has been featured in Quality Progress and ASTD’s InfoLine.  Wilson is the education/social responsibility chair at ASQ’s Portland, Oregon Section 607 and currently serves as the U.S. TAG expert for PC/TAG 302 ISO 19011 auditing management systems.

The post e-Auditing: A Matter of Context appeared first on The Auditor.

PDCA of Audit Evidence

by Cathy Fisher The process…

by Cathy Fisher

The process approach to auditing incorporates Dr. Edwards Deming’s well-known Plan Do Check Act cycle in defining audit scope and criteria. This structure also guides the gathering of evidence to support process-based auditing. The four types of audit evidence that support the process approach include:

  • Plan = Adequacy
  • Do = Conformance
  • Check = Effectiveness
  • Act = Improvement

Let’s consider the ISO 9001:2015 requirement for organizational knowledge: “The organization determines and maintains the knowledge necessary for the operation of its processes.” There are many ways for an organization to accomplish this requirement. For example, the leadership team of one organization may decide to implement a process for capturing lessons learned. This system-level decision encourages the development of a process within the organization to “determine and maintain” this information. The leadership team also decides that the chief information officer will be responsible for this process. From an auditing perspective, this is now a quality management system (QMS) process to audit.

The planning part of the audit evidence for this process may be gathered through interviewing the process owner (in this case, the CIO), and/or reviewing QMS documentation that describes this process. When we’re auditing the planning process, we’re looking for adequacy in its definition. We want to know the answers to such questions as, “What is this process?” “What is the desired output of this process?” and “Who is responsible for this process?”

Audit evidence that demonstrates adequacy for this example may include the following:

Type of Evidence Evidence for Lessons Learned
Recognized name of this process in the organization Organizational lessons learned, process identified in QMS map
Assignment of process ownership (either understood or designated through QMS roles and responsibilities) CIO, interview with leadership team, QMS process ownership matrix
Definition of the output of this process and its criteria/requirements, as well as how that output will be evaluated Searchable database containing lessons learned. Required fields for entry:  QMS process, date added, internal expert.
Identification of triggers or inputs that activate this process QMS processes generating lessons learned, including: investigations of customer complaints, product/service issues, corrective and/or preventive actions, process improvements, project reviews, management reviews, etc.
Description of steps involved in transforming the process inputs into the expected output Explanation of lessons learned process by CIO, procedure or training aid describing use of lessons learned database
Determination of resources needed to accomplish this process Tangible evidence such as computer database program, database administrator, input file mask, or computer server

 

Depending on the nature and complexity of the process being audited, there may be additional evidence that reflects the adequacy of the process.

Next, considering the “Do” in the PDCA process relates to audit evidence that demonstrates conformance. It’s easy to simply look at the execution of the process being audited as reflected in procedures and/or documentation or as described by the process owner. However, this stage of the process begins with the communication of the process and its requirements to those involved. This communication may be included in the “Plan” stage too.

When we think of auditing, conformance is what we typically mean:  Is the plan or process being followed?  Are we doing what we said we would do? The audit evidence of conformance can typically be found in three forms:

  • Tangible evidence: Procedures, records, computer programs
  • Observations: Auditor observing process execution
  • Admissions: Statements of fact by those performing the process. This may include explanation of the process by someone performing it or verification of interacting process as an audit trail.

Audit evidence of conformance also leads to recognition of supporting process audit trails; specifically processes that provide required resources, e.g., training/competency development of those executing the process, maintenance of equipment used in the process, control and identification of materials, availability and currency of process instructions, and control of work environment factors.  Auditing of these supporting processes is an essential part of applying the process approach.

Evidence of conformance for the lessons learned process may include:

Type of Evidence Evidence for Lessons Learned
Communication of plan Tangible: Procedure or training aid for entering lessons learned into database.
Understanding of plan Tangible: Training/briefing record of attendance.

Observation: Demonstration of lessons learned correctly entered into database.

Implementation of plan Observation: Process owners entering lessons learned into database.

Tangible: Contents of database Admission: Reference to database administrator for entry review and posting.

Supporting process audit trails Availability, access to, and back-up of computer database, document revision control of procedure, training of database administrator.

Simply confirming conformance to the plan isn’t sufficient in auditing from the process approach perspective. Evaluating the effectiveness of the plan is also essential in ISO 9001:2015, in which results are emphasized.  There are several prerequisites for auditing the “check” stage of the of process approach:

  • Criteria describing the expected/desired output from the process is clearly defined and can be evaluated (measurable either quantitatively or qualitatively).
  • Output from the process is being evaluated and compared to this criteria.

Evidence of effectiveness for the lessons learned process may include:

Type of Evidence Evidence for Lessons Learned
Criteria defined for evaluating effectiveness can be quantitative or qualitative in nature, objective, or based on perceptions. Number of lessons learned recorded, frequency of applying lessons learned to other processes, familiarity with database.
Process implemented for performing evaluation of process effectiveness, how often data is gathered, collected, and reviewed, and by whom. Number of lessons learned entered in database, which functions are entering them.
Results gathered and evaluated against process output criteria: checklists, check sheets, automatic data collection, trend charts, and surveys. Monthly activity report generated from database, database user satisfaction survey.

The effectiveness of any process is in the results. However, knowing how those results were achieved is also important for improvement.

The “act” phase of the process approach to auditing focuses on improvement.  From an auditing perspective, there are some prerequisites associated with auditing a process for improvement:

  • Threshold for action is established. This can include addressing acute issues of process ineffectiveness (a specific output nonconformity, such as a nonconforming product in a manufacturing process), or when a different output or output criteria is needed based on changes in customer or internal requirement.
  • Prioritization of processes for improvement as limited resources don’t allow for every process to be improved simultaneously.
  • Information about the process before and after actions taken for improvement is documented.

Quite a bit of evidence may exist for auditing an improvement process, including:

Type of Evidence Evidence for Lessons Learned
Recognition of improvement potential, action threshold, decision criteria Is the number of recorded lessons learned less than three per month?
Prioritizing processes for improvement, importance of process output, effect on organization of current process performance Lessons learned from all customer complaints investigated to be identified and included in database.
Process baseline, current process definition, current performance information available Ninety percent of lessons learned entered in quality department database.
Process applied for managing improvement, corrective action, kaizen. Focus workshop involving key process owners to identify and input recent lessons learned from all areas of the organization.
Results from improvement are compared with baseline performance measurements. Three months after workshop, all departments consistently entering an average of five lessons learned in database per month.
Management of change from improvement, update of documentation, possible retraining Examples and definition of lessons learned added to training aid and database instructions.

Keep in mind that when auditing improvement, not all stages of the improvement process may necessarily be complete at the time of the audit. This could initiate a follow up point for future audits. Additionally, not every improvement effort leads to a positive result. This isn’t a nonconformity, but rather an opportunity to look beyond the process being improved to also consider the process and/or methods being used for managing improvements.

To achieve a true process-based audit, questions should be generated during audit planning to evaluate a process’ adequacy, conformance, effectiveness, and improvement. Evidence must be gathered to support each of these evaluators in using the process approach to audit any QMS process.

About the author

Cathy Fisher is founder and president of Quistem LLC, which provides online and onsite management systems implementation, update, and assessment services for manufacturers and other industry sectors. Cathy has more than 30 years of respected auditing expertise, having led internal audit programs at many manufacturing organizations during her career. Cathy also has extensive experience conducting management systems registration audits, as well as establishing supplier evaluation and development programs.

She has held numerous auditor certifications including ASQ CQA, RAB-Certified Quality Systems Auditor, and ISO/TS 16949 IATF-recognized auditor. She has conducted internal and external audits that total more than 1,000 audit days and trained hundreds of management systems professionals as auditors. Cathy is passionate about the value auditing can bring to organizations and enjoys mentoring the next generation of technical professionals to develop their auditor excellence.

 

 

The post PDCA of Audit Evidence appeared first on The Auditor.