Latest News

+ Categories

News Categories

UN Report Examines Credibility of Management Systems Certification

While market surveillance is a…

While market surveillance is a well-known concept in a regulatory context, a similar approach has been shown to be effective when applied to the voluntary certification of quality management systems, according to a recent report by the United Nations Industrial Development Organization.

In a series of articles, The Auditor Online will share highlights from the “Good Practices: Experience in the Market Surveillance of ISO 9001 Quality Management Systems” report, which presents the case for accreditation bodies or stakeholders completing market surveillance visits to certified organizations to ensure the quality of their products or services.

According to the report, user feedback on the performance of ISO 9001-certified suppliers and the accredited certification process reveals ongoing debate around the effectiveness and credibility of accredited certification.

The debate centers on whether organizations are deriving tangible benefits through ISO 9001 certification, if the certification process is being performed effectively, and whether ISO 9001-certified suppliers can be relied upon to provide “consistent, conforming products and services” to customers.

Systematic feedback was gathered between 2009 and 2015 from purchases regarding their perceptions of ISO 9001-certified suppliers, and from ISO 9001-certified organizations about the implementation and certification process. Feedback was gathered through a survey and a market surveillance activity which involved one-day visits to certified organizations. These visits aimed to determine confidence levels related to various aspects of the organization’s quality management system and the overall level of confidence in the certification process.

To reflect their confidence level in organizations being examined, assessors assigned the following grades:

  • Grade One: “Little or no confidence.” Little or no evidence to support the implementation of this topic.
  • Grade Two: “Some evidence presented, but not at all convincing.” Some evidence was presented, but in the professional judgement of the assessor, there would probably be evidence to support a nonconformity if a detailed audit trail were to be followed in a full system audit.
  • Grade Three: “OK—No reason to doubt that this is being addressed correctly.” The “default” grade, where there is no evidence to suggest reasons for concern, based on the assessor’s experience and professional judgement.
  • Grade Four: “Clear evidence that this is being done, and meets the intent of the relevant standard.” Sufficient objective evidence provided a high degree of confidence that the organization meets requirements.
  • Grade Five: “We can be proud to use this organization as a benchmark for this topic.” To be reserved for truly excellent performance.

A key finding was that there were notable differences in the performance and level of confidence in organizations certified by different certification bodies and under accreditation from different accreditation bodies. Other findings included:

  • More than 95 percent of certified organizations surveyed considered the effective implementation and accredited certification of quality management systems to have been a “good” or “very good” investment.
  • Overall perception of the ISO 9001 standard and accredited certification was good, although the role of accreditation wasn’t well understood by purchases or certified organizations.
  • The purchasers surveyed were mainly satisfied with the performance of their ISO 9001-certified suppliers. In general, ISO 9001-certified suppliers performed “better” or “much better” than non-certified suppliers. However, poor responsiveness of certified organizations to customer complaints was one area of concern.

Overall, the performance of the certified organizations that were visited was good, which demonstrates the effectiveness of the accredited certification process. However, between six and eight percent of organizations returned unsatisfactory results, and one percent raised serious doubts about the validity of their certification—which calls into question the effectiveness of the certification process.

According to the report, market surveillance activities can be used in place of traditional accreditation methodologies such as office assessments and witnessed audits to investigate complaints or concerns.

Market surveillance can also uncover trends, strengths, and weaknesses in the implementation of quality management systems in particular regions or sectors—all of which is useful information for certification bodies to focus their attention on during surveillance and renewal audits.

To give the data context, the report describes independent third-party certification as a common method sought by organizations to demonstrate that they have met all the requirements of ISO 9001. The certification consists of a certificate of conformity and ongoing surveillance to ensure that the system is maintained in accordance with the standard.

To demonstrate competence to administer management system certification, a certification body may become accredited by an accreditation body. This accreditation is based on the requirements defined in ISO/IEC 17021-1 and is supplemented by other discipline-specific or sector-specific requirements in some cases.

Continue to read The Auditor Online in the coming weeks for more findings from the “Good Practices: Experience in the Market Surveillance of ISO 9001 Quality Management Systems” report.

The post UN Report Examines Credibility of Management Systems Certification appeared first on The Auditor.

ASQ Survey: Quality Improvement Methods Could Help to Reduce Waste, Debt

Only eight percent of government…

Only eight percent of government workers believe the U.S. federal government is running efficiently, adding that shifting priorities and fears of job loss pose major obstacles to making improvements, according to a recent ASQ survey.

In January, ASQ surveyed its members and customers who work in a diverse range of government sectors including defence, health care, military, transportation, finance, and agriculture.

Thirty-one percent of survey respondents said shifting strategies, goals, and priorities are the biggest challenge to implementing quality improvement methods in the federal government.

Other survey highlights included:

  • 19 percent said the biggest hurdle is overcoming organizational fears about the impact of cost-cutting on jobs and project funding.
  • 14 percent cited the greatest challenge as a lack of awareness about quality methods like Lean Six Sigma and how they can benefit the organization.
  • 12 percent said the biggest challenge is viewing quality improvement as a temporary trend.

“The federal government has a great opportunity with the application of quality tools reduce waste and, as a direct result, decrease the burden on the taxpayers,” said Mark Abrams, chair of ASQ’s Government Division.

“Quality tools have proven successful in a variety of sectors including manufacturing and health care, and can be used at all levels of government to increase operational efficiencies and improve taxpayer experiences.”

In addition to noting challenges with the federal government’s structure, survey participants ranked quality improvement tools they see as having the biggest potential for reducing federal waste. Seventy-six percent said lean would provide the greatest benefit to increasing efficiency, followed by Lean Six Sigma at 71 percent, and Six Sigma at 48 percent.

Quality tools such as Lean Six Sigma have been highlighted by past U.S. presidential candidates as a tool for reducing waste and improving efficiency in federal government. Of those surveyed, 42 percent have implemented Lean Six Sigma in their organization. Of these 42 percent, more than 86 percent said it’s been somewhat, very, or extremely efficient.

Federal Government Agencies That Need the Most Help

Fifty-eight percent of respondents ranked infrastructure as the area federal government could most benefit from reducing waste and cutting costs. Furthermore, 55 percent of respondents said Health and Human Services and National Defense would best benefit from quality tools, followed by Immigrations and Customs at 54 percent, and Homeland Security at 53 percent.

First Steps in Ensuring Success with Quality Methods

Possible first steps suggested by respondents for government to effectively implement quality tools like lean and Six Sigma, include:

  • Generate a stronger commitment from management/leaders and integrate into performance standards.
  • Present clearly defined goals and use selected quality tools to implement processes.
  • Provide training for key members of the administration and government agency management teams.
  • Educate government officials at various levels on the quality tools available and techniques that can be applied to their specific situations.
  • Mandate the use of selected quality tools and give them time to work instead of quickly switching to the latest “great” thing.

The post ASQ Survey: Quality Improvement Methods Could Help to Reduce Waste, Debt appeared first on The Auditor.

Baldrige Foundation to Expand Awards Program

  The Baldrige Foundation is…


The Baldrige Foundation is expanding its leadership awards program with the establishment of the E. David Spong Lifetime Achievement Award.

The first recipient of the award is its namesake: Dr. E. David Spong.

“This new award recognizes an individual who has performed truly extraordinary service and created a lasting legacy which will inspire future generations of leaders” says Al Farber, Foundation president and CEO.

Dr. Spong has been a Baldrige practitioner and advocate throughout his career, and is a two-time recipient of the Baldrige National Quality Award. He led the Boeing Airlift and Tanker Programs to the 1998 award in manufacturing, and in 2003, he led Boeing’s Aerospace Support division to the Baldrige Award in the service sector.

“I am thrilled to lend my name to this award, but the Baldrige Enterprise is the real champion,” says Dr. Spong. “Baldrige recipients serve as role model organizations for everyone else to emulate. Through Baldrige, ‘best practice’ becomes documented, data driven, evidence-based examples of performance excellence. These examples reach every sector of the economy—manufacturing, small business, service, health care, education, the nonprofit sector and government, and beginning last year, cybersecurity. Baldrige is a recipe for first-class performance.”

“David’s career spans over five decades and encompasses many, varied leadership roles,” continues Faber. “At every step David has been a champion of the Baldrige Framework and the Baldrige Enterprise. The Baldrige Foundation is honored to present this award to him, and future leaders who embody his lifetime commitment to quality.”


The post Baldrige Foundation to Expand Awards Program appeared first on The Auditor.

Auditor Profile: Inspired to Improve Food Safety

With more than 35 years…

With more than 35 years experience in the food industry in roles such as kitchenhand, caterer, and auditor, Marjorie Harvey is somewhat of a food safety expert. The Auditor speaks to Harvey to learn how she has channeled her passion for the food industry into a successful food safety auditing and consulting career.

Harvey began her career in the food industry peeling potatoes in a local pub and then as a trade cook. She later went on to own a catering business, manage food services in aged care, and teach hospitality.

After witnessing unsanitary food handling practices and seeing reports of food poisoning outbreaks in the media, Harvey was inspired to improve food safety through designing food safety programs. Her interest in auditing soon followed.

“Food sites had no food handling policies, monitoring records, or food safety systems,” Harvey said. “I enrolled in university so I could learn more about the Food Act and food safety standards to become an accredited trainer and auditor.”

Harvey’s university studies included learning about HACCP, ISO standards, food technology, and auditing. Here, she gained the necessary skills to become the first female third-party food safety auditor for the Department of Human Services in Victoria, Australia. Harvey then went on to establish Australian Food Hygiene Services—an accredited company offering consulting, training, and auditing for the hospitality industry.

Working predominantly in the health care sector, Harvey’s role as director involves auditing, training, writing food safety programs, and consulting.

Throughout her career, Harvey has designed and implemented more than 800 food safety programs for hospitals, aged care facilities, child care sites, and Meals on Wheels. She continues to offer her services to restaurants, hotels, cafés, community services, government bodies, and prisons.

Harvey reflects on highlights of her career as including assisting the industry in all aspects of food safety management and compliance.

“It has been a privilege to work in areas that are high risk and to have the opportunity to assist clients with gaining compliance, and offering an opportunity for continual improvement if they request.”

However the job has its challenges—particularly in regards to travel and the demands of report writing. Getting a start in the industry also isn’t easy.

“From the start it was very challenging as no one had experienced an audit,” Harvey said. “But as time goes by it has become less challenging as most understand the requirements expected from an audit.

“It is also challenging to keep up with new equipment, changed food processes, and current trends.”

Harvey believes a lack of mentoring opportunities and incentives for seasoned auditors to assist new auditors are key issues in the profession, and suggests the following solution:

“Attending relevant industry conferences and auditor forums to ensure they keep abreast with the current requirements.”

The post Auditor Profile: Inspired to Improve Food Safety appeared first on The Auditor.

e-Auditing: A Matter of Context

by Shauna Wilson In my…

by Shauna Wilson

In my travels, I am learning from others that some third-party registrars are not accepting internal audits that are conducted virtually. This growing concern over auditing methods is the antithesis of modern work environments. Obviously, a virtual audit is conducted for a remote office, in which most of the time, teams are working together online to communicate and resolve issues. Shouldn’t the audit method replicate the actual working environment? In this article, I will examine e-auditing validation criteria, the use of the context of the organization, and conclude by reviewing other opportunities gained using e-auditing methods.

An e-audit is a systematic, independent, and documented process to obtain evidence through electronic means to determine the extent of conformity to the audit criteria.  The use of e-auditing is increasing because so much of the technology we use in our daily lives—connecting with friends on Skype, finding jobs through LinkedIn, or attending online classes—is done over the Internet. These activities become a gateway to enhancing and applying online communication techniques. The more familiar we become with technology, the less anxious we feel about its interactive uses.

Validating an e-audit relies on the technology used and the auditor’s skill to facilitate a virtual meeting while coordinating with the remote location to find nonconforming evidence. This coordination of events is not an easy task without technical grounding in information technology and facilitation skills. Realistically speaking, a fair amount of registration auditors are limited in this area due to their intense travel schedules. At best, they are passive listeners in “all hands” online meetings. This is not a reason to stop conducting internal audits virtually. Note that ISO 9001:2015 itself and its requirement to understand the context of the organization seems to be a tacit endorsement of the e-auditing process.

ISO 9001:2015 provides an illustration of how complex businesses have become to compete in a global market to offer affordable products. For example, products that contain batteries often make headlines. We can no longer carry a Samsung Note 7 on an airplane or fly with a motorized skateboard. Let’s examine a fictional scenario to apply the “context of the organization” requirement with e-auditing methods.

Battery ABC Co. is a research and development laboratory that designs and manufactures lithium batteries in a small but powerful format, enabling longer charges and lighter cell phones, tablets, and watches. Battery ABC Co. relies on external providers to manufacture its batteries.  Based on the following strengths-weaknesses-threats-opportunities (SWOT) analysis, what internal audit plan does the company’s management team need to mitigate both internal and external issues to meet the needs and requirements of all parties?



In this scenario, management should consider a hybrid audit model. This would incorporate multiple verification methods: onsite audits, e-audits, document information reviews, and product testing to understand the supply chain quality management system. An audit plan should include design and development, the release of product at remote locations, and external provider reviews to ensure manufacturing processes are managed appropriately. External provider reviews could include line yields and defect Pareto charts, process e-audits, and product testing by a third-party lab to ensure the battery meets product specifications and regulatory requirements.

A hybrid audit model approach is necessary for organizations of this nature to completely verify internal and external issues and that interested parties’ needs and requirements are met. The following chart shows an example of a hybrid audit management plan.

e-Auditing: A Matter of Context

e-auditing is an efficient and effective method for risk-based thinking, working with external providers to ensure process controls are in place, reviewing product-related issues real time, and enhancing understanding among all interested parties. Companies that invest in e-auditing allow remote locations to learn from one another. They gain a better understanding of remote processes and can leverage and standardize common processes across distant locations.  Rather than refuse e-auditing methods, training to use technology while facilitating an audit should be a priority of internal and external auditors.

 About the author

Shauna Wilson is president at Amazon Consulting Inc. She is a performance management consultant who designs efficient and effective quality systems. Wilson is an IRCA-certified auditor and leading expert in remote auditing. She holds a Master’s degree in performance management technologies/instructional design.  Wilson wrote Tools to Create High Performance Remote Teams and co-authored eAuditing Fundamentals: Virtual Communication and Remote Auditing and has been featured in Quality Progress and ASTD’s InfoLine.  Wilson is the education/social responsibility chair at ASQ’s Portland, Oregon Section 607 and currently serves as the U.S. TAG expert for PC/TAG 302 ISO 19011 auditing management systems.

The post e-Auditing: A Matter of Context appeared first on The Auditor.

CEO Blog: We’re Listening!

Talking with our customers, certified auditors, engineers, and training providers, it is clear that the world we work in is continually changing. When many of us started our careers 20 to 30 years ago, everything seemed straightforward. We needed auditor qualifications to demonstrate that we were qualified and competent. As…

bainesTalking with our customers, certified auditors, engineers, and training providers, it is clear that the world we work in is continually changing. When many of us started our careers 20 to 30 years ago, everything seemed straightforward. We needed auditor qualifications to demonstrate that we were qualified and competent. As we extended to a wider range of audit programs, or schemes, we extended the qualifications. Since then, much has changed. We have people starting their careers, through to those retiring, but recognize that those retirees still want to be involved in some capacity. It is important that we listen to the needs of all of our customers from young people at the start of careers, to those diversifying, to those whose reputation is well established. We need to also be sure we add value to all stages. This is our mission. With this in mind, we have a number of initiatives we are following.

Auditor registration is an attractive career path for many. ASQ Certified Quality Auditors (CQA) have already expressed an aptitude for audit and so we are offering Exemplar Global certification to those already CQA qualified. If this is of interest, please contact us. A similar initiative for Certified Quality Engineers is being developed. Both of these reflect the “whole of career” service that both ASQ and Exemplar Global aim to provide.

Recently, Exemplar Global released an ISO 13485 Medical Devices Auditor scheme. Previously a scope of the quality management system scheme, the launch of this scheme reflects a response to a need from customers who specialize in this important sector.

For many years Exemplar Global has offered a Provisional Auditor scheme for those just starting and maybe do not yet have the experience to become fully qualified auditors. Until now, there has been a limit to the time you could remain “provisional.” However, recognizing that many value the qualification but do not have that experience, we have decided that Provisional Auditor status can be retained. Naturally, we still encourage everyone who can to upgrade to fully registered auditor status to demonstrate their qualification and experience.

Finally, for now, we are updating our iNARTE schemes to ensure we continue to add value to this important group of customers.

At Exemplar Global, we are listening to our customers and wider group of stakeholders. Please let us know your feedback.

Revised ISO 31000 DIS Seeks to Simplify Risk Management

The revision of ISO 31000:2009, Risk management –…

The revision of ISO 31000:2009, Risk management – Principles and guidelines has progressed to Draft International Standard (DIS) stage, with the draft now available for public comment.

ISO 31000 provides guidelines on the benefits and values of effective and efficient risk management and aims to help organizations to better understand and address uncertainties.

The revision seeks to simplify the standard using simple language to make risk management easy to understand. To do so, the terminology of ISO 31000 has been reduced to simple concepts and some terms have been moved to ISO Guide 73, Risk management – Vocabulary, which deals specifically with risk management terminology. This is intended to be read alongside ISO 31000.

Jason Brown, chair of ISO technical committee ISO/TC 262, Risk management, that developed the standard said, “The message our group would like to pass on to the reader of the DIS is to critically assess if the current draft can provide the guidance required while remaining relevant to all organizations in all countries.

“It is important to keep in mind that we are not drafting an American or European standard, a public or financial services standard, but much rather a generic international standard,” Brown said.

The draft also includes improvements such as the importance of human and cultural factors in achieving an organization’s objectives and an emphasis on ingraining risk management in the decision-making process. Despite these changes, the overall message of ISO 31000 of integrating the management of risk into a strategic and operational management system remains the same.

The next step in the process is to finalize the revision work to reach the Final Draft International Standard stage. The finalized version of ISO 31000 is expected to be published by the end of the year or early 2018.

The post Revised ISO 31000 DIS Seeks to Simplify Risk Management appeared first on The Auditor.

Revised ISO 44001 Standard to Aid Business Collaboration

A new standard has been…

A new standard has been published to help businesses collaborate with other organizations or individuals.

ISO 44001, Collaborative business relationship management systems  Requirements and framework, aims to assist companies to establish and improve collaborative relationships, both within and between organizations.

The standard provides the overall components of a management system for business relationships as well as operational process requirements.

ISO 44001 follows the common high-level structure, making it easier for anyone using multiple management systems. The standard also features an eight-stage life cycle to ensure sustainable relations, including operational awareness, value creation, knowledge, internal assessment, partner selection, working together, staying together, and exit strategies.

The standard was developed by ISO project committee ISO/PC 286, Collaborative business relationship management – Framework. ISO/PC 286 Secretary Mick Maghar said the standard is applicable to both private and public organizations of all sizes.

“ISO 44001 can be used on many different levels, from a single project or one-to-one partnership to multiple partner alliances across the whole organization,” Maghar said. “It will help businesses to work better together, maximize the benefits of their stakeholder relationships and support their sustainability.”

Click here for more information about ISO 44001:2017.

The post Revised ISO 44001 Standard to Aid Business Collaboration appeared first on The Auditor.

ISO Celebrates 70 Years

2017 is ISO’s 70th anniversary!…

2017 is ISO’s 70th anniversary! The ISO story began in 1946 when delegates from 25 countries gathered in London to discuss the future of standardization. A year later, on 23 February 1947, ISO officially came into existence. In this post-war era, the founding members saw international standards as a key to the world’s reconstruction efforts.

Back in 1947, the purpose of the fledgling organization was to facilitate the coordination and unification of standards developed by its member bodies, all of which were national standardization entities in their respective countries. The founders decided that the organization would be open to every country wanting to collaborate with equal rights and equal duties.

These founding principles still hold true today and the ISO family has blossomed to include 163* members from almost every country in the world. Standardization has come a long way and ISO International Standards, which now cover almost all aspects of technology and business, will continue to ensure positive change in an evolving world.

The first steps

Following the creation of the organization, 67 groups of experts (called technical committees) were set up in specific technical fields such as screw threads, marine technology, food, textiles, paints and laboratory equipment with a mandate to develop International Standards. This led to the birth, in 1951, of the first ISO standard (called “Recommendations” at the time) –ISO/R 1:1951, Standard reference temperature for industrial length measurements. Since then, the ISO portfolio has expanded to include over 22 000 standards supporting all the important technological, environmental and social changes that have taken place in the world.

“For 70 years, ISO has made standards that have shaped our history and accompanied the world’s greatest innovations. From the standardization of materials, components and equipment for the aerospace or automotive industry to the measurement of environmental pollutants, from establishing a management system to ensure food safety in the supply chain to creating guidelines for human-robot interaction, the need for international standardization has always evolved with the needs of industry and society,” says ISO President Dr. Zhang Xiaogang.

Expanding the community

ISO has worked hard over the years to broaden its circle of stakeholders, bringing different audiences to standardization, such as consumers or developing countries. The 1950s saw a number of new ISO member bodies join the organization from the developing world. To respond to these members’ needs, ISO set up in 1961 the ISO Committee for developing country matters (ISO/DEVCO), which helps them get the most out of standards development work. Today, three-quarters of ISO’s members are from developing countries.

Helping to improve the satisfaction and safety of consumers is another vital role of standards. Integrating their views in standards development is therefore essential because these real-life perspectives help ensure that issues such as safety and quality are adequately addressed. The importance of consumer leverage was endorsed by the creation, in 1978, of a Council Committee on Consumer Policy, now officially known as the ISO Committee on consumer policy (ISO/COPOLCO), to promote and encourage consumer interests in standards.

Effective and wide-reaching stakeholder engagement is essential in maintaining the relevance of International Standards. To ensure a strong relationship between standards and innovation, ISO has built collaborative ties with a network of global and regional organizations, including a partnership with the International Electrotechnical Commission (IEC) and the International Telecommunication Union (ITU), and has forged links with over 700 international organizations working in fields related to standardization. Furthermore, the contribution of large and small businesses, regulatory authorities and governments throughout the world is fundamental to the proper functioning of ISO.

Challenges for the future

“For the past 70 years, ISO has developed standards that drive industrial progress, promote global commerce and improve health, safety and the environment. But this is just the beginning,” says Dr. Zhang.

“Looking to the future, it is clear that our world faces many challenges that cut across national borders. Climate change, water scarcity, cyber security and large-scale migration are just some of the issues we face today that require integrated, international action.”

Many of these challenges have been included in the 17 Sustainable Development Goals (SDGs) adopted by the United Nations as part of their 2030 Agenda for Sustainable Development. Launched in 2015, the SDGs set ambitious targets for the next 15 years and will help concentrate international action to end poverty, protect the planet and ensure prosperity for all.

“The ISO community has many standards that can help organizations and companies address this agenda,” says Dr. Zhang. “We are ready to provide efficient tools to help the different communities worldwide face up to these challenges and shape a better world.”

The future of standardization is promising.

This article has been republished in full with permission from ISO.

The post ISO Celebrates 70 Years appeared first on The Auditor.


PDCA of Audit Evidence

by Cathy Fisher The process…

by Cathy Fisher

The process approach to auditing incorporates Dr. Edwards Deming’s well-known Plan Do Check Act cycle in defining audit scope and criteria. This structure also guides the gathering of evidence to support process-based auditing. The four types of audit evidence that support the process approach include:

  • Plan = Adequacy
  • Do = Conformance
  • Check = Effectiveness
  • Act = Improvement

Let’s consider the ISO 9001:2015 requirement for organizational knowledge: “The organization determines and maintains the knowledge necessary for the operation of its processes.” There are many ways for an organization to accomplish this requirement. For example, the leadership team of one organization may decide to implement a process for capturing lessons learned. This system-level decision encourages the development of a process within the organization to “determine and maintain” this information. The leadership team also decides that the chief information officer will be responsible for this process. From an auditing perspective, this is now a quality management system (QMS) process to audit.

The planning part of the audit evidence for this process may be gathered through interviewing the process owner (in this case, the CIO), and/or reviewing QMS documentation that describes this process. When we’re auditing the planning process, we’re looking for adequacy in its definition. We want to know the answers to such questions as, “What is this process?” “What is the desired output of this process?” and “Who is responsible for this process?”

Audit evidence that demonstrates adequacy for this example may include the following:

Type of Evidence Evidence for Lessons Learned
Recognized name of this process in the organization Organizational lessons learned, process identified in QMS map
Assignment of process ownership (either understood or designated through QMS roles and responsibilities) CIO, interview with leadership team, QMS process ownership matrix
Definition of the output of this process and its criteria/requirements, as well as how that output will be evaluated Searchable database containing lessons learned. Required fields for entry:  QMS process, date added, internal expert.
Identification of triggers or inputs that activate this process QMS processes generating lessons learned, including: investigations of customer complaints, product/service issues, corrective and/or preventive actions, process improvements, project reviews, management reviews, etc.
Description of steps involved in transforming the process inputs into the expected output Explanation of lessons learned process by CIO, procedure or training aid describing use of lessons learned database
Determination of resources needed to accomplish this process Tangible evidence such as computer database program, database administrator, input file mask, or computer server


Depending on the nature and complexity of the process being audited, there may be additional evidence that reflects the adequacy of the process.

Next, considering the “Do” in the PDCA process relates to audit evidence that demonstrates conformance. It’s easy to simply look at the execution of the process being audited as reflected in procedures and/or documentation or as described by the process owner. However, this stage of the process begins with the communication of the process and its requirements to those involved. This communication may be included in the “Plan” stage too.

When we think of auditing, conformance is what we typically mean:  Is the plan or process being followed?  Are we doing what we said we would do? The audit evidence of conformance can typically be found in three forms:

  • Tangible evidence: Procedures, records, computer programs
  • Observations: Auditor observing process execution
  • Admissions: Statements of fact by those performing the process. This may include explanation of the process by someone performing it or verification of interacting process as an audit trail.

Audit evidence of conformance also leads to recognition of supporting process audit trails; specifically processes that provide required resources, e.g., training/competency development of those executing the process, maintenance of equipment used in the process, control and identification of materials, availability and currency of process instructions, and control of work environment factors.  Auditing of these supporting processes is an essential part of applying the process approach.

Evidence of conformance for the lessons learned process may include:

Type of Evidence Evidence for Lessons Learned
Communication of plan Tangible: Procedure or training aid for entering lessons learned into database.
Understanding of plan Tangible: Training/briefing record of attendance.

Observation: Demonstration of lessons learned correctly entered into database.

Implementation of plan Observation: Process owners entering lessons learned into database.

Tangible: Contents of database Admission: Reference to database administrator for entry review and posting.

Supporting process audit trails Availability, access to, and back-up of computer database, document revision control of procedure, training of database administrator.

Simply confirming conformance to the plan isn’t sufficient in auditing from the process approach perspective. Evaluating the effectiveness of the plan is also essential in ISO 9001:2015, in which results are emphasized.  There are several prerequisites for auditing the “check” stage of the of process approach:

  • Criteria describing the expected/desired output from the process is clearly defined and can be evaluated (measurable either quantitatively or qualitatively).
  • Output from the process is being evaluated and compared to this criteria.

Evidence of effectiveness for the lessons learned process may include:

Type of Evidence Evidence for Lessons Learned
Criteria defined for evaluating effectiveness can be quantitative or qualitative in nature, objective, or based on perceptions. Number of lessons learned recorded, frequency of applying lessons learned to other processes, familiarity with database.
Process implemented for performing evaluation of process effectiveness, how often data is gathered, collected, and reviewed, and by whom. Number of lessons learned entered in database, which functions are entering them.
Results gathered and evaluated against process output criteria: checklists, check sheets, automatic data collection, trend charts, and surveys. Monthly activity report generated from database, database user satisfaction survey.

The effectiveness of any process is in the results. However, knowing how those results were achieved is also important for improvement.

The “act” phase of the process approach to auditing focuses on improvement.  From an auditing perspective, there are some prerequisites associated with auditing a process for improvement:

  • Threshold for action is established. This can include addressing acute issues of process ineffectiveness (a specific output nonconformity, such as a nonconforming product in a manufacturing process), or when a different output or output criteria is needed based on changes in customer or internal requirement.
  • Prioritization of processes for improvement as limited resources don’t allow for every process to be improved simultaneously.
  • Information about the process before and after actions taken for improvement is documented.

Quite a bit of evidence may exist for auditing an improvement process, including:

Type of Evidence Evidence for Lessons Learned
Recognition of improvement potential, action threshold, decision criteria Is the number of recorded lessons learned less than three per month?
Prioritizing processes for improvement, importance of process output, effect on organization of current process performance Lessons learned from all customer complaints investigated to be identified and included in database.
Process baseline, current process definition, current performance information available Ninety percent of lessons learned entered in quality department database.
Process applied for managing improvement, corrective action, kaizen. Focus workshop involving key process owners to identify and input recent lessons learned from all areas of the organization.
Results from improvement are compared with baseline performance measurements. Three months after workshop, all departments consistently entering an average of five lessons learned in database per month.
Management of change from improvement, update of documentation, possible retraining Examples and definition of lessons learned added to training aid and database instructions.

Keep in mind that when auditing improvement, not all stages of the improvement process may necessarily be complete at the time of the audit. This could initiate a follow up point for future audits. Additionally, not every improvement effort leads to a positive result. This isn’t a nonconformity, but rather an opportunity to look beyond the process being improved to also consider the process and/or methods being used for managing improvements.

To achieve a true process-based audit, questions should be generated during audit planning to evaluate a process’ adequacy, conformance, effectiveness, and improvement. Evidence must be gathered to support each of these evaluators in using the process approach to audit any QMS process.

About the author

Cathy Fisher is founder and president of Quistem LLC, which provides online and onsite management systems implementation, update, and assessment services for manufacturers and other industry sectors. Cathy has more than 30 years of respected auditing expertise, having led internal audit programs at many manufacturing organizations during her career. Cathy also has extensive experience conducting management systems registration audits, as well as establishing supplier evaluation and development programs.

She has held numerous auditor certifications including ASQ CQA, RAB-Certified Quality Systems Auditor, and ISO/TS 16949 IATF-recognized auditor. She has conducted internal and external audits that total more than 1,000 audit days and trained hundreds of management systems professionals as auditors. Cathy is passionate about the value auditing can bring to organizations and enjoys mentoring the next generation of technical professionals to develop their auditor excellence.



The post PDCA of Audit Evidence appeared first on The Auditor.