+ Sidebar

UN Report Examines Credibility of Management Systems Certification

Screen-Shot-2017-03-14-at-4.10.47-PM.png
While market surveillance is a…

While market surveillance is a well-known concept in a regulatory context, a similar approach has been shown to be effective when applied to the voluntary certification of quality management systems, according to a recent report by the United Nations Industrial Development Organization.

In a series of articles, The Auditor Online will share highlights from the “Good Practices: Experience in the Market Surveillance of ISO 9001 Quality Management Systems” report, which presents the case for accreditation bodies or stakeholders completing market surveillance visits to certified organizations to ensure the quality of their products or services.

According to the report, user feedback on the performance of ISO 9001-certified suppliers and the accredited certification process reveals ongoing debate around the effectiveness and credibility of accredited certification.

The debate centers on whether organizations are deriving tangible benefits through ISO 9001 certification, if the certification process is being performed effectively, and whether ISO 9001-certified suppliers can be relied upon to provide “consistent, conforming products and services” to customers.

Systematic feedback was gathered between 2009 and 2015 from purchases regarding their perceptions of ISO 9001-certified suppliers, and from ISO 9001-certified organizations about the implementation and certification process. Feedback was gathered through a survey and a market surveillance activity which involved one-day visits to certified organizations. These visits aimed to determine confidence levels related to various aspects of the organization’s quality management system and the overall level of confidence in the certification process.

To reflect their confidence level in organizations being examined, assessors assigned the following grades:

  • Grade One: “Little or no confidence.” Little or no evidence to support the implementation of this topic.
  • Grade Two: “Some evidence presented, but not at all convincing.” Some evidence was presented, but in the professional judgement of the assessor, there would probably be evidence to support a nonconformity if a detailed audit trail were to be followed in a full system audit.
  • Grade Three: “OK—No reason to doubt that this is being addressed correctly.” The “default” grade, where there is no evidence to suggest reasons for concern, based on the assessor’s experience and professional judgement.
  • Grade Four: “Clear evidence that this is being done, and meets the intent of the relevant standard.” Sufficient objective evidence provided a high degree of confidence that the organization meets requirements.
  • Grade Five: “We can be proud to use this organization as a benchmark for this topic.” To be reserved for truly excellent performance.

A key finding was that there were notable differences in the performance and level of confidence in organizations certified by different certification bodies and under accreditation from different accreditation bodies. Other findings included:

  • More than 95 percent of certified organizations surveyed considered the effective implementation and accredited certification of quality management systems to have been a “good” or “very good” investment.
  • Overall perception of the ISO 9001 standard and accredited certification was good, although the role of accreditation wasn’t well understood by purchases or certified organizations.
  • The purchasers surveyed were mainly satisfied with the performance of their ISO 9001-certified suppliers. In general, ISO 9001-certified suppliers performed “better” or “much better” than non-certified suppliers. However, poor responsiveness of certified organizations to customer complaints was one area of concern.

Overall, the performance of the certified organizations that were visited was good, which demonstrates the effectiveness of the accredited certification process. However, between six and eight percent of organizations returned unsatisfactory results, and one percent raised serious doubts about the validity of their certification—which calls into question the effectiveness of the certification process.

According to the report, market surveillance activities can be used in place of traditional accreditation methodologies such as office assessments and witnessed audits to investigate complaints or concerns.

Market surveillance can also uncover trends, strengths, and weaknesses in the implementation of quality management systems in particular regions or sectors—all of which is useful information for certification bodies to focus their attention on during surveillance and renewal audits.

To give the data context, the report describes independent third-party certification as a common method sought by organizations to demonstrate that they have met all the requirements of ISO 9001. The certification consists of a certificate of conformity and ongoing surveillance to ensure that the system is maintained in accordance with the standard.

To demonstrate competence to administer management system certification, a certification body may become accredited by an accreditation body. This accreditation is based on the requirements defined in ISO/IEC 17021-1 and is supplemented by other discipline-specific or sector-specific requirements in some cases.

Continue to read The Auditor Online in the coming weeks for more findings from the “Good Practices: Experience in the Market Surveillance of ISO 9001 Quality Management Systems” report.

The post UN Report Examines Credibility of Management Systems Certification appeared first on The Auditor.

Auditing the Tapestry of ISO 9001:2015 Requirements

fisher-fig-1.png
Many new requirements in ISO…

Many new requirements in ISO 9001:2015 challenge auditors to look beyond typically prescribed audit evidence and explore the interconnected nature of an organization’s quality management system (QMS) processes. In addition, these new ISO 9001:2015 requirements can be considered from multiple levels of an organization. Figure 1 shows that there are five levels an organization can directly control.

 

The ISO 9001:2015 requirements can be viewed from this business ladder structure relative to defining and auditing the QMS activities. The business itself weaves the horizontal layers in a kind of tapestry. As an example, let’s consider ISO 9001:2015 requirement section 4.1 Understanding the organization and its context, which requires organizations to “Determine external and internal issues relevant to its purpose and its strategic direction.”

At the system level, this requirement can be considered from the standpoint of initially defining the QMS and its activities. For example, in defining the scope of an organization’s QMS, consideration is given to such external and internal issues, (illustrated in ISO 9001:2015 section 4.1). The external issues could include markets served and products and services offered. From an internal perspective, the organization’s technical competencies, available equipment, and location, etc., affect the definition of the organization’s QMS scope. Auditors look to the scope of an organization’s QMS to determine what technical expertise is required in auditing that organization’s QMS, as well as in anticipating the types of processes that would be included.

ISO 9001:2015 requirements can also be viewed from a strategic perspective, where the influence changes to the QMS at the system level. Looking at the “Context of the Organization” requirement again, the organization’s leadership would consider external and internal issues when formulating its business plan, whether this is long term (more than five) or near term (one to three years).

Certainly, changes in external and internal issues (refer to ISO 9001:2015 subsection 9.3.2, Management Review Inputs), could shape an organization’s strategic direction going forward. For instance, if new technology alternatives are quickly absorbing market share, the organization would need to consider whether to adopt such new technology or identify an alternative business strategy that may in turn change the scope of the organization’s QMS. Likewise, organizations faced with an aging work force internally need to consider succession planning and retention of organizational knowledge (refer to ISO 9001:2015 subsection 7.1.6) as part of their strategic direction. Auditors will find evidence of consideration for these changing external and internal issues in strategic/business planning discussions, management review results, and even annual reports for publicly traded companies.

For the Planning level of the business ladder, specific customer requirements may also drive external and internal issues that affect the organization’s QMS. If a customer were to request a new product that require the organization to perform extensive development activity, feasibility and risk consideration would highlight potential internal issues. This could include limitations in existing resources (refer to ISO 9001:2015 subsection 7.1.1) and external issues, such as availability of technology and employee competency to support the organization’s development process. Thus, connections between context of the organization, (which is discussed in ISO 9001:2015 section 4.1), actions to address risks and opportunities, (ISO 9001:2015 section 6.1), and operational planning and control (ISO 9001:2015 section 8.1), are recognized. Audit evidence at this level may include results from gathering customer and other project requirements and review of these requirements relative to the organization’s capabilities.

Considering external and internal issues from the process level of the business ladder, organizations encounter daily issues in their ability to consistently control their processes needed for producing products and/or delivering services (refer to ISO 9001:2015 subsection 8.5.1). Daily internal issues could include  attendance of the work force—especially during cold and flu season, unexpected equipment breakdowns, or even a workplace fire or other disaster. External issues that affect an organization’s operations could include availability of material/inputs, especially if there is a supply shortage or disruptions in utility services such as electricity.  Auditors can look for evidence of how organizations identify these issues,  (ISO 9001:2015 section 6.1), communicate them (ISO 9001:2015 section 7.4), and act upon them (ISO 9001:2015 subsection 8.5.1).This evidence might be found in production schedules, daily operations meetings, or shift changeover activities.

Finally, external and internal issues can exist at the the product level of the business ladder, as well. For example, external issues could include near-term changes in customer demand affecting operating schedules and inventory levels. Recalls or other field events of competitors could also affect the sales of an organization’s product. Internal issues at the product level could include processing errors that generate a large quantity of nonconforming outputs (ISO 9001:2015 section 8.7), or inventory inaccuracies that affect availability of product for shipment to customers. These events are excellent triggers for selecting appropriate audit samples to evaluate the effectiveness of an organization’s QMS in consistently fulfilling customer requirements.

To adequately define and audit an organization’s QMS, the application of each ISO 9001:2015 requirement at these different business ladder levels should be considered. In doing so, the interaction of ISO 9001:2015’s requirements—and therefore an organization’s QMS processes—becomes apparent.

In addition, each of these processes along the business ladder that address ISO 9001:2015 requirements should also be viewed from the Plan-Do-Check-Act perspective. This is the “wrap” that holds the tapestry of the organization’s business ladder together.

Rather than viewing ISO 9001:2015’s requirements linearly or only relative to one level of the business, auditors can assist organizations in recognizing the tapestry these requirements weave for defining, implementing, maintaining, and improving their QMS which optimally should function at all levels of the business.

Challenge: Select any ISO 9001:2015 requirement. Identify what processes in your organization’s QMS relate to that requirement at each of the business ladder levels. Also consider what audit evidence would be available in these processes to support the selected ISO 9001:2015 requirement. Remember  audit evidence can be tangible, observation, or statement of fact. Share your ideas in the comments below.

The post Auditing the Tapestry of ISO 9001:2015 Requirements appeared first on The Auditor.

Report Reveals Experience of ISO 9001 Market Surveillance

A new report from the…

A new report from the United Nations Industrial Development Organization presents good practices in applying market surveillance methodology to monitor the effectiveness of ISO 9001 certification in manufacturing enterprises and evaluate the performance of respective accredited certification bodies.

“Good practices: Experience in the Market Surveillance of ISO 9001 quality management systems” concludes that the proper use of ISO 9001–based quality management systems assists developing countries in promoting sustainable trade—thereby helping them achieve inclusive and sustainable industrial development and the 2030 development agenda.

The case studies mentioned in the report show that market surveillance methodology can be used or adapted as an effective tool in other regions and contexts—and for other management systems.
Click here to view a full copy of the report.

The post Report Reveals Experience of ISO 9001 Market Surveillance appeared first on The Auditor.

Auditing ISO 9001:2015 Without Documents

One of the most frequent…

One of the most frequent concerns raised by auditors about ISO 9001:2015 is how to audit a quality management system (QMS) that has little or no documentation. ISO 9001:2015 doesn’t include specific requirements for documented procedures and doesn’t require a quality manual. However, it does require “documented information” related to a number of requirements. Several of the new requirements: context of the organization (clause 4.1), actions to address risks and opportunities (clause 6.1), and organizational knowledge (subclause 7.1.6), have no such reference. So how can these “documentless” processes be audited?

ISO 9001:2015 defines an audit as a “systematic, independent and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled.” The standard defines audit criteria as a “set of policies, procedures or requirements used as a reference against which objective evidence is compared.” Finally, ISO 9001:2015 defines audit evidence as “records, statements of fact or other information, which are relevant to the audit criteria and verifiable.”

It may appear from these definitions that audit evidence and audit criteria must be documented. However, the key questions that come to mind when limited or no QMS documentation is available are:

  • How are audit criteria established?
  • What audit evidence is available to evaluate conformance?

The answer to these questions is found in understanding the QMS from the process approach and applying essential auditing skills.

Consider that every activity within an organization is a process that—by definition—takes inputs and converts them to an output typically of greater value through defined steps. Thus, the basic audit criteria for any process can then be derived through a set of process questions:

  • What is the desired output?
  • What input triggers action toward the desired output?
  • What steps are taken to transform the input to the output?

Every process must have a process owner who’s responsible for managing the process and its related outputs. Specifically, a process owner is responsible for:

  • Clearly identifying process output requirements
  • Determining process interfaces, including input triggers
  • Defining how the process is to be executed (process sequence and actions)
  • Establishing process performance goals
  • Evaluating potential process risks in achieving output requirements and process performance goals
  • Determining appropriate process and output controls
  • Identifying, obtaining, qualifying, and maintaining process resources
  • Monitoring ongoing process performance (process execution and outputs, both internal and external)
  • Changing/improving the process as necessary

Recognizing the process owner’s role makes it clear that audit criteria can be determined by interviewing the process owner. The process owner’s responses to the questions then become the basis for gathering the objective evidence to verify conformance to the stated audit criteria. This approach requires auditors to exercise several critical auditing skills:

  • Initiating the audit by interviewing the process owner to establish the audit criteria. This will challenge auditors to carefully listen to the process owner’s responses to audit questions and quickly organize this information into a process framework.
  • Be able to quickly develop open-ended audit questions based on the process owner’s response to the questions and gather relevant audit evidence from personnel working in the QMS process, including the process owner. This technique for gathering objective evidence is often referred to as corroboration.
  • Be capable of synthesizing auditee responses to determine alignment with audit criteria as described by the process owner and recognize relevant audit trails for exploring the sequence and interaction of QMS processes.

While this approach to auditing certainly depends heavily on auditors’ listening skills and ability to organize information, it also offers greater flexibility in the depth of questioning that can be pursued during an audit. The auditor is no longer limited to questions related to whatever is stated in QMS documentation.

This does mean a bit more work for the auditor—especially during the audit—and perhaps auditees will be nervous not having a script to follow when responding to auditors’ questions. However, the potential for exploring potential risks and opportunities related to QMS processes is much greater. These benefits will increase the value of audits and the information they can provide to process owners and the organization’s leadership in better utilizing their QMS for increased customer satisfaction and improved business performance.

About the author

Cathy Fisher is founder and president of Quistem LLC, which provides online and onsite management systems implementation, update, and assessment services for manufacturers and other industry sectors. Cathy has more than 30 years of respected auditing expertise, having led internal audit programs at many manufacturing organizations during her career. Cathy also has extensive experience conducting management system registration audits, as well as establishing supplier evaluation and development programs.

She has held numerous auditor certifications including ASQ CQA, RAB-Certified Quality Systems Auditor, and ISO/TS 16949 IATF-recognized auditor. She has conducted internal and external audits that total more than 1,000 audit days and trained hundreds of management systems professionals as auditors. Cathy is passionate about the value auditing can bring to organizations and enjoys mentoring the next generation of technical professionals to develop their auditor excellence.

The post Auditing ISO 9001:2015 Without Documents appeared first on The Auditor.

Guidance for SMEs Using ISO 9001 for Quality Management Just Released

ISO’s essential guide for small…

ISO’s essential guide for small and medium enterprises (SME) wishing to implement a quality management system (QMS) has just been updated, providing practical advice and concrete examples tailored specifically for small businesses.

ISO 9001:2015 for Small Enterprises  What to do? has just been updated to align with the newly revised version of one of ISO’s most popular standards, ISO 9001, Quality management systems – Requirements, updated in 2015.

The handbook was written by a group of experts from ISO/TC 176/SC 2, the technical subcommittee that developed ISO 9001:2015, and features useful information on everything from how to get started right through to guidance for those who choose to seek certification. It includes practical advice on the different ways of approaching a QMS as well as detailed guidance on each element of ISO 9001:2015.

Click here to preview the handbook.

Nigel Croft, Chairman of ISO/TC 176/SC 2, said: “This handbook recognizes that small businesses have different needs and challenges compared to large organizations, with different ways of working and often with limited resources. This handbook offers tailored advice to help them implement a quality management system that can truly be useful, and help them to improve their overall business performance.”

“It includes a step-by-step guide to implementing a QMS, providing sector-specific examples for different types of small businesses, such as consultancies, manufacturers, and distributors.”

ISO 9001:2015 for Small Enterprises  What to do? also provides a clear explanation of what a QMS is and how it can help organizations improve the quality of the work they do and the products and services they deliver, thereby improving the confidence of their customers and other stakeholders.

ISO 9001 is one of the world’s most widely used QMS standards, with over one million organizations certified to it in over 170 countries around the world.

ISO 9001:2015 and ISO 9001:2015 for Small Businesses  What to do? are available for purchase from your national ISO member and the ISO Store.

This article has been republished in full with permission from ISO.

The post Guidance for SMEs Using ISO 9001 for Quality Management Just Released appeared first on The Auditor.