CONDUCTING QUALITY AUDITS IN OFFSHORE MOTOR COMPONENT FACTORIES

by Michael Bloom
Chairman, Sinotech, Inc.

Presented at: Electrical Insulation Conference and Electrical Coil Winding Technology Conference September 24th 2003

Abstract: Many motor manufacturers are going offshore to purchase or manufacture motor components. These include commutators, brushes, armatures, magnets, laminations, etc. Offshore factories vary considerably in the robustness of their process capabilities and quality systems and visitors are at a clear disadvantage in attempting to audit in an alien environment. This is complicated by cultural and language issues.

This paper presents a practical hands-on approach to conducting the audit beginning with the first handshakes through parting words. Sinotech has over 12 years experience in performing these on-site audits and shares its experience starting with preparation for the audit. Topics covered include the pre-audit on-site meeting, how to select participants, how to determine the best order for moving from process to process, how to analyze the robustness of each process, and how to avoid being fooled by undocumented statements.

Key words: Chinese motor components factory audit

Introduction

Whether you are purchasing commutators or magnets, shafts or laminations, windings or brushes, nothing will cause a manufacturing project to fail more quickly than choosing the wrong factory.

As a global sourcing partner to many motor manufacturers, we have developed methods that assure timely delivery of quality motor parts. This paper provides the Sourcing Engineer, Purchasing Manager, and Materials Manager the skills to do an on-site audit. Since China is currently the pre-eminent offshore source of these components the examples provided in this paper will refer to China. However, the strategy presented is applicable to any country with comparable facilities.

Basics of Auditing

Why an on-site audit? To be blunt, offshore factories are often less forthcoming about its limitations than domestic facilities. In some cultures, such as those of the Pacific Rim, the incentive to "save face" is pervasive. Managers are loath to admit shortcomings and promotional literature is rife with exaggeration. Based on our nearly 15 years of experience the only way to assure the selection of a qualified factory is though the performance of an on-site audit.

Who should perform the audit? Ideally, the auditor should be a Supplier Quality Engineer (SQE) with auditing experience, product knowledge and experience in relevant manufacturing processes. Hands-on experience in a manufacturing plant making the exact same parts would be icing on the cake. In practice, the best available auditor may be inexperienced in auditing and have little experience with related manufacturing processes. Although, almost any fairly technical person can learn to audit a factory, the closer to the ideal the better. Sometimes an auditing team can be assembled so that there are two or more observers present with complementary skills.

When to audit the factory? Plan on spending between � and 1 full day auditing the factory. Arrive early so you can extend the day if more time is needed. Be sure to audit the factory on a day where production is at a maximum. Ask in advance what is the best day to see products made that are as close as possible to the products you wish the factory to produce.

Selecting the Auditing Standard

Audit to QS-9000 Standards. Factories often claim quality certifications and may indeed possess certificates indicating that they have been granted ISO-9000 or QS-9000 status. It has been our experience that the issuance of an ISO-9000 certificate proves little more than that the quality system has been documented. Although QS-9000 is directed towards products in the automotive industry we have found that auditing to QS-9000 standards is the best determinant of quality. Therefore, we audit all factories to this standard. It should be noted that not all projects require the stringent QS-9000 quality level, but evaluating against this level will provide a yardstick that can be used to analyze tradeoffs between quality, size of project, and cost.

Before the Audit

Preparing for the audit. It is assumed that before the date of the audit the factory has submitted a quotation with sufficient factory capability and competitive pricing to stir interest in auditing the factory in the first place. Make sure that an interpreter is present who does not have a stake in the factory and who will not be tempted to enter the conversation as a participant. The factory should be informed that it will need to make available the factory manager, engineering director, director of quality control, and the laboratory manager. Explain that documents will be requested and that you will expect to see actual documents. Ask permission to take photographs of the factory and its manufacturing processes. Carte-blanche permission may be granted, it may be denied completely, or permission may be granted to photograph only those processes that are not proprietary.

The Auditing Day

The Flow of the Day. We recommend a particular logical flow to the day's activities. Here is the recommended 10-step program order:

Introductions by factory leadership
Introductions by auditor or auditing team
Request for production of documents
Analysis of documents produced
Visit incoming raw material testing and storage
Visit tooling workshops, tool storage and design areas
Visit all manufacturing processes in order of the Control Plan (see below)
Visit final production test area and storage areas
Visit laboratories
Report results and say farewell

Introductions by factory leadership. The factory wants to put its best foot forward and this is the factory's opportunity to tell you why you and the factory should become "partners". In China, this is often accomplished around a large oval table with an omnipresent tea service. Your cup will be re-filled often. The highest level manager will usually start with an historical review of the factory's beginnings and progress. Capabilities will be described and key customers may be named. Listen carefully to the customer list for two very different reasons. First, are they major international companies who are known to demand high quality? Second, if this kind of information is revealed easily it may indicate that the factory does not honor non-disclosure agreements since such agreements typically hold that even the existence of business between the factory and its customer is a trade secret.

In this introduction expect to hear about the number of workers as well as the factories land and building size. If not more clearly defined you may wish to ask about the number of engineers, managers, and technicians instead of just the total workforce. Listen for indications of the factory ownership. In general, government owned enterprises are of lower quality, with poorer management, and lack an entrepreneurial spirit. Many of the best factories are now privately owned. In general you will not be visiting many Joint-Venture factories, unless your company is part of the J-V, as most of these do not market outside the J-V.

Many of the factory introductions are part informational and part ritual. Enjoy the ritual as experience shows that nerves may get frayed as the auditing becomes more aggressive later in the day.
Introductions by auditor or auditing team. Introduce the individuals on the auditing team and explain their roles within your company. Take time and provide a clear view of your company, its goals, and its position in the marketplace. Although this is the occasion of an audit it is also a valuable opportunity to sell your company. The more the factory views your company as a potential strategic partner the more cooperation you will receive. Be very clear that this is a crucial day in the business between the companies. Explain the 10-step process you will be following and reassure those present that you will provide full and honest feedback at the end of the day. While a thorough audit can be nerve-wracking for the factory, it demonstrates that your company is serious about doing business.
Request for production of documents. Before you get a first glance of the factory floor a document analysis will tell you 90% of what you need to know. It is very tempting to rush out to the factory floor as soon as possible. Resist the temptation. There is some disagreement about whether it is better to provide a written list of required documents in advance or if the list should be presented without prior notice. We prefer the later as it is often informative to see how readily the factory can access documents. The following documents should be requested for a part that is as close in design to yours and in quantities that are similar.
1. Quality Certifications
2. Drawing of a similar part
3. Process Flow Chart
4. Control Plan
5. Dimensional Layout
6. PFMEA (Production Failure Mode Engineering Analysis)
7. Capability Study
8. Gage R&R (Gage Reliability and Repeatability study)
9. Material Certification/Material Tests
1. Quality Certifications. These are documents with official-looking stamps and seals representing that the factory has met internationally published standards.
2. Drawings. Ask to see the original drawings and any drawings made subsequently by the factory.
3. Process Flow Chart. This is a relatively simple graphic representation of the process steps. We recommend that a Process Flow Chart be requested at time of quotation as it provides at least a minimal indication that the factory has thought through the manufacturing requirements.
4. Control Plan. This is a critical document and will provide much of the basis for the visit to the factory floor. In essence it is a cookbook with a recipe involving every process step. Each step is clearly numbered in sequence. The process for each step is described along with the equipment to be used. If in-process inspections are to be done they will be listed here. One reason this document is so critical is that once you reach the factory floor you are going to "walk the Control Plan" from one process to the next.
5. Dimensional Layout. This is a chart that reports the measured vs. nominal dimensions for parts that have been produced. For example a part may have dozens of measurable parameters. Typically from 3 to 10 such parts will be measured.
6. Production FMEA (Failure Mode Engineering Analysis). This complex-sounding document asks the question "In production what happens when something goes wrong?". For every production step (the ones numbered in the Control Plan) it asks: "How severe is the failure to the operation of the product? How detectable is the failure? And how often is it likely to occur?" Each of these is scored from 1-10. A problem with a high severity, but that is simple to detect, and happens very infrequently may not be a problem. A score called the RPN is calculated that equals Severity X Detectability X Occurrence. It can be as low as 1 or as high as 1000.
7. Capability Study. Is the factory truly capable of consistently manufacturing to the drawing specifications while running at full production speed? A capability study provides statistical verification of this question. Contrary to popular belief a capability study cannot be done just by making x number of parts and testing all of them. Capability studies should be performed during multiple work shifts while the production line is running at full speed. Random samplings should be taken every nth part and measurements taken. This will result in a large number of measured samples (typically in the low hundreds). A statistical analysis is then performed resulting in a Cpk value.
8. Gage R&R (Gage Reliability and Repeatability study). Are the measuring instruments and the technicians who use them capable of producing results that are both reliable and repeatable. The Gage R&R provides statistical verification of this. For each instrument (i.e. a micrometer) several technicians are asked to measure several dimensions on a reference part. The statistical result therefore reports on both the instrument and the operators. Note that it does not report on the calibration of the instrument and is therefore not indicative of absolute accuracy.
9. Material Certifications/Material Tests. A material certificate is a document, provided by the raw material supplier, stating that the material meets a stated specification. Material tests are actual laboratory results that may include such tests as chemical composition, metallurgy, tensile tests, magnetic tests etc.
Analysis of Documents Produced. Not all documents may be produced for your inspection by the factory. The documents that are produced and that are not produced will give you a good first indication of the factory's strengths and weaknesses. The best factories will be able to produce all documents. Even fairly good factories may not be familiar with FMEAs, Capability Studies, and Gage R&Rs. Now it's time to examine each document carefully. If the documents are in another language you may need some interpretation.
1. Analysis: Quality Certifications. As stated earlier ISO-9000 certificates provide limited evidence of quality. However, some certifying organizations are more credible than others. For example SGS or TUV certificates provided by technically advanced countries are more credible than those issued by Chinese domestic organizations. QS-9000 certificates from credible certifiers are often a good indication of quality. In any case check on the issuance date of the certifications, see if they are still valid and ask for proof of ongoing diligence by both the factory and the certifying organization. Factories are monitored periodically by these organizations and there are typically areas of improvement noted. It is fair to ask exactly what were the results of the last monitoring visit and what steps have been taken to improve processes.
2. Analysis: Drawings. Each parameter in the drawing should be numbered corresponding to entries in the Dimensional Layout. This is also an opportune time to ask about the CAD capabilities of the company and to remind the factory that you would like to see its design, data storage and archiving facility later in your visit.
3. Analysis: Process Flow Chart. One key question to ask concerning the Process Flow Chart is which, if any, of the processes will be conducted outside the factory at another factory or at a 3rd party facility. This can be crucial information as it may indicate the need for additional audits and it may point out potential hot spots for quality issues.
4. Analysis: Control Plan. The Control Plan must be a complete and accurate representation of all the processes needed to make a part. If you are already familiar with the manufacturing processes required to make a part then step through the Control Plan in your mind and see if it makes sense! Hold onto it because you are going to need it in the analysis of the PFEMA.
5. Analysis: Dimensional Layout. It is amazing how many dimensional layouts contain failing data and actually show that a part is failing. Yet the box marked "pass" is checked even though the parameter failed. Look carefully at the data and if tolerances are given without actually stating the upper and lower limits compute them yourself. Then spot check the data against the limits. If you find a discrepancy confront the Quality Manager. Also, check the drawing against the dimensional layout to see if all parameters have been measured. For example, a factory without an optical comparator or projector may simply skip some hard-to-measure angular parameters. A well executed dimensional layout will also list the type of instrument used for each measurement thus making it easier for a 3rd party laboratory or customer to verify measurements using the same techniques and setup.
6. Analysis: Production FMEA (Failure Mode Engineering Analysis). Compare the Control Plan to the PFMEA. Every process step listed in the Control Plan should have a corresponding entry in the PFMEA. Well done FMEAs match the item numbers between the documents. For some companies RPN scores below 40 are acceptable while for others scores below 70 are acceptable. Whatever the definition of an "acceptable" RPN, scores above the cutoff must be addressed with a plan of corrective action. An acceptable RPN score must result once the plan is implemented. The determination of the Severity, Detectability, and Occurrence score is somewhat subjective so you may wish to choose a few of the scorings and ask the Quality Manager to explain why a particular score was given.
7. Analysis: Capability Study. Observe the conditions under which the study was performed. Due to lack of understanding of the purpose of the study many factories do not sample parts at full production speed on multiple shifts. This is understandable since running such a large volume implies a substantial risk of loss should the capability study fail. Also, the factory may require a production order in advance prior to running a capability study in order to assure the salability of the product produced during the course of the study.
  The resulting Cpk data are measures of the stability of the process. All measured parts may be within specification but fail due to a low Cpk value. Ideally the distribution of measurements follows a bell-shaped curve yielding an acceptable Cpk; but if the data is skewed then the Cpk is unacceptably low and the process is susceptible to producing parts out of tolerance. As a rule of thumb a Cpk greater than 1.33 is acceptable and a Cpk greater that 1.67 is considered exceptional.
8. Analysis: Gage R&R (Gage Reliability and Repeatability study). The analysis of a Gage R&R is beyond the scope of this paper; however the presence of Gage R&Rs is an indication that attention has been given to quality.
9. Analysis: Material Certifications/Material Tests. Look closely at the Material Tests. Were they performed by the factory's own laboratory? Does the factory have the equipment necessary to do all the tests? If not, where were the parts tested? Material test documents should include the nominal parameters as well as the measured data. As in the case of dimensional layouts, is the reporting of the result consistent with the actual result?
Some factories may not wish you to take these documents with you when you leave the factory. This should be no problem. Be sure to take copious notes since details are likely to become a blur later. By now everyone is glad to leave the meeting room to start the factory tour. You are ready to "walk the Control Plan" but you should already have a good idea of what you are about to see.
Visit incoming raw material testing and storage. One immediate indicator of factory quality is the management of the raw material storage. Is the area secure against the elements? Materials should be grouped in logical order and clearly labeled with lot numbers, dates, quantities etc. Ideally, the factory should be able to trace any manufactured part back to its raw materials. This is rarely accomplished even in the best factories we've visited. Is there a system for First-In-First-Out material usage (FIFO)? Is the storage area used for anything other than raw materials?
Incoming inspection of raw materials should be evident. Ask to see records of past inspections. How are defective raw materials handled? Are they clearly labeled and identified (i.e. colored bins) and are they quarantined.
Visit tooling workshops, tool storage and design areas. If a factory does not do its own tooling that should not necessarily be considered a drawback. Some of the best factories only tool simpler projects and contract out more complex projects to professional tool-making workshops. Many tooling workshops are no more than machining workshops dedicated to tooling. High quality tooling workshops often have advanced equipment such as CNC lathes, wire cutting machines, EDM, etc.

How are tools stored? Custom tooling such as molds and fixtures are one of the factory's major assets. How do they protect that asset? Tools should be kept clean, well lubricated, separated from each other to prevent damage, and may need to be kept in an air-conditioned space with temperature and humidity control. The tooling is often Customer-Owned-Tooling so extra care should be taken with regard to security.

Does the company have a dedicated design area? Is it old fashioned with paper and pen in use or is it computer automated? What CAD systems are in use? What design file formats can the factory accept? How conversant are the designers with Geometric Dimensioned Tolerances? If files are stored on paper, how are the paper files protected? If the files are stored electronically, how are they safely backed up and archived?
Visit all manufacturing processes in order of the Control Plan. This is where "walking the Control Plan" really begins. Self-discipline is required to stick to the Control Plan. Physical stamina may help as well because the location of manufacturing processes within the factory may cause you to run from one end to the other and back in order to follow the Control Plan.
Regardless of whether the factory is producing motor shafts, commutators, brushes, wound armatures, or bearings every factory consists of nothing more than processes laid end-to-end. Therefore, each process has a number of attributes that are common to manufacturing processes in general.

Here are some attributes to look for:

Posting of work instructions. A laminated worksheet should be present at every station. The instructions should be specific to the part being made.

Quality of Machinery. Equipment should be of sufficient capability to perform the operation and should show documentary evidence of maintenance. Generally speaking imported capital equipment from the U.S., Japan, Germany, etc is of better quality than domestic Chinese equivalents. Gauging should be appropriate. For example, if the process requires an air pressure of 4 psi and the air pressure gage scale is from 0 - 400 psi an indication as low as 4psi will be inherently inaccurate.

Quantity/Availability of Equipment. Discuss the throughput of the various processes with the Factory Manager. Every process chain has some bottleneck. Try to locate the bottleneck (i.e. the process with the slowest throughput). Is this bottleneck readily overcome? If the bottleneck is a shortage of simple grinding machines breaking the bottleneck may be easy. If the shortage involves expensive CNC equipment expanding throughput may be nearly impossible.

How about availability? What percentage of capacity is already committed to other customers? How many shifts are presently working? How difficult would it be to recruit and train additional workers to expand production or work another shift? Beware of promises as some factory managers try to overbook current resources in the hope they can bootstrap the factory with the new orders.

Use of Fixturing. Repetitive processes with tight tolerances typically require fixturing in order to clamp or position the work in progress. Observe the operator and see if the fixturing prevents unintended mis-positioning.

Measuring Tools. At many if not all stations measuring tools will be present to perform in-process testing. Whether the testing is 100% or sampled all instruments must be calibrated and demonstrably so. Every calibrated tool must have a unique ID number permanently marked on it. Pick up any instrument, walk to any scale and ask to see the calibration certificate for that particular instrument. Read the certificate. Who issued it? There are many 3rd party companies that go from factory to factory calibrating instruments. National metrology laboratories maintain primary standards from which secondary standards are calibrated. And so begins a long line of calibration that is traceable to the primary standard. Every calibration certificate should indicate its traceability to that primary or secondary standard.

Materials Handling. The way that materials are handled as they are inputted to the process, processed, or outputted to the next process can affect the quality of the product. For example, in one commutator factory a worker with gloves was seen loading parts into a cleaning process. The cleaned commutators were than removed from the cleaning bath by a bare-handed operator!

Material Transport. Close coordination between process flow and material transport increases production efficiency and reduces the likelihood of damaging parts during transit between processes. Are the processes modular so that materials can be efficiently routed?

Handling of Non-Compliant Material. Flaws in the manufacturing process may lead to the production of non-compliant material. It may be economically desirable for the factory to rework these parts and return them to the manufacturing process flow. Some manufacturing contracts specifically prohibit this. But if not prohibited, there should be a well-designed system for identifying, labeling, quarantining and documenting the failed parts and their rework. Such parts may be subject to additional non-standard testing.

Importantly, actually watch the process being performed, do not settle for just having the process described to you. You are likely to spot an anomaly that cannot be detected any other way. It may be that the operator varies the way each part is subject to the process. Maybe the operator perspires into a critical area. It is very difficult to predict exactly what you will find, but the odds are that you will find some interesting areas for improvement.
Visit final production test area and storage areas. This is the back-end of the process chain. One interesting observation that can be made here is how the factory addresses the question of statistical testing versus 100% product testing. Some factories boast that they test every part leaving the production line. The case can readily be made however that with design for reliability and good process controls that 100% testing should never be needed and is, in fact, wasteful of resources. Here, as in in-process inspections, the identity of the inspector should be documented.
Quality manufacturers are rightfully proud of their low defect rates and often post them publicly. Such postings show a clear intent to maintain high quality levels. Finally, inspect the finished goods storage area. Check the export packaging to see if it is robust and well organized. Again look for evidence of FIFO handling and clear lot numbering.
Visit laboratories. This is one of the most enjoyable parts of the on-site audit since it tends to highlight the most advanced technology that the factory has to offer. Be sure to look for instrumentation that is appropriate to the products being tested. So, if testing commutators look for spin testing equipment, hi-pot testers, and pull testers. If testing motor brushes look for chemistry labs to test the carbon mixture and life-testing apparatus. If testing motor shafts, look for hardness testers, spectrophotometers for chemical analysis, tensile test equipment, roughness and roundness testers. For castings that are geometrically complex look for CMM testing equipment. See if there is a calibration laboratory on premises. Is there equipment for environmental testing such as temperature cycling and salt spray? Of course check to see that laboratory equipment is well maintained and calibrated.
Report results and say farewell. Now is the time for all stakeholders to return to the conference table, break out the tea and heave a sigh of relief. The auditing process is grueling for everyone. By now you have formed a very specific lists of the positives and negatives of the factory. You may even have decided that you "can't wait to start a project here" or "heck, this place is awful." Still, the factory has extended its courtesy to you and has probably been a gracious host. In return you should honestly report your findings in the context of trying to assist the factory in improving its quality. Factory managers know that certain deficiencies exist in the factory and are usually quite anxious to have your suggestions.
Begin with the positives. Take as much time with the positives as you intend to spend on the negatives. Then move honestly but constructively on to the negatives. For every negative try to provide some practical advice. In practice most of the advice you give will be relatively simple to implement. At the end of your report the factory manager may take you aside to see if you really intend to do business. If you intend to say so! If not, explain that if the factory improves in the areas discussed you will be more than happy to re-consider the factory having already established the relationship and having strengthened it on that day.

Michael Bloom received his BS in electrical engineering from The City College of New York and an MS in Counselor Education from CW Post College of Long Island University. His professional career includes teaching and counseling as well as technical journalism as President of the Technology Communications Group. In his current position he is founder and President of Sinotech (USA) Inc. and Director of SinoTECH International (China) Ltd.. Sinotech is a major offshore manufacturer of various motor components as well as other mechanical and electromechanical parts.