AEi Systems is an electrical engineering services company, with a primary focus on providing Worst Case Circuit Analysis. Other analysis services we provide include Electrical SDRLs such as FMECA, MTBF, Stress & Derating analysis, as well as, Hardware Failure Resolution, Trouble-shooting, and Component and Circuit Test and Characterization. We also are experts in SPICE Modeling and make, or have made the majority of Power IC models for Linear Technology, National s Webench, Texas Instruments, ON Semiconductor, Analog Devices, as well as many others. We know that the Worst Case scenario can happen when you least expect it. As a result, we end up reviewing a lot of the WCCA performed by other companies to make sure it is up to industry standards. Our desire to see the level of analysis improved across the aerospace industry has motivated us to write this paper. Shown above are some WC examples. At the top right is a picture of the Large Hadron Collider at CERN. AEi Systems analyzed and fixed the power supply for the Atlas Experiment and then designed a new supply for the challenging environment (300Gauss Magnetic field, High TID and Neutron Radiation). AEi Systems, LLC 1
Is worst case analysis a thought process?? We re often asked Why do you guys always take such a pessimistic view of everything? Our response is Why do engineers find this process so uncomfortable? No one likes to be told their baby is ugly. But the objective of WCCA is to look for potential issues in an effort to improve product quality and reliability with the added bonus of saving the current program money while making future products more robust. In 17 years, we have not analyzed a design that has passed a WCCA with 100% compliance. On the other hand, every WCCA has resulted in an superior end product and a good deal of time and money saved. AEi Systems, LLC 2
AEi Systems reviews a lot of worst case analysis. So we thought we d hit some key points related to the issues and problems we ve encountered. This presentation looks at two aspects; Lessons Learned from HOW we do WCCA and Lessons Learned from actually DOING WCCA. On AEi Systems web site there is a white paper on WHY you might want to do a WCCA. http://www.aeng.com/design_analysis.htm. In addition, Aerospace Corporation has put out a TOR guideline which covers similar territory, as well as some details on what to analyze for different types of circuits. AEi Systems, LLC 3
Analyses that we review from all types of companies, from prime contractors to their smallest subcontractors, suffer from the same issue LACK OF RIGOROUSNESS. Two factors are clearly evident; insufficient analyses and insufficient detail in the analyses that are performed. The selection of the parameters to be analyzed should NOT be generated by the circuit designer. Mistakes in the design will often be repeated in the analysis. Circuits that the designer believes are too simple, obvious, or heritage may be ignored. This is often where problems lie. In fact, we often look for which analyses were NOT presented in order to determine where to look for design problems. Finally, is the documentation complete. Each analysis must be explained in a comprehensive and organized fashion complete with the functional description, methodology of analysis, detailed assumptions, model correlation, analysis results and conclusions such that it can be easily reviewed and referred to later in the product s life cycle. AEi Systems, LLC 4
Companies should not do their own WCCA. If they do, it should be done by an independent group. Independence puts checks and balances into the analysis. The project engineer is often under great schedule pressure, the program budget pressure and the company political pressure. When was the last time you walked into your boss s office and said, wow, look how I can bring my circuit to its knees. Thinking worst case is not normal for most engineers. WCCA is the last line of defense for assessing whether the design will perform properly throughout its lifetime. Its just too important. There are just too many biases and we should not let our egos or budgets rule this critical analysis. The worst situation is when a WCCA is bid before the circuitry is defined and/or there is schedule pressure. Both will lead to inadequate and often erroneous results. A WCCA should be bid by someone that both knows the details of the circuitry that will be built AND has performed a WCCA in the past. The availability, or lack thereof, of test data, accurate models, tolerance information, radiation data, and the scope of the analyses that should be performed, all factor into the cost and time needed to properly perform a WCCA. The selection of personnel is also a factor. Its one thing to have a junior engineer working along side a seasoned veteran, but its not enough to simply have the review done by senior engineers. It must be performed by experienced personnel. If they are not available, you need to call AEi Systems because we have the experience and the resources. AEi Systems, LLC 5
An engineer from a prime satellite manufacturer once told us that correlating and verifying models from a vendor should be the exception not the rule and that its ok to assume that vendor models are accurate. WHAT A DISASTER! With a little bit of investigation its clear that the exact opposite is true. There are two aspects to verify with EVERY SPICE model; does the model emulate the particular characteristic that the analysis requires and second, how accurately is the characteristic emulated over the operating range that the model will see in the simulation. AEi Systems, LLC 6
The models available from IC vendors, or in your SPICE package, are generally NOT tested against hardware. Most are made from data sheet information. Data sheets are not technical documents, they are marketing tools. Much of the data is misleading and from conflicting operating conditions, making model generation from a coherent set of data that covers the entire operating range of the part difficult if not impossible. Without correlating your own models you don t know what level of accuracy and fidelity you have. Recently, we tested a linear regulator model from an IC vendor for output impedance. We also tested the real part. The data was not in the data sheet. The results didn t match. Without the right output impedance, what characteristics are not correct? Stability, step load, ripple rejection; not much right? You must test and validate every single model you use for every characteristic that has to be right. AEi Systems, LLC 7
Yes, its true. With SPICE its garbage in, garbage out. But with high quality/high fidelity models in, the results can be dead on accurate. In this simulation of the power bus in the Space Station, AEi predicted, two years ahead of time, that a master computer reset would be triggered via a bus dropout every time the spacecraft went through an eclipse. Sure enough, as the picture on the right shows, that s just what preliminary circuitry did. Using the accurate model, AEi was able to confirm a fix for the circuit error. Investment in your models is required to built up a library that you have confidence in. The investment is well worth it and will save you time and schedule on the next program. Accurate WCCA relies to a large degree on accurate SPICE models and modeling capabilities, and AEi Systems is the number one source in the world in terms of custom SPICE modeling. AEi Systems has proprietary agreements and relationships with all the key component manufacturers, including TI, Linear Technology, Analog Devices, Microsemi, Intersil and National Semiconductor. That means that AEi Systems has or is able to obtain proprietary information that is not easily available to others. This information is critical to getting key performance assumptions correct and enables AEi to perform WCCA faster than other resources. AEi Systems, LLC 8
In order to perform a WCCA you need to be able to get reliable test data. Correlation of early hardware to test data is essential for model correlation and a successful WCCA. This takes a great deal of skill AND experience, as well as the right test equipment. We often find inadequate test equipment (insufficient bandwidth) and poor interconnects and probes at the heart of bad data. In this case, the circuit designer didn t test their regulator with sufficient bandwidth (left). AEi s model revealed a problem with the output impedance and ESR (in the nominal case). Further testing confirmed the finding. AEi worked interactively with the product manufacturer and customer to assess the impact of the poor stability and implement a fix. AEi Systems, LLC 9
Correct computation of EOL tolerances and good models are not the only hurdles to using SPICE. Applying the tolerances can be tricky. One mistake and your simulation results will be incorrect. It takes experience and a deep knowledge of SPICE parameters to get this right. The first thing to check is that your models behave correctly over the entire range of tolerances. Are the tolerances you applied correlated or uncorrelated. Does the SPICE simulator you use truncate the distribution? Datasheet parameters rarely correspond to a specific SPICE parameter, especially in subcircuit macro-models. Take this simple example of the forward voltage tolerance on a diode; its not as straightforward as it seems. How do you get the model to give you the right Vf min-max variation during a Monte Carlo analysis given there isn t a VF SPICE parameter (diodes use two parameters, IS and N to define the forward voltage)? You can t just apply the EVA tolerance on the N and/or IS parameters. The knowledge of how to accurately apply tolerances to SPICE models can only be obtained over years of experience in doing WCCA something that AEi Systems brings to the table. AEi Systems, LLC 10
Most analysts perform Monte Carlo analysis incorrectly. They mistakenly believe that you can simply chose an arbitrary number of runs and use the worst simulation results (end-points) as the worst case performance. This should scare those of you that do this as you have been making design decisions based on an erroneous assumption. Monte Carlo results must be embedded in a valid statistical framework (normally a confidence interval plus probability/population coverage). Without one you are not assessing the results correctly. You basically don t know what you have as a result. AEi Systems uses two proven methods; Tolerance Intervals for Normal Distribution and Distribution Free Tolerance Intervals (next slide); the only correct way to determine the number of runs such that you can use the simulation family s end-points. AEi Systems, LLC 11
The Distribution Free Tolerance Interval method provides a basis for the selection of the number of runs necessary for the simulation end-points to be used as the worst case result given a specific confidence interval and population coverage. AEi Systems, LLC 12
People often find reasons to not perform a WCCA that don t quite make sense when you think about them. AEi Systems, LLC 13
A large commercial space hardware manufacturer once told us that they only do Stress & Derating analysis, not WCCA. I told them that is a part based analysis, stress doesn t check circuit functionality. They countered with the fact that their missions were short. When asked whether that short mission life made it any less important that their product perform successfully they countered with the argument that heritage, testing and budgetary constraints prohibited WCCA. They didn t seem to understand that the only way to know, even over BOL tolerances, if a circuit is going to function properly is WCCA. They also didn t understand that compared to additional testing beyond specification required parameters WCCA can REDUCE costs. There is a reason a majority of the WCCA performed is based on derived requirements, many of which can not easily be tested. Simply put, you can t verify performance with test alone and most mission anomalies are due to flaws in the design. They also didn t understand that WCCA provides benefits beyond the analysis, improving future product generations and lowering insurance costs. AEi Systems, LLC 14
Many of the problems we see during the course of analysis are in the simplest circuits, which are often given little attention. These also tend to be the most critical circuits. Some of the most common analysis findings are shown above. In addition to EOL issues, WCCA is extremely valuable during the initial stages; when we correlate the nominal models and BOL tolerances with unit test data. This is where product quality and confirmation of circuit fixes occur. AEi Systems, LLC 15
While there are many lessons to be learned from WCCA, one of the biggest is loop stability. The popular multi-winding Flyback power supply, with its outputs post regulated, is a common configuration for power generation in satellites. While often used, its complexities are frequently misunderstood and underestimated. This topology is one of the hardest to analyze for EOL performance due to the complexity of the power transformer. The actual flight configuration magnetic must be measured in order for a model to be derived. Without the data you can not make a valid model and will, therefore, not get most of the analysis (stability, cross-regulation, CS, step load, etc.) of your power supply correct. Prediction of leakage tolerances is even less understood. An example of AEi s power transformer modeling technique can be found on the AEi website in the paper AEi Systems Component Test and Model Summary. AEi Systems, LLC 16
While most WCCA guidelines documents specify only a phase/gain margin stability assessment, it is possible that such an assessment does not actually quantify the stability of the power supply or regulator under investigation. In the picture (left) we see the phase margin as 50 degrees. But the step load response has a Q of 4 or 5, clearly not corresponding to 50 degrees. The stability of this power supply would not be properly assessed if only left up to the phase/gain margin compliance. AEi Systems has developed special test equipment to test for stability and performs testing as a service. Again, like incorrectly applying Monte Carlo methods, inexperienced users can stub their toe on one of the many pitfalls associated with testing and modeling for WCCA. AEi Systems has published a paper on this topic in the June 2012 Issue of Power Electronics Technology entitled When Bode Plots Fail Us. AEi Systems, LLC 17
What does poor stability mean? It can be difficult to quantify. But without an accurate assessment of every control loop, both in terms of test data and EOL performance, which can be drastically different than the typical performance, you will never know. Given capacitor tolerances, EOL performance can be over 20 to 30 degrees worse than initial/bol performance. This is why WCCA guidelines specify that ALL control loops must be assessed, even those that are not exposed outside the IC. For those, output impedance/step load testing (BOL) and simulation (EOL) are the only mechanisms available for the assessment. Performing WCCA on all your regulators is right way to go and leads to clean system power. Poor stability leads to degraded system performance. AEi Systems, LLC 18
Shown above are the results of a survey taken by AEi Systems. You can review and take the survey, as well as see the results at : http://www.aeng.com/regulator_survey.htm. Based on WCCA reviews we have done, based on known issues with data sheets, based on poor test data and methods, based on poor models that we know to be out there - that we have done hardware correlations on, based on incorrect Monte Carlo analysis techniques, based on who is doing the analysis, and based on this survey, its clear that the level and quality of WCCA being performed by most companies is inadequate and often used to drawn erroneous conclusions. AEi Systems, LLC 19
Thanks for reading through this presentation. Hopefully you ve discovered some areas for improvement in your own attempts at WCCA. Next time, you might consider letting the experts at AEi Systems handle the WCCA and other SDRLs for you. WCCA is what we do for a living and as such we can confidently say we know exactly what organizations like the Aerospace Corporation, NASA, as well as all the prime satellite manufacturers want from a WCCA. That s because we are an official SME (Subject Matter Expert) on WCCA and helped to set the guidelines as part of the MAIW 2012 (Mission Assurance Improvement Workshop) initiative. Because of our years of experience, vast library of SPICE models, and large database of tolerances, we are able to give clients a TOR compliant WCCA faster and for a lot lower cost than in-house resources. If you have any questions, please feel free to contact us at any time. AEi Systems, LLC 20