FocusStrong petchems inventory build-up in China limits trade

Optimization Using Symbolic Derivatives

By Alan Weiss

Most Optimization Toolbox™ solvers run faster and more accurately when your objective and constraint function files include derivative calculations. Some solvers also benefit from second derivatives, or Hessians. While calculating a derivative is straightforward, it is also quite tedious and error-prone. Calculating second derivatives is even more tedious and fraught with opportunities for error. How can you get your solver to run faster and more accurately without the pain of computing derivatives manually?

This article demonstrates how to ease the calculation and use of gradients using Symbolic Math Toolbox™. The techniques described here are applicable to almost any optimization problem where the objective or constraint functions can be defined analytically. This means that you can use them if your objective and constraint functions are not simulations or black-box functions.

Running a Symbolically Defined Optimization

Suppose we want to minimize the function x + y + cosh(x – 1.1y) + sinh(z/4) over the region defined by the implicit equation z2 = sin(z – x2y2), –1 ≤ x ≤ 1, –1 ≤ y ≤ 1, 0 ≤ z ≤ 1.

The region is shown in Figure 1.

Figure 1. Surface plot created in the Symbolic Math Toolbox notebook interface showing the region defined by the implicit equation z2 = sin(z – x2y2), –1 ≤ x ≤ 1, –1 ≤ y ≤ 1, 0 ≤ z ≤1. Click on image to see enlarged view.

The fmincon solver from Optimization Toolbox solves nonlinear optimization problems with nonlinear constraints. To formulate our problem for fmincon, we first write the objective and constraint functions symbolically.

We then generate function handles for numerical computation with matlabFunction from Symbolic Math Toolbox.

The returned output structure shows that it took fmincon 20 iterations and 99 function evaluations to solve the problem. The solution point x (the yellow sphere in the plot in Figure 3) is [-0.8013;-0.6122;0.4077.

Figure 3. Constraint set and solution point. Click on image to see enlarged view.

Solving the Problem with Gradients

To include derivatives of the objective and constraint functions in the calculation, we simply perform three steps:

1. Compute the derivatives using the Symbolic Math Toolbox jacobian function.
2. Generate objective and constraint functions that include the derivatives with matlabFunction
3. Set fmincon options to use the derivatives.

The following code shows how to include gradients for the example.

Notice that the jacobian function is followed by .’. This transpose ensures that gradw and gradobj are column vectors, the preferred orientation for Optimization Toolbox solvers. matlabFunction creates a function handle for evaluating both the function and its gradient. Notice, too, that we were able to calculate the gradient of the constraint function even though the function is implicit.

The output structure shows that fmincon computed the solution in 20 iterations, just as it did without gradients. fmincon with gradients evaluated the nonlinear functions at 36 points, compared to 99 points without gradients.

Including the Hessian

A Hessian function lets us solve the problem even more efficiently. For the interior-point algorithm, we write a function that is the Hessian of the Lagrangian. This means that if ƒ is the objective function, c is the vector of nonlinear inequality constraints, ceq is the vector of nonlinear equality constraints, and λ is the vector of associated Lagrange multipliers, the Hessian H is

∇2u represents the matrix of second derivatives with respect to x of the function u.

fmincon generates the Lagrange multipliers in a MATLAB® structure. The relevant multipliers are lambda.ineqnonlin and lambda.eqnonlin, corresponding to indices i and j in the equation for H. We include multipliers in the Hessian function, and then run the optimization¹.

The output structure shows that including a Hessian results in fewer iterations (10 instead of 20), a lower function count (11 instead of 36), and a better first-order optimality measure (2e-8 instead of 8e-8).

¹ For nonlinear equality constraints in Optimization Toolbox version 9b or earlier, you must subtract, not add, the Lagrange multiplier. See bug report.

MATLAB Digest – March 2010

Basel II Compliance and Risk Management Analysis: Calculating Economic Capital

By Marco Folpmers, Capgemini
Send e-mail to Steve Wilcockson

Economic capital (EC), the amount of capital that an organization must set aside to offset potential losses, is a key metric for many European banks and financial institutions. It is also a central requirement of Pillar 2 of the Basel II regulatory framework. While Capgemini and many of its clients view EC as the best measure of risk inherent in a portfolio, calculating EC is not a straightforward analytical exercise. Portfolios heavily weighted in a particular sector, for example, carry a significant amount of concentration risk, complicating EC analysis.

Using MATLAB®, Capgemini has developed a process for calculating EC that takes portfolio concentration into account. The process involves four main steps:

• Gathering inputs, including information about individual loans in the portfolio
• Preprocessing the data
• Running Monte Carlo simulations to estimate portfolio loss
• Presenting the results

We chose MATLAB because its matrix-based infrastructure is ideal for organizing the kinds of data that we deal with and the operations that are applied to this data, including the linear algebra operations that are needed for calculating EC. The ability to perform Monte Carlo simulations in MATLAB gives us another key advantage when modeling EC and other kinds of risk.

Gathering Inputs

Before calculating EC for a loan portfolio, we must determine some standard risk parameters for each loan in the portfolio. These parameters, which include probability of default (PD) and loss given default (LGD), are often provided in the databases that our clients already use for Basel II compliance. To enable the calculation of concentration risk, each loan must also be assigned to a sector—for example, utilities, energy, or automotive.

Our clients store this information in data warehouses and databases from a variety of vendors. We use Database Toolbox™ to import the information into MATLAB from any ODBC/JDBC-compliant database and from data warehouses such as Teradata. If the data is provided in spreadsheets, we use a simple call to xlsread() to read it in. After importing the data into MATLAB, we clean it by computing missing values and identifying outliers.

Calculating the Correlation Matrix and Default Thresholds

Because accounting for correlation risk is a key requirement of Basel II, we must calculate a correlation matrix that reflects the way European macroeconomic sectors are linked. Based on equity return information in all these sectors derived from multiple data sources, including Bloomberg and Dow Jones Stoxx, we calculate correlations between sectors and store them in a matrix (Figure 1). The matrix is used in the Monte Carlo simulations to incorporate sector information into the likelihood of default for each loan.

Figure 1. A correlation matrix for 18 European supersectors. Click on image to see enlarged view.

Our credit risk models are based on the Merton model, in which an obligor (a loan customer) defaults when the asset return generated in the simulation falls below the Z-default threshold. Z-default is defined as the normal inverse of the PD, which ensures that, in the long run, the obligor will default as many times as is predicted by the PD.

Running Monte Carlo Simulations

We run a Monte Carlo simulation of the portfolio for up to one million different scenarios. For each scenario (or iteration), we do the following:

• Determine which loans default
• Estimate the loss for each defaulting loan
• Sum the individual loan defaults to find the portfolio loss

To identify default loans in the portfolio, a standard normally distributed random number is determined for each loan. This number depends on a random number corresponding to its sector, which is common to all loans in that sector, and an idiosyncratic random number, drawn for each loan separately. In this way, the health of the sector that the obligor belongs to influences whether this obligor defaults.

We generate the random numbers per sector using the Statistics Toolbox™ mvnrnd() function. These numbers are drawn from a multivariate normal distribution that takes into account the intersector correlations (as specified in the correlation matrix). The use of the normal distribution is not a constraint. Sometimes a multivariate t distribution (a t copula) is used if the client wants to enhance the level of tail dependence (the dependence among extreme outcomes of the asset returns) in the model.

We estimate the loss for a defaulting loan by sampling from a beta distribution based on the LGD. For example, if the LGD for a loan is 15%, we set up the parameters α and β of the distribution so that individual default losses may range from 0% to 100% but in the long run the results will be 15%.

The final step of each iteration is to total the losses for all loans in the portfolio and store the results in a loss vector. When all iterations are complete, the loss vector holds the distribution of losses for the portfolio. To compute the expected loss (EL) and the EC, we use two simple MATLAB functions:

EL = mean(Loss);EC = prctile(Loss,99.95) – EL;

The example above calculates EC using the 99.95^th percentile, the amount of capital needed to protect the bank against losses that could occur 99.95 percent of the time. The percentile can change according to the bank’s target credit rating. Since the expected loss can also be calculated analytically, it is an ideal statistic for validating the calculations. The alignment of Monte Carlo Expected Loss with analytical Expected Loss is a standard checkpoint in our routines.

Presenting the Results

We validate and present the results to clients using a MATLAB histogram of the credit loss vector (Figure 2).

Figure 2. A statistical loss distribution for a sample portfolio. Click on image to see enlarged view.

The histogram depicts the level of default correlation, enabling us to rapidly identify irregularities. If correlations are high, the histogram tends to be concentrated on the left (low losses) and right (high losses) ends of the distribution. If correlations are low, the histogram tends to be heavy in the middle of the distribution.

We use MATLAB to generate reports of the analysis, write the results to a spreadsheet, or save them in a database, depending on the client’s needs. In most cases, we provide the MATLAB source code so that the client can see how the model works and modify it for future calculations.

Many of our clients are MATLAB users. For those who are not, we use MATLAB Compiler™ to build a standalone application with a graphical user interface that lets them run sophisticated analysis and simulations without installing MATLAB.

Optimizing Performance

When running Monte Carlo simulations that require hundreds of thousands of iterations, any step that accelerates a single iteration significantly reduces simulation time. Wherever possible we use built-in MATLAB functions, which are typically much faster than those we develop ourselves. We also take advantage of MATLAB vector and matrix operations and look for opportunities to move calculations outside the simulation iterations and to eliminate nested loops. For example, many of our clients find it useful to allocate EC to each obligor. It is inefficient to use a new simulation for each obligor, so we made this an optional calculation of the main EC loop. We can then allocate EC only when we need to, and speed the calculation of overall EC when we do not.

Another example of code optimization is the declaration of vectors needed in the simulation loop with the help of a zeros or ones statement prior to starting this loop instead of using a vector that grows during the execution of the simulation loop.

It is advisable to allocate EC to the loan level so that it is clear how much risk is generated by each loan. The results can be presented graphically for risk management purposes. For example, in Figure 3 we have plotted each loan as a circle (‘o’) on the two axes: loan size (or Exposure at Default) and risk size (or EC Contribution divided by the loan size).

Figure 3. Risk analysis at the loan level. Click on image to see enlarged view.

Traditional risk management is concerned with monitoring large loans. This is the perspective expressed by the x-axis. With the help of the contributions, a second perspective can be added on the y-axis: the risk that each loan carries.

A Versatile Environment for Modeling Risk

We use MATLAB to model a number of other risk types. For example, we model interest rate risk in the banking book (another Pillar 2 Basel II requirement) to determine the bank’s exposure to adverse movements in interest rates. Here, instead of multivariate normal distributions, we use a t copula and generate random observations from a multivariate t distribution using mvtrnd().

We are seeing an increasing demand from rating agencies and banks for models of structured credit products, such as collateralized debt obligations and mortgage-backed securities. MATLAB helps us build and simulate exceptionally fine-grained models that take into account every instrument in a reference portfolio.

Developing the World’s Most Advanced Prosthetic Arm Using Model-Based Design

By James Burck, Michael J. Zeher, Robert Armiger, and James D. Beaty, Johns Hopkins University Applied Physics Laboratory

Few of us are aware of the complex interactions between neural, mechanical, and sensory systems required to perform a task as simple as picking up a ball. To create a prosthetic arm capable of natural movement, it is necessary to mimic these sophisticated systems, as well as the intricate interactions between them, using cutting-edge actuators, sensors, micro­processors, and embedded control software. That was the challenge we faced when we embarked on the Defense Advanced Research Projects Agency (DARPA) Revolutionizing Prosthetics program.

Johns Hopkins University Applied Physics Laboratory (APL) is leading a worldwide team including government agencies, universities, and private companies whose mission is to develop a prosthetic arm that far exceeds any prosthetic available today. The final version of the arm will have control algorithms driven by neural inputs that enable the wearer to move with the speed, dexterity, and force of a real arm. Advanced sensory feedback technologies will enable the perception of physical inputs, such as pressure, force, and temperature.

A key project milestone was the development of the Virtual Integration Environment (VIE), a complete limb system simulation environment built using MathWorks tools and Model-Based Design. With a standardized architecture and well-defined interfaces, the VIE is enabling collaboration among domain experts at more than two dozen partner organizations.

Model-Based Design with MathWorks tools was used in other key phases of development—including modeling the limb mechanics, testing new neural decode algorithms, and developing and verifying control algorithms.

The two prototype limbs developed for the DARPA program use Targeted Muscle Reinnervation, a technique pioneered by Dr. Todd Kuiken of the Rehabilitation Institute of Chicago. This technique involves the transfer of residual nerves from an amputated limb to unused muscle regions near the injury. In a clinical evaluation, the first prototype enabled a patient to complete a variety of functional tasks, including pulling a credit card from a pocket.

Virtual Integration Environment Architecture

The VIE architecture consists of five main modules: Input, Signal Analysis, Controls, Plant, and Presentation.

The Input module comprises all the input devices that patients can use to signal their intent, including surface electromyograms (EMGs), cortical and peripheral nerve implants, implantable myoelectric sensors (IMESs) and more conventional digital and analog inputs for switches, joysticks, and other control sources used by clinicians. The Signal Analysis module performs signal processing and filtering. More important, this module applies pattern recognition algorithms that interpret raw input signals to extract the user’s intent and communicate that intent to the Controls module. In the Controls module, those commands are mapped to motor signals that control the individual motors that actuate the limb, hand, and fingers.

The Plant module consists of a physical model of the limb’s mechanics. The Presentation module produces a three-dimensional (3D) rendering of the arm’s movement (Figure 1).

Figure 1. A 3D rendering of the prosthetic arm.

Interfacing with the Nervous System

Simulink® and the VIE were essential to developing an interface to the nervous system that allows natural and intuitive control of the prosthetic limb system. Researchers record data from neural device implants while the subjects perform tasks such as reaching for a ball in the virtual environment. The VIE modular input systems receive this data, and MATLAB® algorithms decode the subject’s intent by using pattern recognition to correlate neural activity with the subject’s movement (Figure 2). The results are integrated back into the VIE, where experiments can be run in real time.

Figure 2. A MATLAB application developed by the University of New Brunswick, used to record motion data for pattern recognition. Click on image to see enlarged view.

The same workflow has been used to develop input devices of all kinds, some of which are already being tested by prosthetic limb users at the Rehabilitation Institute of Chicago.

Building Real-Time Prototype Controllers

The Signal Analysis and Controls modules of the VIE form the heart of the control system that will ultimately be deployed in the prosthetic arm. At APL, we developed the software for these modules. Individual algorithms were developed in MATLAB using the Embedded MATLAB™ subset and then integrated into a Simulink model of the system as function blocks. To create a real-time prototype of the control system, we generated code for the complete system, including the Simulink and Embedded MATLAB components, with Real-Time Workshop®, and deployed this code to xPC Target™.

This approach brought many advantages. Using Model-Based Design and Simulink, we modeled the complete system and simulated it to optimize and verify the design. We were able to rapidly build and test a virtual prototype system before committing to a specific hardware platform. With Real-Time Workshop Embedded Coder™ we generated target-specific code for our processor. Because the code is generated from a Simulink system model that has been safety-tested and verified through simulation, there is no hand-coding step that could introduce errors or unplanned behaviors. As a result, we have a high degree of confidence that the Modular Prosthetic Limb will perform as intended and designed.

Physical Modeling and Visualization

To perform closed-loop simulations of our control system, we developed a plant model representing the inertial properties of the limb system. We began with CAD assemblies of limb components designed in SolidWorks® by our partners. We used the CAD assemblies to automatically generate a SimMechanics™ model of the limb linked to our control system in Simulink.

Finally, we linked the plant model to a Java™ 3D rendering engine developed at the University of Southern California to show a virtual limb moving in a simulated environment.

Clinical Application

Given the powerful virtual system framework, we were also able to create a useful and intuitive clinical environment for system configuration and training. Clinicians can configure parameters in the VIE and manage test sessions with volunteer subjects using a GUI that we created in MATLAB (Figure 3).

Figure 3. A MATLAB based user interface for configuring prosthesis parameters. Click on image to see enlarged view.

Clinicians interact with this application on a host PC that communicates with the xPC Target system running the control software in real time. A third PC is used for 3D rendering and display of the virtual limb. During tests of actual limbs, we can correlate and visualize control signals while the subject is moving.

Looking Ahead

Using Model-Based Design, the Revolutionizing Prosthetics team has delivered Proto 1, Proto 2, and the first version of the VIE ahead of schedule. Currently we are in the process of developing a detailed design of the Modular Prosthetic Limb, the version that we will deliver to DARPA.

Many of our partner institutions use the VIE as a test bed as they continue to improve their systems, and we envision the VIE continuing as a platform for further development in prosthetics and neuroscience for years to come. Our team has established a development process that we can use to rapidly assemble systems from reusable models and implement on prototype hardware, not only for the Revolutionizing Prosthetics project but for related programs as well.

As we meet the challenge of building a mechatronic system that mimics natural motion, we strive to match the perseverance and commitment that our volunteer subjects and the amputee population at large demonstrate every day.

Approved for Public Release, Distribution Unlimited.

Newly endorsed system integrators recognized for commitment