Services

Model calibration is a multi-faceted process and our services reflect that. In this section, we discuss various aspects of our work and our offerings.

Advisory

Documentation Review

As a development process, calibration has many technical and procedural elements that banks typically codify in their model development standards. It is obviously important to create and maintain comprehensive documentation around these policies and standards.

Experience shows that some aspects of calibration – such as the use of expert opinion – can be difficult to render into text; others require more conventional technical modelling knowledge, but are challenging nonetheless.

Radius can review, advise and even generate the documentation that provides the foundation for calibration processes and governance. Prior to that of course we can advise on any aspect of modelling methodology associated with calibration; this is our core expertise.

Methodology Guidance

As we hope this site communicates, we have a wealth of experience and domain knowledge in the area of PD model calibration, across an extremely diverse set of calibration scenarios and portfolio types. At the heart of all this experience are deep skills in the relevant modelling methodologies.

We think our knowledge of the calibration process, the relevant model architectures, and parameter estimation techniques, could be an extremely valuable contribution to a bank’s overall methodology base. As such, we are more than happy to review-and-comment on documentation on modelling standards, and individual calibrations. This is core to what we do.

Calibration Architecture

As has been mentioned in several areas of the site, calibrations are not just some footnote to the core model build; they are an architectural consideration in their own right. And of course, there can be more than one calibration for a model.

Our perspective on calibration architecture is hierarchical: we have calibration templates that start at the conceptual level and run all the way down to physical implementation and everything in-between. We can review and advise on the overall architecture to any degree of complexity, to any level of detail, on any aspect.

Calibration Audit

Probably the easiest way to engage Radius is for us to review your:

  • Current approaches to calibration (i.e. methodology)
  • Recent validation results (calibration performance of specific models); and
  • Other elements such as the overall architecture, and road map.

Following a review, we would derive a set of recommendations and actions, to whatever depth and scope is most beneficial. This process can be a light touch, akin to review and triage, or we can treat it more like an audit.

$$\mathbb{P}\left ( D\leq d;\gamma , \rho , n \right )=\int_{-\infty }^{+\infty}\sum_{k=0}^{d}\binom{n}{k}\left ( \Phi \left ( \frac{\gamma-\sqrt{\rho}x}{\sqrt{1-\rho}} \right ) \right )^{k}\left ( 1-\Phi \left ( \frac{\gamma-\sqrt{\rho}x}{\sqrt{1-\rho}} \right ) \right )^{n-k} \phi \left ( x \right )dx$$

Development

The heart of calibration is modelling. Parameter estimation. Sampling. Exploratory data analysis (EDA). Diagnostics. Validation. And, sometimes, computationally-intensive processes such as simulation and the development of auxiliary models.

Calibration doesn’t have quite the intensity of a full PD model build, but it is still a modelling exercise, of high materiality, and needs to be conducted to the same standards. We treat calibrations in exactly this way: as a bona fide model build, to be undertaken to the highest standards, with an independent review process, and multiple stakeholders to be managed along the way.

As we’ve indicated in an earlier section, there are multiple calibration layouts or configurations possible. These layouts largely depend on the core model’s architecture, but there is usually some flexibility to inject additional structure if desired.

Radius can cover any of these of these layouts or configurations and we can design a calibration solution to fulfil a specific output pattern or calibration archetype. This is discussed at a high level in the next section.

We can provide anything from a first pass validation of calibration performance on your data, all the way up to a full re-calibration. We can also calibrate models that have just been built although this is an unlikely scenario given the narrow window within which this usually takes place.

As an important corollary, we can also review, validate and rebuild a bank’s master rating scale (MRS). This is obviously a ‘model-agnostic’ exercise, with long-term implications, and a lot of preliminary analysis required (often including the collection of external benchmarking data).

Radius can cover any of these of these layouts or configurations and we can design a calibration solution to fulfil a specific output pattern or calibration archetype. This is discussed at a high level in the next section.

We can provide anything from a first pass validation of calibration performance on your data, all the way up to a full re-calibration. We can also calibrate models that have just been built although this is an unlikely scenario given the narrow window within which this usually takes place.

As an important corollary, we can also review, validate and rebuild a bank’s master rating scale (MRS). This is obviously a ‘model-agnostic’ exercise, with long-term implications, and a lot of preliminary analysis required (often including the collection of external benchmarking data).

Solution Archetypes

As we’ve suggested in another section, calibrations fall into archetypes, and these fundamental patterns collectively describe a spectrum of model output types. In what follows, ‘FDR’ denotes forecast default rates (“forecast” because PD models are forward-looking) and ‘ODR’ stands for observed default rates (i.e. the actual portfolio performance through time, independent of the model).

At the most basic level we have what we call a ‘Base’ calibration; this archetype matches the average of model-predicted values to long-run default rates. This is the simplest, most common, and least demanding of calibrations to define, analyse, model, implement and maintain.

At the most basic level we have what we call a ‘Base’ calibration; this archetype matches the average of model-predicted values to long-run default rates. This is the simplest, most common, and least demanding of calibrations to define, analyse, model, implement and maintain.

The second calibration archetype we’ll briefly cover here has for years been termed a ‘Point in Time’ (PIT) calibration. This pattern is defined by model outputs tracking ODRs closely through time, replicating their levels and capturing the structure of the cycle. PIT models are considered by practitioners to reflect the cycle as it is happening, rather than reflecting long-term trends; an attribute that has implications for other regulatory frameworks.

Market prices for debt instruments, and models based on these, are classic PIT patterns.

At the other end of the spectrum is a calibration type that is characterised by PDs that do not vary with changing conditions in the business environment, reflecting only the long view: TTC or ‘through the cycle’ calibrations. These calibrations generate ‘flat’ PDs through time, filtering all of the information related to the credit cycle out of the signal.

Radius can specify, build and integrate a cluster of solutions from these archetypes that have target properties, to serve a specific purpose.

We can take any rating model, regardless of its intrinsic cycle signature, and build a set of calibration solutions that fall in to these archetypes. Obviously, the more advanced archetypes (especially PIT) is a challenging, multi-stage modelling exercise and results in a dynamic, multi-level architecture.

Deployment

Like their associated models, calibration solutions require deployment.

Despite the apparent simplicity of a calibration function, there are many issues that need to be considered when implementing an industrial-strength solution, and complications can mount. The days of adding (or modifying) one or two lines of a modeller-written script, are over.

Deployed solutions have layers. At the heart of a solution there is a kernel that embodies the solution as a mathematical construct, but it is (or should be) wrapped and supported by a host of other calculations that orchestrate, monitor and log.

Radius has experience with a wide range of architectural patterns and implementation solutions, taking our offering far beyond of that of mere advisory and development.

Radius can take a solution designed and built by us, or any other party, and provide code to deploy that solution as a callable asset or resource. But far beyond that, Radius can also host the solution as a web-based callable service (or any combination of the two). This possibility is described in more detail elsewhere

We are familiar with a range of third-party services and libraries, and have also written our own.

Later in 2023 we will be launching a SaaS platform that covers the full life cycle of a model and its supporting calibration sub-models. This platform will include a comprehensive, governance-driven deployment process as described above.

We are familiar with a range of third-party services and libraries, and have also written our own.

We are familiar with a range of third-party services and libraries, and have also written our own.

Later in 2023 we will be launching a SaaS platform <link: Engagement Models/Modelling Platform> that covers the full life cycle of a model and its supporting calibration sub-models. This platform will include a comprehensive, governance-driven deployment process as described above.

We are familiar with a range of third-party services and libraries, and have also written our own.