Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Financial institutions rely more and Banks are also modeling many key As a result, the danger stemming from
more upon advanced mathematical, elements of business planning and rare yet plausible occurrences might
statistical and numerical models development, such as the appropriate be understated, as such occurrences
to measure and manage risk bundling of product features, customer might not appear in reference data
and manage their businesses.1 or on-customer income, or churn rates. sets or in historical data patterns.
Models are used to calculate economic
capital charges for various types of As the use of these models increases, In order to mitigate potential emerging
exposure, subject to different risk so does the possibility that any risks, the discipline of model risk
types – such as credit, market, or particular model may not properly management (MRM) has evolved
operational. Through their individual capture financial risk. Model error over the years to support emerging
components the bank’s current liquidity refers to the simplification or regulatory and business agendas.
position is projected under alternative approximation of reality within models, Until recently, industry MRM efforts
scenarios, as well as the projection incorrect or missing assumptions, have focused upon the management
of the balance sheet and income incorrect design processes, and of risk for individual models; now,
statements for use in stress testing.2 errors in measurement or estimation. however, we are seeing more
Model misuse is the application of institutions focus on quantifying and
models outside the use for which aggregating enterprise-wide model risk.
they were designed and/or intended.
2
Regulatory Guidance
As the use of complex models has become prevalent in the industry, regulators have continued to push financial
institutions to invest in model risk management, with focus on establishing comprehensive frameworks for
active model risk management including robust development, validation and monitoring capabilities.
In this regard, a high proportion of bank • As with other types of risk, model risk Within this framework, there are elements
decisions are automated through decision can never be eradicated, only controlled of good governance that should be observed.
models, which can be statistical in nature, by effective risk management, including In particular, in our view the three elements
or a methodology that constitutes a rule-set.3 comprehensive model development and of model ownership (identification and
There are several areas in which we see this rigorous model validation. measurement of the model risk to which the
trend manifest itself. For example, an increased institution is exposed); model control (setting
use of electronic trading platforms known • Institutions should establish an MRM limits, following up and independently
as algorithmic trading that execute trade framework that is approved and overseen validating models); and model compliance
commands which have been pre-programmed by senior management and the board (establishing processes to help perform
(e.g., through timing, price or volume), and can of directors. ownership and control rules in accordance
be initiated with manual intervention.4 • A well-documented and structured with accepted policies) are essential.
At this time, supervisory guidance on approach to model development and Overall, the supervisors expect institutions
model risk management is limited, validation can be useful in managing to create MRM frameworks that feature
and the regulations published to date model risk, but is not a substitute for the model development and validation criteria
tend to be based on principles; that is, continuous process of improving models. which are formalized, promote careful
they lack specificity in how to define • Prudent use of models is likely to involve model use, establish criteria to assess model
and treat model risk. elements such as conservative adjustments, performance, and define policy governance
stress testing or capital buffers, but an for consistent implementation of framework
In the US, the Federal Reserve and the Office and applicable standards of documentation.
of the Comptroller of the Currency (OCC) over-reliance on such elements may lead
to the misuse of models. This holistic approach to model risk
combined to publish the Supervisory Guidance management continues to be adopted by the
on Model Risk Management, which lays out • There are many potential sources of model financial services industry, and we expect it
basic principles for model risk management:5 risk and therefore institutions should to become increasingly prevalent as more
• Model risk is to be managed like other risks, consider the interaction of these factors institutions use such an approach.
in that model risk managers should identify and try to quantify the aggregated model
the sources of risk, assess the likelihood of risk that results from their combination.
occurrence and the severity of any specific
model failure.
3
Model Risk Governance Framework
As seen in Figure 1 below, the organizational structure for MRM should establish clear lines for development
and monitoring and provide the right incentives while promoting cross-functional involvement in model risk
discussions.
In our view, these are the key • Model Administrator. The model • Model Risk Committee. This is the
roles involved in the model administrator is usually the supervisor senior management committee with
governance process: of the model owner and may be responsible responsibility for overseeing the model
for assisting the model owner with governance program. The committee
• Model Owner. Ideally a person is assigned certain governance responsibilities. reports on risk issues and trends to
responsibility for developing or maintaining the risk committee of the board, or
a model, including documenting the model. The model administrator guides the takes other action as appropriate.
The model owner will also lead the model model developers in his or her team to
implementation process, along with the complete the model documentation, • Internal Audit. The internal audit function
model user. The model owner is usually complying with the model validation undertakes the independent review of all
responsible for ongoing model monitoring. department’s model document template. model risk and governance activities.
• Model User. The model user has • Model Validation and Review. Independent While some overlaps may occur among the
responsibility for the use of a model from the model owner, a validation unit is roles of model owners, model users and
for business purposes. Each model assigned the responsibility to implement model administrators, the model validation,
can have multiple model users. and oversee the model validation and model risk committee and internal audit
review functions and the model governance functions should remain independent.
program on a daily basis.
Board
Finance
Enterprise analytics
Risk
Internal audit
Line of
business Line of business Model administrator(s)
4
The Analytics Function
As seen in Figure 2 below, the preferred approach to setting up an efficient modeling function is to have a
modular team structure with specialized groups and/or centers of excellence addressing different aspects of
the model lifecycle.
Enterprise analytics
5
The Validation Function
Finally, the key part of a model risk management framework in our view is establishing a strong and
independent validation function. This function should be able to address the quantitative and qualitative
review of models across the five broad areas of data, methodology, documentation, processes and governance.
Figure 3 highlights some of the key themes that should be considered by the validation team while going
through the process.
As institutions continue to build out their with specialized skill sets to help drive An emerging trend in model validation is with
validation function, they should consider the validation for all internal and vendor- regard to models for stressed capital (e.g., as
the following building blocks: based models. used in applications such as Comprehensive
Capital Analysis and Review (CCAR) or
a) Well established policies and procedures c) Access to a wide variety of tools and the Dodd-Frank Act stress test (DFAST)),
to support consistent execution across applications such as centralized model currently at an early phase of development as
all validations. submission capabilities, standardized compared to other types of models, such as
documentation templates, and reporting methodologies for measuring market Value-
b) Access and appropriate alignment tools for better governance and control. at-Risk (VaR) or credit default prediction.6
of a strong pool of human capital
6
Various techniques exist to help validate Compared to just over a decade ago, we are This could be of concern if the
a stressed capital model, each capable of seeing a heightened supervisory focus on assessment of overall capital adequacy
informing on the robustness of a few of the validation of stressed capital models.7 is an important application of the model.
the desired model properties. Note that Based on our work in this area, stressed Improvements in these areas could include
some validation techniques are effective in capital model parameter harmonization and further benchmarking and industry-
the examination of only certain properties, cross-bank benchmarking are now viewed wide exercises, backtesting, profit and
where by effective we mean the ability of by regulators as weak points in model loss analysis and stress testing.
a test to detect departures from a desired development. On the positive side, supervisors
state of affairs. For example, many validation may have the impression that banks are Additionally, institutions should recognize
procedures exist that effectively assess doing well with respect to quantifying risk that when validation is partially or improperly
risk sensitivity (e.g., does the stressed on a relative basis (i.e., between LoB or completed, users of those models and senior
capital model reflect the state of the risk types) and producing models that exhibit management should be informed of the
economy?). Yet, in measuring the accuracy sensitivity to risk factors (e.g., to the state situation. Such communication is in our
of economic capital (i.e., is the estimator of the economy). Although there is view necessary so that model users and
of the loss distribution quantile estimate scope for practices to improve further, senior management understand that there
predictively accurate?), few techniques are the signs of progress in these areas is greater uncertainty around the models’
available to do this reliably. When used are moderately encouraging. output and that the model output should be
in combination and in an environment of treated with greater conservatism. In that
controls and model governance, a broad In other respects, we believe industry vein, model users and senior management
set of validation techniques could offer validation practices are weak, particularly should understand and explore the
more substantial evidence regarding when the total capital adequacy of the potential costs of using models that have
performance of a stressed capital model. bank and the overall calibration of the not been validated (i.e., if key assumptions
model is an important consideration. It is in the models prove to be inaccurate).
There still remains in our view much scope recognized within the industry that this
for the industry to enhance the robustness validation task is intrinsically difficult since
of its validation practices. These can inform it typically requires the evaluation of high
on how well stressed capital models are quantiles of loss distributions over long
calibrated, in particular for cases where periods combined with data scarcity, and
the assessment of overall capital is an coupled with technical difficulties such as tail
important application of the model (e.g., estimation. We also feel validation practices
to set a financial institution’s total risk will depend on what the model is being used
“budget”), as opposed to situations where for. Nevertheless, weaknesses in validation
it is not (e.g., where the model is used to practices targeted at the evaluation of overall
establish the relative risks of different LoB). performance might result in banks operating
with inappropriately calibrated models.
7
The Audit Function
A key development in the industry is the active involvement of the internal audit function in managing
model risk. As the focus on capital and liquidity management has gained prominence, it has put incremental
pressure on financial institutions to revisit their audit function’s operating models. As the third line of defense
to manage risk, the audit function is increasingly expected to perform self-testing that should challenge the
model’s conceptual design, data reliability, and risk management controls. In this capacity, the role of auditing
and self-testing is not to duplicate model risk management activities, but rather to assess how effectively a
model risk management framework meets business and regulatory requirements. Figure 4, shows a suggested
framework for conducting a MRM audit review.
The incremental expectations have created Traditionally, internal audit would become entire model development and validation
in our view a staffing challenge for internal involved toward the end of the process, after lifecycle as opposed to limited involvement at
audit functions looking for talent with the review of the model development process the end of the process. In response to the
specialized skills in model development, by independent validation. In this new model, increased expectations placed on audit
validation and in-depth knowledge of existing internal audit have check-point reviews as functions, institutions should consider
and emerging regulations. model developers remediate issues raised developing MRM relevant audit processes
by MRM or by regulators, independently of and policies and build a dedicated team with
Another emerging trend in model the model validation process. This allows the specialized quantitative skills across credit,
audit is the paradigm of real-time audit. audit to be more closely involved across the market and operational risks.
9
Model Monitoring Framework:
Risk Rating and Inventory
A risk rating systems for models is The Federal Reserve notes in its CCAR should incorporate model risk management
essential, along with a comprehensive guidelines that banks with lagging practices and the complexity of the change as
model inventory. The level of review were unable to identify all models in their inputs. Change management should
applied to each model should be capital computation process (see Figure 5).8 have standardized processes for change
commensurate with the risk posed by the recommendation (deciding when a change
model. More frequent and more extensive To be effective, model monitoring should is required) as well as for execution. This
review steps should be applied to the most be regular and comprehensive, covering should also include a periodic review process,
complex and most high-impact models. all aspects of model performance (not just to be carried out at least annually or when
discrimination performance). A “traffic light” there is a significant change to a model or
Similarly, a model inventory should system for model diagnostics has been to the applicable portfolio, to assess if a full
be an organized, enterprise-level repository shown to work well in this area as shown model validation should be performed.
of all models within the organization, or, at in Figure 6. When a change in models is
a minimum, of all Tier 1 and Tier 2 models. required, the change management process
Dropping/addition of variables
11
Measurement and Aggregation of
Model Risk
Measuring and aggregating model risk is a new and rapidly evolving field; multiple methodologies are under
consideration but none has yet been established as a standard (see Figure 7).
Approaches include: With this approach, each inventoried model • Model risk mitigation factors include
is assessed based on factors such as inherent ongoing model risk management by model
1. Model Risk Scorecard model risk, model risk mitigation and overall owners (the first line of defense) in areas
The firm creates a qualitative scorecard model risk. such as model performance monitoring,
that considers inherent model risk factors ongoing model benchmarking and ongoing
• Inherent risk factors include model
and model risk mitigation activities to present model backtesting, as well as MRM control
complexity, higher uncertainty about inputs
an aggregate view of model risk across functions (the second line of defense)
and assumptions, broader use and larger
the institution. such as model validation and other model
potential impact.
control activities.
Approach 1 Approach 2
Model risk scorecard Advanced operational risk
1 Ap
ch
pro
Approa
ach 2
Measurement
and aggregation
of model risk
Ap
proach 3
Approach 3
Model uncertainty
A bottom-up, quantitative approach to measuring model risk based on model risk sensitivity and
model risk control activities such as benchmarking, backtesting, consideration of alternative
modeling techniques, and corresponding impact.
12
Overall model risk is a function of inherent This approach is empirically grounded and It can be used to capture model
model risk and model risk mitigation subject to model validation, and it aggregates interdependence and also as input for
activities; used to measure and monitor and quantifies model risk in a coherent estimating economic capital for model risk.
model risk for individual models and in the manner. Institutions already using AMA can Its disadvantages include the subjectivity of
aggregate, and for periodic reporting to leverage existing infrastructure and processes. measurement and aggregation assumptions;
senior management and the board. Historical model risk event data tends to the time and resources needed to develop
be sparse, so scenario-based frequency and individual, model-specific approaches; and
The model risk scorecard can effectively severity estimates are often subjective and the need for ongoing performance of model
aggregate model risk across the institution judgment-based. Distributional assumptions control activities such as benchmarking,
and can leverage existing model inventory are also subjective and could affect backtesting, and external data comparisons
and risk classifications. It provides clear outcomes. Finally, in our view this approach to perform effective maintenance.
and easy-to-understand reporting tools is more complex to implement, especially
for senior management, and can be rolled without an existing AMA infrastructure.
out relatively quickly. However, while it
aggregates model risk, it does not attempt to 3. Model Uncertainty
quantify it. Scores, weights and overall results The firm uses a bottom-up, quantitative
are subjective, and this approach has little approach based on model risk sensitivity
ability to capture model interdependence. and model risk control activities such as
benchmarking, backtesting, consideration
2. Advanced Operational Risk
of alternative modeling techniques, and
This is a quantitative approach to model corresponding impact. This approach uses
risk aggregation based in part on the event information from multiple sources and
and scenario modeling techniques of the translates model risk into metrics such as
AMA to model risk. This approach views profit and loss (P&L) or capital impact.
model risk as a function of the frequency
Model uncertainty can provide bottom-up
and severity of potential model risk events,
quantification of risk for individual models,
based on a combination of historical
and it establishes a more rigorous and less
experience and scenario analysis.
subjective measurement of model risk.
13
Conclusion
14
Reference
1. A classic example in the field of credit 4. A recent example of this is an automated 6. “Validation of economic capital models:
risk is the structural modeling framework, trade command that occurred on May State of the practice, supervisory
which underlies any of the leading models 6, 2010 and that resulted in a $4.1 expectations and results from a bank
used in the industry (e.g., JP Morgan Chase billion “flash crash” of the New York study,” Michael Jacobs Jr., Journal of Risk
& Co’s CreditMetrics), the classic reference Stock Exchange (NYSE). The NYSE fell Management in Financial Institutions,
being Merton, R., 1974, “On the pricing more than 1,000 points, subsequently Volume 3, Number 4, Autumn 2010.
of corporate debt: The risk structure of recovering to its previous value in only
interest rates,” Journal of Finance 29(2): 15 minutes time. “Findings regarding the 7. “Range of practices and issues in
449–470. Access at: http://dspace.mit. market events of May 6, 2010, Report economic capital frameworks – final
edu/bitstream/handle/1721.1/1874/ of the staffs of the CFTC and SEC to the paper,” Basel Committee on Banking
SWP-0684-14514372.pdf?sequence=1. Joint Advisory Committee on Emerging Supervision, March 2009. Access at:
Regulatory Issues.” Securities & Exchange http://www.bis.org/publ/bcbs152.htm.
2. Refer to the following regulatory guidance; Commission and Commodity Futures
“Principles for Sound Stress Testing 8. Refer to the following regulatory guidance;
Trading Commission, September 30, 2010. “Principles for Sound Stress Testing
Practices and Supervision - Consultative Access at: https://www.sec.gov/news/
Paper,” The Basel Committee on Banking Practices and Supervision - Consultative
studies/2010/marketevents-report.pdf. Paper,” The Basel Committee on Banking
Supervision, May 2009. Access at: http://
www.bis.org/publ/bcbs155.htm. Academic, 5. A classic example in the field of credit Supervision, May 2009. Access at: http://
“A coherent framework for stress testing,” risk is the structural modeling framework, www.bis.org/publ/bcbs155.htm. Academic,
Jeremy Berkowitz, Journal of Risk, Winter which underlies any of the leading models “A coherent framework for stress testing,”
1999/2000. Access at: https://www.risk. used in the industry (e.g., JP Morgan Chase Jeremy Berkowitz, Journal of Risk, Winter
net/data/basel/pdf/basel_jor_v2n2a1.pdf. & Co’s CreditMetrics), the classic reference 1999/2000. Access at: https://www.risk.
being Merton, R., 1974, “On the pricing net/data/basel/pdf/basel_jor_v2n2a1.pdf.
3. “Automated Decision Making Comes of of corporate debt: The risk structure of
Age,” Thomas H. Davenport and Jeanne interest rates,” Journal of Finance 29(2):
G. Harris, MIT Sloan Management 449–470. Access at: http://dspace.mit.edu/
Review, July 2005. Access at: http:// bitstream/handle/1721.1/1874/SWP-0684-
sloanreview.mit.edu/article/automated- 14514372.pdf?sequence=1. “Supervisory
decision-making-comes-of-age/. Guidance on Model Risk Management,”
The Board of Governors of the Federal
Reserve System, Office of the Comptroller
of the Currency, SR 11-7, April 4, 2011.
Access at: http://www.federalreserve.
gov/bankinforeg/srletters/sr1107a1.pdf.
15
About the Authors Stay Connected About Accenture
Luther Klein Accenture Finance & Risk Services Accenture is a leading global professional
Luther is a Managing Director and leads www.accenture.com/us-en/financial- services company, providing a broad range of
the North America Finance and Risk services-finance-risk-business-service.aspx services and solutions in strategy, consulting,
Analytics organization. He brings sizable digital, technology and operations. Combining
experience from industry and consulting Connect With Us unmatched experience and specialized
partnering with over 30 financial services www.linkedin.com/ skills across more than 40 industries and
companies to deliver projects across analytic groups?gid=3753715 all business functions—underpinned by the
modeling (CCAR, credit/market/operational world’s largest delivery network—Accenture
risk, Basel, RWA), governance, validation, Join Us works at the intersection of business and
operating model design, investment www.facebook.com/accenturestrategy technology to help clients improve their
governance, M&A and risk management. www.facebook.com/accenture performance and create sustainable value
for their stakeholders. With approximately
Michael Jacobs Jr, PhD, CFA Follow Us 373,000 people serving clients in more than
Mike is Principal Director, Accenture www.twitter.com/accenture 120 countries, Accenture drives innovation
Risk Analytics. He has over 25 years of to improve the way the world works and
financial risk modeling and analytics Watch Us lives. Visit us at www.accenture.com.
experience in industry and consultancy. www.youtube.com/accenture
Specialized in model development and
validation for CCAR, PPNR, credit/market/
operational risk; Basel and ICAAP; model risk
management; financial regulation; advanced
statistical and optimization methodologies.
Mike has presented at high profile industry
venues and has been widely published in both
refereed practitioner and academic journals.
Akber Merchant
Akber Merchant is a Senior Manager,
Accenture Finance & Risk Services.
Based in New York, Akber has solid
consulting and industry experience in
financial services and risk management.
He has worked with global and regional
financial institutions across North America,
Asia and Europe. His work in enterprise
risk management, regulatory compliance,
liquidity risk management, capital
management and analytics helps clients
become high-performance businesses
while meeting regulatory mandates.
Disclaimer
This document is intended for general informational
purposes only and does not take into account the
reader’s specific circumstances, and may not reflect
the most current developments. Accenture disclaims,
to the fullest extent permitted by applicable law, any
and all liability for the accuracy and completeness of
the information in this document and for any acts or
omissions made based on such information. Accenture
does not provide legal, regulatory, audit, or tax advice.
Readers are responsible for obtaining such advice from
their own legal counsel or other licensed professionals.