Level III

Evaluating Portfolio Performance

www.ift.world

Graphs, charts, tables, examples, and figures are copyright 2014, CFA Institute.
Reproduced and republished with permission from CFA Institute. All rights reserved.
Contents
1. Introduction
2. The Importance of Performance Evaluation
3. The Three Components of Performance Evaluation
4. Performance Measurement
5. Benchmarks
6. Performance Attribution
7. Performance Appraisal
8. The Practice of Performance Evaluation

www.ift.world 2
1. Introduction
• Performance evaluation is the ex post analysis of investment
performance
 Performance measurement
 Performance attribution
 Performance appraisal

• Fund sponsors and investment managers are concerned with

performance evaluation
 Fund sponsors are owners of large pool of investable assets

www.ift.world 3
2. The Importance of Performance Evaluation
• Fund sponsor perspective:
 What is the fund’s performance relative to
investment objectives Feedback
 What are the investment program’s strengths and and Control
weaknesses Mechanism
 What are the successful and unsuccessful strategies
• Investment manager perspective:
 Virtually all fund sponsors will insist on
performance evaluation Feedback
 Determine effectiveness of various elements of and Control
investment process and examine relative Mechanism
contributions of those elements

www.ift.world 4
 3. Three Components of Performance Evaluation
Account: one or more portfolios managed by one or more investment managers

Three questions related to investment performance of an account:

Measurement

Attribution

Appraisal

www.ift.world 5
4. Performance Measurement
• Rate of return with no external cash flows
• Rate of return with cash flows at start or end of period
• Total rate of return
• Time weighted rate of return
• Money weighted rate of return
• TWR versus MWR
• Linked internal rate of return
• Annualized return
• Data quality issues

www.ift.world 6
www.ift.world 7
www.ift.world 8
Total Rate of Return

Prior to 1960s performance measurement focused on income

Since then the focus has shifted to total rate of return which measures
increase in wealth due to income and capital gains

Two major methods:

Time-weighted rate of return (TWR) reflects the compound rate of growth of

$1 invested at T = 0

Money-weighted rate of return (MWR) measures compound growth rate of all

funds invested in the account over the evaluation period

www.ift.world 9
Review of TWR

www.ift.world 10
 Time (Days) Cash Flow Account
Value
Starting Balance 1,000,000
T=5 + 30,000 1,045,000
T = 16 + 20,000 1,060,000
Final Value (T = 30) 1,080,000

www.ift.world 11
Time (Days) Cash Flow Account
Value
Starting Balance 1,000,000
T=5 + 30,000 1,045,000
T = 16 + 20,000 1,060,000
Final Value (T = 30) 1,080,000

Wealth relative

www.ift.world 12
Time (Days) Cash Flow
Starting Balance +1,000,000
T=5 + 30,000
T = 16 + 20,000
Final Value (T = 30) 1,080,000

www.ift.world 13
 TWR versus MWR
TWR MWR
• Represents growth of a single unit of • Represents average growth of all
currency invested money invested
• Unaffected by external cash flows • Sensitive to size and timing of external
• Appropriate measure if investment cash flows
manager has little or no control over • Appropriate measure if investment
external cash flows manager has control over timing of
• Generally required under GIPS® external cash flows (for example with
• Requires valuation on every day that private equity)
an external cash flow takes place • Requires valuation at start and end of
period
TWR can be approximated by calculating the MWR over reasonably frequent time
intervals and then chain linking those returns  Linked Internal Rate of Return (LIRR)
Example 7
www.ift.world 14
 Under “normal” conditions TWR and MWR
will produce similar results

www.ift.world 15
www.ift.world 16
Data Quality Issues
• Quality of performance management process depends on quality of input
data
• For accounts invested in liquid and transparently priced securities, reported
rates are likely to be reliable
• For accounts invested in illiquid and infrequently priced assets, the
underlying valuations may be suspect
 Estimated prices may be derived based on dealer-quoted prices for similar assets 
matrix pricing
• Should have appropriate data collection procedure; stated account value
should:
 Reflect impact of unsettled trades
 Reflect income owed to or by the account

www.ift.world 17
 5. Benchmarks
A few different ways to think of a benchmark…
• Collection of securities or risk factors and associated weights
that represent the persistent and prominent investment
characteristics of an asset category or a manager’s investment
process

• Passive representation of manager’s investment style

• Manager’s area of expertise

www.ift.world 18
P = Portfolio Return
B = Benchmark Return
M = Market Return

P = B + (P – B)

P= B + A

P = M + (B – M) + A

P=M+S+A

www.ift.world 19
www.ift.world 20
Properties of a Valid Benchmark

(Area of expertise)

www.ift.world 21
 Types of Benchmarks
Benchmark
Absolute. An absolute return is
the return objective.
Manager Universes. Median
manager or fund from a broad
universe.
Broad Market Indexes.
Style Indexes. Represent specific
portions of an asset category.
Advantages Disadvantages
Simple Not investable and does not
satisfy benchmark validity criteria
Simple to understand Fails most benchmark validity
Measurable criteria
Well recognized, easy to At times manager’s style might
understand, widely available and differ from style reflected in a
satisfies most properties of a valid market index
benchmark
Well recognized, easy to Might not pass tests of
understand, widely available benchmark validity; certain
weights might be too high; style
might be ambiguous
www.ift.world 22
 Types of Benchmarks
Benchmark Advantages Disadvantages

Factor-Model-Based. Use a set of factor Capture systematic Not intuitive: very few think in
exposures as a benchmark. sources of return terms of factor exposures when
Market Model: Example 10 Easy to see manager’s designing a portfolio
Multi-factor Models investment style Not easily investable
Normal portfolio
Returns-Based. Benchmark constructed Easy to use Might hold positions that
using 1) series of manager’s account returns Intuitive manager finds unacceptable
and 2) series of returns on several Useful when only Requires many months of data
investment style indexes over the same information is account
period. Then identify combination that most return information
closely tracks the account’s returns.

Custom Security Based. Represents Satisfies all validity Expensive to construct and
manager’s research universe weighted in a criteria maintain
particular fashion. Not published and might lack
transparency
www.ift.world 23
Market Model

Multi-Factor Model

Normal portfolio is a portfolio with exposures to sources of systematic risk

that are typical for a particular manager

www.ift.world 24
Building Custom Security-Based Benchmarks

www.ift.world 25
Critique of Manager Universes as Benchmarks
• Placing above the median of a universe of investment managers
is a reasonable objective, but it is NOT a suitable performance
benchmark because:
 It can not be specified in advance
 It is not investable
 It is not unambiguous (who’s the median manager? Is style appropriate?)

• Manager universes are subject to survivorship bias

www.ift.world 26
 Tests of Benchmark Quality
Criteria Comments
Systematic Biases Minimal systematic biases or risks in the benchmark relative to the account
Historical beta of account relative to benchmark ≈ 1 on average

Manager’s ability to identify attractive and unattractive investment opportunities

should be uncorrelated with whether the manager’s style is in or out of favor
relative to overall market
Correlation between A = (P – B) and S = (B – M) ≈ 0 on average

Tracking Error Benchmark should capture important aspects of manager’s investment style
Volatility of active returns (P – B) should be low relative to volatility of (P – M)

Risk Characteristics Account’s exposure to systematic sources of risk should be similar to those of the
benchmark over time

www.ift.world 27
 Tests of Benchmark Quality
Criteria Comments
Coverage Coverage = proportion of portfolio market value that is contained in the
benchmark
High coverage is good  strong correspondence between manager’s universe and
benchmark

Turnover Benchmark turnover = proportion of benchmark’s market value allocated to

purchases during periodic rebalancing of benchmark
Low turnover is better; otherwise investability is impacted
Positive Active Active position = security weight in portfolio – weight in benchmark
Positions Largely positive active positions is good
Largely negative active positions implies that benchmark is a poor representation
of manager’s investment approach (this shows that manager has no investment
opinion on many securities)

www.ift.world 28
Hedge Funds and Hedge Fund Benchmarks
• With long-short hedge funds the net value of
the portfolio is very small; hence, standard
return measures don’t work
• We need another performance measure

• Hedge fund definition is vague which makes

it difficult to identify suitable benchmarks
• Sharpe ratio is often used and compared
with Sharpe ratio of other hedge funds
 Comparing with median performance has issues
 Use of σ as measure of risk is problematic
because of high skewness of returns

www.ift.world 29
6. Performance Attribution
• Performance attribution is the comparison of an account’s
performance with that of a designated benchmark and the
identification and qualification of sources of differential returns

• Macro attribution: performance attribution at the fund sponsor

level

• Micro attribution: performance attribution at the investment

manager level

www.ift.world 30
Impact Equals Weight Times Return
• Two possible reasons for a positive active return
1. Selecting superior performing assets
2. Owning superior performing assets in greater proportion relative to the benchmark

Example 11

• Assets can be divided or combined into all sorts of categories: economic

sectors, financial factors, investment strategies, etc.

www.ift.world 31
 Macro Attribution Overview
• Account refers to total fund consisting of
investments in various asset categories

• For each category we can have multiple

investment managers

• Performance attribution can be carried out based

on a rate of return metric

• It is useful to also think about performance

attribution in monetary terms

www.ift.world 32
Macro Attribution Inputs

www.ift.world 33
www.ift.world 34
 Conducting a Macro Attribution Analysis
Apply increasingly complex investment strategies and observe the incremental value-add

www.ift.world 35
Investment Strategies
• Risk-Free Asset: Assumes that everything is invested in the risk-free asset

• Asset Categories: Assumes funds are invested in asset categories per policy
allocation

• Benchmarks: Measures impact of the managers’ investment styles

• Investment Managers or Value of Active Management

• Allocation effect is a reconciliation factor (it is a plug)

www.ift.world 36
Micro Attribution Overview
Micro attribution based on sector weighting/stock selection
• Investment returns of individual portfolios relative to designated benchmarks
• Think of a portfolio as a collection of sectors which in turn are a collection of
securities
• Manager’s value-add is based on sector selection and security selection within
each sector
Micro attribution based on fundamental factor model
• Security-by-security micro attribution is difficult if you have a large number of
securities
• The alternative is to use a factor model
• Factors represent common elements with which security returns are
correlated and can be defined in many ways
 Sector or industry membership variables
 Financial variables such as balance sheet or income statement items
 Macroeconomic variables such as changes in interest rates, inflation or economic growth
 Movement of a broad market index
www.ift.world 37
 Stock Selection Micro Attribution

Portfolio

Stock A: Weight = 0.8

Stock B: Weight = 0.2
A returns 30%
B returns 0%
Benchmark

Stock A: Weight = 0.5

Stock B: Weight = 0.5

www.ift.world 38
www.ift.world 39
 Sector Weighting Micro Attribution

Benchmark

Sector A: Weight = 0.5

Sector B: Weight = 0.5
A returns 30%
B returns 0%
Portfolio

Sector A: Weight = 0.8

Sector B: Weight = 0.2

www.ift.world 40
Sector Weighting/Stock Selection Micro Attribution

www.ift.world 41
 Example 13

Example 14 & 15

www.ift.world 42
www.ift.world 43
www.ift.world 44
www.ift.world 45
 Fundamental Factor Model Micro Attribution
• Decide which factor model to use
• At start of evaluation period, determine exposure of the portfolio and the
benchmark to the factors of the fundamental factor model
• At end of evaluation period, determine performance of each factor

Example 12 Market Model

Portfolio Benchmark

www.ift.world 46
 Example 16

Normal portfolio return

represents manager’s
investment style

1 Skill measured by:

Actual Portfolio Return –
Normal Portfolio Return
= 6.02 – 5.85 = 0.17%

Can’t be explained by
3 factor model; hence
4 attributed to investment
www.ift.world manager 47
Fixed Income Attribution
• Some concepts from sector weighting/stock selection can be
applied to fixed income, but…

• The model needs to be enhanced to consider the major

determinants of fixed income returns such as changes in:
 General level of interest rates (represented by shifts in the treasury yield curve)
 Sector spreads
 Credit quality

www.ift.world 48
Consider the AA-rated 10-year industrial bond sector
Two major determinants of return are:
1. changes in interest rates and
2. changes in nominal spread

www.ift.world 49
www.ift.world 50
 Captures effect of sales and
purchases over a given period:
total portfolio return – all other
components

www.ift.world 51
 Broughton claims expertise in:
1) Interest rate management
2) Security selection

Mathews claims expertise in:

1) Identifying undervalued sectors

Do results validate their claims?

www.ift.world 52
 7. Performance Appraisal
• Risk-Adjusted Performance Appraisal Measures
 Ex Post Alpha
 Treynor Measure
 Sharpe Ratio
 M2
 Information Ratio

• Quality Control Charts

• Interpreting the Quality Control Chart

www.ift.world 53
Risk-Adjusted Performance Appraisal Measures
Ex Post Alpha

Treynor Measure

Sharpe Ratio

M2

www.ift.world 54
M2 and Sharpe ratio will evaluate manager skill in the same way

Treynor Measure and Ex Post Alpha will evaluate manager skill in the same way

It is possible that M2/Sharpe and Treynor/Ex Post give us a different conclusion

when manager takes a large amount of non-systematic risk

We can think of the Sharpe ratio as representing self-

financing strategy where we borrow at the risk free rate

Sharpe ratio can be generalized to use the manager’s

benchmark rather than the risk free asset: this gives us
the information ratio: active return/active risk

www.ift.world 55
Quality Control Charts
• Quality control charts help us evaluate an active manager’s performance
relative to his benchmark
• Three assumptions underlying quality control charts
 Null hypothesis: manager has no investment skill
 Manger’s value-added returns are independent from period to period and are normally
distributed around expected value of 0
 Manager’s investment process does not change from period to period

Say σ of value-added returns = 4.1%

Ex ante, 80% of expected returns

will be 1.28 σ from mean  +/-
5.2%

www.ift.world 56
www.ift.world 57
www.ift.world 58
8. The Practice of Performance Evaluation

www.ift.world 59
www.ift.world 60
 Manager Continuation Policy
Some fund sponsors have adopted formal, written manager continuation policies (MCP) to guide
their manager evaluations

The purpose of a MCP is as follows:

www.ift.world 61
 Manager Continuation Policy
MCP can be viewed as a two-part process: manager monitoring and manager review

www.ift.world 62
 Manager Continuation Policy as a Filter
• Divide managers into three categories
 Positive value-add  beat benchmark by 2% per year on average
 Zero value-add
 Negative value-add  Lose to their benchmark on 1 percent per year on average
• We can view MCP as a statistical filter designed to remove negative-value added
managers retain positive value-added managers
• Two types of decision errors may occur
 Type I error: keep managers with zero value-add
 Type II error: reject managers with positive value-add

www.ift.world 63
Manager Continuation Policy as a Filter
• Fund sponsor must determine how fine a filter to construct
• Course filter  More Type I errors
• Fine filter  More Type II errors
• Both types of errors are expensive
• Control probabilities of Type I and Type II errors by adjusting width of
confidence band within quality control chart
• Many fund sponsors endure the discomfort of keeping several unskillful to
avoid the expense of firing a truly superior manager

www.ift.world 64
Conclusion
• Examples

• Summary

• Practice Problems

• Learning Objectives

www.ift.world 65