Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Examination Committee
Chairperson: Prof. Horácio Cláudio de Campos Neto
Supervisor: Prof. Rui Fuentecilla Maia Ferreira Neves
Members of the Committee: Prof. Aleksandar Ilić
November 2015
Resumo
A otimização de regras de negociação que se adaptam às mudanças nos mercados tem
merecido crescente atenção por parte do mundo financeiro e por parte dos académicos. Este
trabalho propõe uma metodologia para criar regras de negociação com a característica de
seguir tendências e que combina várias escalas temporais. As regras de negociação são
baseadas em indicadores técnicos tais como o RVI e o MACD e são individualmente otimizadas
para 19 índices bolsistas e 11 mercadorias no período entre 2006 e 2014. São usadas
estruturas em árvore para representar as regras que estas são gradualmente melhoradas
através de algoritmos evolucionários. As melhores soluções para negociar estes ativos são
depois simuladas individualmente e em Portfolios.
A utilização de duas escalas temporais permite a redução do risco visto que são combinados
dois perfis de negócios diferentes, naquilo que poderá ser designado por diversificação de
sistemas de negociação. Um inovador algoritmo para eliminar soluções semelhantes é
também apresentado, usando o Erro Quadrático Médio dos sinais gerados.
As regras de negociação geradas por este método obtêm lucros em tendências ascendentes e
descendentes reagindo rapidamente a quedas fortes. Nas quedas dos mercados em 2008 as
regras otimizadas não só impediram quebras fortes no capital como conseguiram lucrar com
as quedas.
i
ii
Abstract
The optimization of trading rules that adapt to changing market conditions has been subject
to increasing attention from the financial world and from academics. This work presents a
methodology to develop trading rules with a trend-following approach combining several time
frames. Rules are based on technical indicators like the RVI and MACD and are individually
optimized for 19 stock indices and 11 commodities in the period from 2006 to 2014. Tree
structures are used to represent the trading rules which are gradually evolved through
evolutionary algorithms. The best solutions to trade these assets are then simulated
individually and in portfolios.
The use of two time frames allows a reduction in risk since two different profiles of trades are
combined in what can be described as trading system diversification. An innovative algorithm
to delete similar rules is also presented, based on the Mean Squared Error of the generated
trading signals.
The trading rules obtained by this method are able to profit from upward and downward
trends and react fast to sharp falls. In the bear market of 2008 the optimized rules not only
prevented sharp drops in capital but managed to profit from the declining prices.
iii
iv
Acknowledgements
I thank my supervisor Rui Neves for his guidance, support and feedback. I am grateful for his
openness and flexibility, allowing me a great deal of freedom in my work. I also thank my co-
supervisor, Nuno Horta, for his input and feedback.
I thank my family for her support, especially my mother for her continued support in all
domains and my young son, Miguel.
v
vi
Dedication
I dedicate this work to my father, Fernando Machado.
vii
viii
Table of Contents
RESUMO............................................................................................................................................................ I
ACKNOWLEDGEMENTS .................................................................................................................................... V
1 INTRODUCTION ....................................................................................................................................... 1
ix
3.1.6 Genetic Programming ..................................................................................................................... 33
3.1.6.1 Representation ....................................................................................................................................... 33
3.1.6.2 Genes...................................................................................................................................................... 35
3.1.6.3 Reproduction Operators ......................................................................................................................... 36
3.1.7 Data organized in vectors................................................................................................................ 37
3.1.8 Interpretation of Genomes .............................................................................................................. 38
3.1.9 String representation of Genomes .................................................................................................. 39
3.1.10 Enforcing Trend-following rules .................................................................................................. 41
3.1.11 Evaluating the performance ....................................................................................................... 42
3.1.11.1 Sharpe Ratio and Sortino Ratio .............................................................................................................. 42
3.1.11.2 Risk Return Ratio .................................................................................................................................... 43
3.1.11.3 Modified Risk Return Ratio ..................................................................................................................... 44
3.1.11.4 Lake Ratio ............................................................................................................................................... 44
3.1.12 Fitness Function .......................................................................................................................... 45
3.2 PORTFOLIO MODULE................................................................................................................................. 45
3.3 TECHNICAL INDICATORS AND RULES .............................................................................................................. 46
3.3.1 Moving Averages ............................................................................................................................. 46
3.3.2 Price Channels ................................................................................................................................. 47
3.3.3 MACD .............................................................................................................................................. 48
3.3.4 Relative Volatility Index ................................................................................................................... 50
3.3.5 Stop and Reverse ............................................................................................................................. 51
3.3.6 Composite indicators ....................................................................................................................... 52
3.4 CHAPTER CONCLUSIONS............................................................................................................................. 53
4 IMPLEMENTATION ................................................................................................................................. 55
5 RESULTS ................................................................................................................................................. 63
x
5.5.1 Example Genome from S&P500 ...................................................................................................... 71
5.6 EQUAL WEIGHT PORTFOLIO (REFERENCE) ....................................................................................................... 72
5.7 SETUP 10 PORTFOLIO ................................................................................................................................ 73
5.8 SETUP 10 PORTFOLIO - LONG AND MEDIUM TERM COMPONENTS ....................................................................... 74
5.9 CHAPTER CONCLUSIONS............................................................................................................................. 76
REFERENCES ................................................................................................................................................... 81
xi
xii
List of tables
Table 1 – Array to be optimized (element of the population). ................................................ 17
Table 2 – Element from the population representing a possible portfolio. ............................ 20
Table 3 – Overview of different approaches for single asset optimization. ............................ 21
Table 4 – Overview of different approaches on portfolio optimization. ................................. 25
Table 5 – Meaning of outputs from a chromosome that produces long and short signals .... 38
Table 6 – Combination of outputs in sub-systems with two chromosomes ........................... 38
Table 7 – Logical table of using Majority Vote on 3 single indicator rules .............................. 52
Table 8 – Results from optimization with all operators (3 levels) ........................................... 64
Table 9 – Results from optimization with Boolean operators (3 levels) .................................. 65
Table 10 – Average results for setups with depth 2 and Boolean genes. ................................ 66
Table 11 – Average results for setups with depth 1 and Boolean genes. ................................ 67
Table 12 – Results from optimization with setup 10 – best genes with tree depth 1 ............. 69
Table 13 – Results summary of an equal weight portfolio ...................................................... 72
Table 14 – Results summary of portfolio with setup 10 optimizations ................................... 74
Table 15 – Annualized percentage returns of portfolio with setup 10 optimizations ............. 74
Table 16 – Results summary comparing portfolios with different time frames ...................... 75
Table 17 – Annualized percentage returns from portfolios with different time frames ......... 75
Table 18 – List of stock indices used ........................................................................................ 85
Table 19 – List of commodities used ........................................................................................ 85
Table 20 – Gene groups and characteristics ............................................................................ 87
Table 21 – Description of LONG Technical Primitive genes ..................................................... 90
Table 22 – Performance metrics calculated for Long, Short, and Global trades ..................... 94
Table 23 – Performance metrics calculated exclusively for global trades ............................... 94
xiii
xiv
List of figures:
Figure 1 – Triangle formation in EUR/USD, 4H time frame ....................................................... 8
Figure 2 – Example of a resistance that becomes support. ....................................................... 8
Figure 3 – Example of trading with Parabolic Stop and Reverse (PSAR).................................... 9
Figure 4 – Types of trends ........................................................................................................ 11
Figure 5 – Basic flowchart of a Genetic Algorithm ................................................................... 14
Figure 6 – Tree structure of a GP computer program .............................................................. 14
Figure 7 – Example of the crossover breeding operator ......................................................... 15
Figure 8 – Example of the mutation operator. ........................................................................ 15
Figure 9 – A tree genome and the equivalent portfolio. ......................................................... 22
Figure 10 – Overall architecture of the proposed solution ...................................................... 27
Figure 11 – Sliding Window Optimization ................................................................................ 28
Figure 12 – Global flowchart of the optimization of an asset .................................................. 29
Figure 13 – Flowchart of the Genetic Programming optimization of one window ................. 30
Figure 14 – Example of algorithm to eliminate similar genomes ............................................ 31
Figure 15 – S&P500 chart with the signals of a trend following trading system with two
subsystems ................................................................................................................... 32
Figure 16 – Example of a genome structure ............................................................................ 34
Figure 17 – Structure of a Chromosome .................................................................................. 34
Figure 18 – Gene details ........................................................................................................... 35
Figure 19 – Process of obtaining the trading signal vectors .................................................... 39
Figure 20 – Lake Ratio graphical representation ..................................................................... 44
Figure 21 – 100-day Exponential Moving Average applied to the close of the S&P500 index 47
Figure 22 – Price Channel applied on S&P500 ......................................................................... 48
Figure 23 – MACD applied to a chart of S&P500 index............................................................ 48
Figure 24 – Relative Volatility Index in an S&P500 chart ......................................................... 50
Figure 25 – SAR Line applied to the S&P500 index .................................................................. 51
Figure 26 – Results for S&P 500 ............................................................................................... 68
Figure 27 – Results for Brent .................................................................................................... 70
Figure 28 – Results for Palladium ............................................................................................. 71
Figure 29 – Equal weight portfolio ........................................................................................... 72
xv
Figure 30 – Returns of portfolio with optimizations obtained with setup 10. ........................ 73
Figure 31 – Portfolio with setup 10 optimizations – medium-term only ................................ 74
Figure 32 – Portfolio with setup 10 optimizations - long-term only ........................................ 75
Figure 33 – Generic structures of the cache ............................................................................ 92
xvi
List of Acronyms:
TA Technical Analysis
FA Fundamental Analysis
GA Genetic Algorithms
GP Genetic Programming
EA Evolutionary Algorithms
MT Medium Term
LT Long Term
DJI Dow Jones Industrial Average (American Stock Index)
S&P500 Standard & Poor’s 500 (American Stock Index with 500 stocks)
ETF Exchange Traded Funds
B&H Buy and Hold strategy
MPT Modern Portfolio Theory
ROI Return On Investment
xvii
ROA Return on Assets
CCI Consumer Confidence Index
CPI Consumer Price Index
GDP Gross Domestic Product
xviii
1 Introduction
Markets fulfill a basic necessity of mankind and exist in many parts of the world since
thousands of years. They allow the trade of goods and services between those who have them
and those who wish to acquire them. As time progressed, markets became more regulated,
sophisticated, liquid, global and more accessible. At present time, stock, commodity, financial
and currency markets have a crucial role in our economy and life, moving incredible amounts
of money each day. With the democratic access to markets, the general public can also
participate (mostly in stock markets) which means that crises in the markets can directly affect
millions of people.
In consequence of the markets being so central and important in our economies and societies,
they have been studied for many decades and many theories have been proposed. Some
argue markets are efficient [1] meaning that all relevant information is already incorporated
in the prices and it’s impossible to predict its future movements. This supposedly leads to a
market behavior known as Random Walk [2]. Others refute this view and defend the market
is not efficient and it’s possible to predict the future behavior of the markets to some extent
[3]. Increasing evidence point to markets not being efficient which means that methods can
be developed to use relevant information to increase profits and/or reduce risks [3], [4].
When market participants try to predict the future behavior of an asset, two very different
approaches have been used. One, called Fundamental Analysis [5] looks at the quality of the
asset or its intrinsic value and determines if it’s undervalued or overvalued. And the other
major market analysis method used is called Technical Analysis [5], [6], which studies past
prices on the premise that all relevant information is reflected there and that it’s possible to
predict it’s the future behavior by analyzing the patterns formed by price.
In the field of technical analysis, several techniques were developed in order to extract the
useful information contained in past prices. Some try to identify graphical formations that
correspond to a higher probability of a particular outcome. They are called Chartists. Others
use technical indicators calculated with past prices, which filter relevant information that can
then be used to decide what asset to trade and when.
Each market has its own particular characteristics and certain trading techniques used in one
market may not work well in other markets. Also, each market may exhibit different behavior
in different phases, consequence of underlying factors, trader psychology of simply as a result
of system dynamics. In summary, each market is different and markets change over time. This
leads to the necessity of adapting the trading approach or the trading rules as the market
changes, to maximize profits and minimize risk. In the past decades, much attention has been
devoted by academics to the problem of optimizing trading rules. Several optimization
algorithms have been experimenting, like Neural Networks, Fuzzy Logic, Monte Carlo, Particle
Swarm, Hidden Markov Models, Support Vector Machine and Evolutionary Algorithms.
1
Evolutionary Algorithms are particularly suited to optimize trading rules because of their
power and flexibility and is the optimization method used in this thesis. Trading rules
exclusively based on technical indicators will be optimized with Genetic Programing, a form of
Evolutionary Algorithm. Several chosen assets (stock indices and commodities) will be
independently optimized and portfolios will be simulated using those optimized trading rules.
In order to increase the robustness of the solutions, the optimization will be restricted in order
to produce solutions which follow market trends, since that is one of the most structural and
frequent market patterns. Since markets can exhibit different behaviors in different time-
scales, rules tuned to different time frames will be combined in order to increase the
robustness of the solutions.
A secondary goal is to combine the individual and independent optimizations of the assets
and simulate a portfolio using the best generated rules.
3
4
2 Related Work
This section presents several fundamental concepts necessary to better understand the work
that was developed. Some theories regarding the behavior of financial markets are addressed
like the random walk theory and the efficient market hypothesis. Technical Analysis (TA) and
Fundamental Analysis (FA) are briefly described and compared since they are important tools
in the analysis of financial assets and portfolio optimization. Evolutionary Algorithms (EA) and
in particular Genetic Algorithms (GA) and Genetic Programming (GP) are presented as good
choices for the optimization of trading rules and portfolios. Several relevant papers are
reviewed and help to contextualize the work developed here.
In an efficient market, price variations are independent from each other and past data can’t
be used to predict future data - asset prices assume a random walk behavior [2]. This has
serious implications for technical analysis since its premise is precisely that past data can be
used to predict the probable future behavior of the market [1].
In an efficient market, fundamental analysis can still be useful if the investor can interpret the
relevant information faster than the other market participants and take advantage of the
difference between the current asset price and the intrinsic value he calculates. After the
market integrates the new information, it’s allegedly impossible to beat the buy-and-hold
strategy.
5
The random walk theory in financial markets was first studied by French stockbroker Jules
Regnaul in 1863 [8] and was popularized in 1973 by Burton Malkiel [2]. Many researchers
defend markets have a high degree of efficiency and prices follow a random walk behavior,
like in [1] and [2].
The random walk theory is criticized by several researchers like in [3] where the study of
weekly stock returns can’t be explained by this theory. Most market returns don’t follow a
normal distribution having fat tails [9] which is evidence that, at least in certain periods,
markets have a non-random behavior.
The relevant facts considered to determine the intrinsic value of the asset depends on the
type of asset. For stocks, financial statements (business’s assets, liabilities and earnings),
management, business concept, markets and competition are analyzed. Several ratios can be
used to objectively evaluate the situation of the company. These ratios allow for comparison
between stocks and also with historical levels. Some examples are: Price to Earnings Ratio
(PER), Earnings per Share (EPS), PER to Growth (PEG), Dividend Yield (DY), Return on Equity
(ROE) and Price to Book Value (PBV).
If the economy as a whole is being analyzed - important for stocks, stock indices and currencies
– economic indicators issued by central banks and other institutions are studied. Some of the
most important are the Unemployment Rate, Consumer Price Index (CPI), Consumer
Confidence Index (CCI), Gross Domestic Product (GDP), Interest Rates, Public Debt, and New
Home Sales.
For the readers interested in more information on Fundamental Analysis, the classic book
from Graham and Zweig on value investing may be a good option [10]. And the book from
Bernard Baumohl [11] has many information on economic indicators, useful to understand
the economy as a whole.
6
weekly, daily or intraday prices: Open (O), High (H), Low (L) and Close (C), the Volume (V) and
Open Interest (OI) in the case of futures. In special cases, other data may be used, like
commitment of traders (futures) and prices from other assets that influence the one being
studied [5], [6].
TA assumes all relevant information for an asset is already reflected in its historical prices and
it’s the best source of useful information to predict its behavior. In the case of stocks, the
company situation and future expectations are considered to be already reflected in the stock
past prices. TA studies certain repeating patterns formed by price, consequence of
human/trader behavior, system dynamics and fundamental underlying factors.
TA was used for many decades by traders, allegedly with much success in certain cases but
wasn’t taken seriously in the academic world and by many professional fund managers.
Gradually, empirical evidence and academic work showed some forms of TA are helpful in
predicting future price action and in the past two decades it has received much attention from
researchers [4].
TA can be used in many different ways and has several sub-categories. For example, Chartists
are technical analysts that look for graphical formations in price that indicate a higher
probability of some price behavior in the future, like continuation or inversion of a trend. Some
of the most popular formations include double or triple tops and bottoms, head-and-
shoulders, triangles and cup-and-handle. Detailed information on chart formations can be
consulted in the book by Bulkowski [12]. Figure 1 shows a triangle formation which is
composed of an upper and lower line that converge as time advances. Price bounces between
those two lines until it consistently crosses one of them. When this happens, price is expected
to continue in that direction and a price projection can be calculated using the initial height of
the triangle, applied to the breaking point. Using a computer to automatically identify this
graphical formations can be a challenge since rigid rules may not be the best way to deal with
price formations that exhibit noise and ambiguities. An experienced technical trader or a
computer neural network can probably better manage these ambiguities.
Some technical analysts pay close attention to price levels where the asset has recently
reversed direction and where many investors have entered or exited the market. These levels
may have psychological and emotional relevance to the market participants and influence
future trading decisions. Since stop loss and profit taking orders may also be entered near
those levels, the market may react strongly near them and reverse direction in the short-term.
These levels are sometimes called “support” if they are below the current price and
“resistance” if they are above the current price. Figure 2 shows a chart of currency pair
GBP/USD in the 4H time frame, with three horizontal lines representing price levels where
price clearly reacted. These horizontal levels can form in all types of assets and time frames
but are more common in Forex, especially in intraday charts. With the study of relevant past
levels and the way price reacts to them, it’s sometimes possible to gain important information
regarding the probable future behavior of the asset.
7
Figure 1 – Triangle formation in EUR/USD, 4H time frame
Technical indicators calculated from raw prices is another very popular tool used by TA. Those
indicators can extract useful information from the historical data and highlight it. The list of
technical indicators is very long and include moving averages, indicators that show if there is
a trend in progress and how strong it is (like the Directional Movement Index [6]) and
indicators that show if the asset is overbought or oversold in the short-term (like the Relative
Strength Index [6] and Stochastic Oscillator [6]). Kaufman compiled many of the most
important and most used indicators in [6]. Figure 3 shows an example of a chart with a TA
8
indicator superimposed (Parabolic Stop-and-Reverse [6]) which was used to generate trading
signals.
Several technical indicators used in this work will be discussed in 3.3 as well as algorithms that
generate trading signals based on those indicators.
The two most commonly repeating market patterns, which can be identified and exploited,
are:
Trends: Price tends to move preferably in one direction for an extended period of time
(autocorrelation of price). The strategy to profit from this behavior can be called “trend
following” and consists in opening positions in the same direction of the trend for as
long as it exists.
Mean Reversion: The price tends to overshoot a mean line that can be considered as
the consensus of a fair price. The resulting behavior is an oscillating price around the
mean line. The strategy to profit from this behavior can be called “swing trading” and
consists in catching these shorter-term oscillations of the market, buying at the bottom
and shorting at the top.
These patterns tend to alternate between themselves of coexist, alongside random noise
created by the chaotic trader activity. These two patterns form repeatedly in most markets
and in most time frames. In this work great care will be taken so that the developed trading
solutions have a trend following philosophy.
9
2.1.3.1 Nomenclature and definitions
It’s important to clarify the meaning of certain words or expressions that will be used in this
text. When dealing with stocks trading, there usually are only 2 market operations: Buy and
Sell and most the investors/traders can only earn money when the markets go up. When
dealing with commodities, stock indices, financial assets or currencies, it’s possible to earn
money when the market goes up but also when it goes down and there’s no negative stigma
associated with earning money in falling markets since contrary to stock markets,
commodities, currencies and financial assets have a cyclic nature.
To profit from a rising market, the trader must “buy” to open the trade and “sell” to close it –
that is called a “long” trade. To profit from a falling market a trader must “short” to open the
trade and “cover” to close it – that is called a “short” trade. In this work the terms “buy” and
“short” will be used to refer to the action of opening a trade and the terms “long” and “short”
will be used to refer to types of trades. The term “short” will therefore be used in two similar
contexts.
TA and FA try to achieve the same using different methods: identify good trading or
investment opportunities to maximize profits. The question of which is better is probably the
wrong question since research points to both techniques being at least partially
complementary [13], [14]. This work will deal exclusively with technical analysis.
2.1.5 Trend-Following
A trend in a market is a price pattern that consists in price going consistently up or down for
some time. This movement is normally done in successive swings and not in a straight line.
Mathematically a trend corresponds to high auto-correlation of price. The slope, duration and
regularity of a trend can vary greatly. A trend can be upwards and can be referred as an “up-
trend” or “long trend” and it’s said the market is “bull” as a reference to a bull which attacks
with his horns in an upward movement. In an up-trend prices usually form higher highs and
higher lows as prices rise in a zigzag fashion, Figure 4. The trend can also be downwards and
can be referred as a “down-trend” or “short trend” and the market is said to be “Bear”. In a
down-trend prices usually form lower highs and lower lows as prices fall in a zigzag fashion,
10
Figure 4. Finally the market can have no defined trend and can be referred as a sideways,
lateral or oscillatory market, or can be said to be consolidating. Figure 4 exemplifies the 3
situations.
Trend following is a trading approach used by technical analysts who take advantage of
market trends and in simple terms consist in:
Trends exist in almost all markets but some are historically more prone to be trending than
others, which translates in better returns from trend following strategies.
Brock et al. studied two of the most popular trend following strategies: moving average (MA)
crossover and trading range breakout (TRB) [15]. The authors applied these strategies to the
Dow Jones Index and the positive results obtained contradicted the once prevalent theories
about the markets (1992), namely the Random Walk Theory.
Ülkü and Prodan studied another very popular technical indicator used in trend following
strategies: the “Moving Average Convergence Divergence” (MACD) along with MA crossover
[16]. The trading rules were applied to stock indices and results confirmed these strategies are
viable, specially the MA crossover. The authors also found volatility to be beneficial to these
trend following strategies.
11
Trends are present in all time frames, from intraday to monthly but, the higher the time frame,
the more reliable are the trends. Intraday prices usually have too much noise and trend
following strategies can be a bad choice for trading in those shorter time frames. Lohpetc and
Corne used GP to develop trading rules for S&P500 and compared several time frames: Daily,
Weekly and Monthly [17]. Their results show that solutions obtained with monthly data easily
outperform the Buy-and-Hold strategy, and as the time frame decreases to weekly and daily,
the results become weaker. Using daily data, the results were still positive but equivalent to
the B&H. This confirmed previous works using different time frames and is a strong indication
that at least in some assets the longer time frame is important and should be taken into
account when generating trading rules. These results coincide with the empirical knowledge
of experienced traders who do not trade against the longer-term trend.
Portfolios are usually constructed with a desirable risk level as reference and portfolio assets
are chosen in order to maximize the expected returns maintaining the pre-chosen risk level.
Greater returns are usually obtained at the expense of greater risk and that tradeoff must be
defined according to the risk profile of the portfolio owner.
Passive management is also strongly linked to the “Modern Portfolio Theory” (MPT) whose
major contributor was Harry Markowitz. In his 1952’s work [19] he proposed the “Mean-
Variance Model” and mathematically demonstrated that the risk of a portfolio depends on
the covariance of returns from its components and not on the average risk of each component.
This model gave managers a mathematical tool to precisely construct portfolios which in
theory have the desired balance between risk and return (highest return for a desired risk or
lowest risk for a desired return).
12
2.2.2 Active management
In active management [18] it’s assumed the market is not efficient and the portfolio
components and/or its weights are constantly updated in order to reflect the changes in the
markets. The objective it to have, at all times, a portfolio whose components have the best
potential. The methods used to evaluate the assets and construct the portfolio vary and can
be based on FA, TA or a combination of the two. The process of managing the portfolio or fund
may be totally discretionary, based on the analysis made by a person or group of persons, or
can be partially automated with information coming from TA and FA algorithms or can be
totally automated, where complex algorithms select assets, quantities and the best timings to
update the portfolio.
Evolutionary algorithms (EA) are inspired in nature, they follow Darwinian principles and have
gained much popularity in recent decades [20]. They use a population of elements, each one
representing a solution to the problem being optimized. All the elements in the population
compete between themselves and are evaluated with a fitness function. The best scored
elements usually have a greater chance of reproducing with other elements or may move
intact to the next generation (elitism). Reproduction is made using one or more operators,
usually crossover and mutation.
Evolutionary Algorithms perform global optimization with a meta-heuristic character and are
trial and error problem solvers. They are random processes and results are never guaranteed,
but they deal better than other search algorithms with the problem of getting stuck in local
maxima because they use a population with a diversity of solutions. For these reasons EA are
robust, flexible and practical, being successfully used in many optimization problems, namely
in finance [21].
GP allows more complex solutions when compared with GA. This happens because the tree
representation is much more flexible and also because it can use powerful operators like if-
then-else. This makes GP much more powerful and flexible than GA and capable of finding
novel and sometimes unexpected solutions to problems.
Its flexibility and power can also have negative consequences because it may increase the
search space and curve-fitting, which is a real danger that must be addressed.
14
Programs from one generation breed and their offspring will integrate the next generation.
The most used breeding operators are Crossover and Mutation. In crossover, a child is created
using random parts of the tree programs of two or more parents, Figure 7. In mutation a child
is created by randomly changing parts of a parent’s program tree, Figure 8.
15
2.4 Existing Solutions
The academic work in this area can be divided in two groups:
1) Works that focus on developing trading rules for a specific asset (ex: S&P500) or groups
of assets (ex: stocks from S&P500). The optimized rules generate signals to enter and
exit the market in precise moments. Results are measured in an individual basis and
usually no attempt is made to construct an optimized portfolio of assets.
Greater focus will be given to works of the first group since the main goal of this work is the
development of trading rules.
Evolutionary algorithms have been successfully used to create rules for automatic trading of
financial assets, in many cases able to beat the Buy-and-Hold strategy, which contradicts the
efficient market hypothesis. The assets chosen are mainly stocks, stock indices and forex
currencies. Usually, daily historic data from the assets is used but other time frames have been
tested, including intraday, weekly and monthly.
A sliding window methodology is frequent, meaning a portion of historical data is used for
training (training window) and a posterior portion of data is used for testing (testing window).
This procedure is repeated several times, sliding both of these windows in time.
16
2.4.1.1 Solution using Genetic Algorithms
One possible methodology to optimize trading solutions is to use GA with a pool of pre-
selected rules (with fixed parameters) that can be chosen. The number of rules can be
relatively small or very large, including variations of a basic rule, for example: a MA crossover
rule with different parameters for the MA. In this case the variable to be optimized is an array
of values that will select some of these fixed rules or that will give weights to these rules. The
final decision can be made by majority vote of the signals from the selected rules, by weighted
averaging of the signals of the rules or some other similar solution.
This optimization method doesn’t allow the creation of new elementary rules which can be a
serious limitation but, on the other hand limits the search space, curve-fitting and the
solutions found may be more robust. Other more flexible methods usually have to employ
procedures to limit the search space, or the complexity of the solutions.
Korczak and Roger used GA with fixed rules to develop trading solutions for 24 French stocks
[25]. More than 200 different fixed rules were available for selection and include: price
channel breakouts, close/MA crossing, ROC greater/smaller than a constant and rules base on
the ease of movement indicator. The optimization selects the combination of rules that
produce the best solution and the final decision is given by averaging the outputs of all the
selected rules.
Each element in the population is represented by a binary array of size N, equal to the number
of rules available. Each position in the array corresponds to one rule, which is either selected
(1) or not selected (0), Table 1.
Mendes et al. proposed a different approach, using a pre-selected set of trading rules whose
parameters are optimizable [26]. GA were used to create trading solutions for two forex pairs,
EUR/USD and GBP/USD. Each element in the population is a trading system and has a total of
10 rules, 5 for opening a trade and 5 for closing. In each of these groups of 5 rules, majority
vote is used in the first 4 to validate a signal and the fifth rules gives the price for
opening/closing the trade. Several intraday time frames were tested and positive results were
obtained in the majority of the simulations.
Fundamental Analysis can also be used to create rules that generate trading signals in the
same formats discussed so far. And it’s possible to mix TA and FA rules in the same
optimization, allowing the best combinations to be selected. This was experimented by
Contreras et al. in [14]. They used GA on 100 stocks from S&P500 with daily data from 1994
17
to 2004. Each element of the population has 23 genes/variables that can be optimized and
which are the parameters used in 8 predefined rules. 4 rules are based on TA and 4 rules are
based on FA. The TA rules use the indicators MA, MACD and RSI and also support and
resistance levels. The FA rules are based on the following ratios and indicators: PER, PBV, ROA
and Sales Growth. Each of the 8 rules has an optimizable weight associated which defines its
contribution to the final decision. The signals generated by each of the 8 rules are summed
and a final signal is validated if the sum goes above/below a threshold that can be adjusted to
obtain a more aggressive/conservative trading style.
Training was performed from January 1994 to December 2003 and the year 2004 was used for
testing. In this period the cumulative return was 830% which compares with the return of
180% obtained by the Buy-and-Hold strategy in the same period.
The operators used in the tree can be of many different types, namely, arithmetic, logic,
comparison, mathematical, based on TA and on FA. If basic arithmetic operators are used as
well as raw data (Open, High, Low, Close, Volume), the search space can become very large,
curve fitting will probably occur and solutions won’t be robust. Usually more sophisticated
operators based on TA and FA are used, extracting useful information from raw prices.
Working with processed information instead of raw historical prices reduces the search space
to areas with greater probability of containing robust solutions.
In GP, curve fitting is a real danger and must be addressed. Becker and Seshadri discuss
methods of decreasing curve fitting and methods for improving the comprehensibility of the
solution in [27]. They present three main approaches for decreasing curve fitting:
Their approach was to include a factor that penalizes complexity in the fitness function. They
compare the results of this method with an equivalent one without this factor and obtain
solutions with less complexity and more generalization power. The simpler solutions obtained
better results in the out-of-sample data and beat the buy-and-hold strategy.
To increase the comprehensibility of the solutions in tree format, they suggest using known
and proved technical indicators calculated from the historical data, instead of more general
operators. And they also suggest decreasing the complexity of the solution by the method
described above.
18
Following on their previous work [27], Becker and Seshadri used GP with two chromosomes,
one to generate buy signals and another for sell signals [28]. They opted to reproduce the two
chromosomes independently, meaning that a buy and sell chromosome will never exchange
genes and evolve separately. But since a trading solution must have synchronized buy and sell
signals to be coherent, the evolution of the two chromosomes is linked and interdependent -
the authors named this process as cooperative coevolution. There are several options to
manage the two types of chromosomes and some were tested in this work:
1) Each buy chromosome is paired with a sell chromosome in a solution and they evolve
together;
2) Each chromosome of one type is paired with the best 5 chromosomes of the other
type, from the previous generation.
3) Each chromosome of one type is paired with 5 randomly chosen chromosomes of the
other type.
The results showed option 1) with paired chromosomes performed best and the authors argue
it may mean that compatibility between the chromosomes is more important than diversity.
Tests were performed in S&P500 from 1954 to 2002.
Allen and Karjalainen used GP with tree representation to generate trading rules for American
Stock Index S&P500, tested from 1936 to 1995 [29]. The operators used were arithmetic,
logical, SMA, maximum price in an interval, minimum price in an interval, >, <, If-Then-Else
and Lag (offsets a vector received from a child node). The authors tried to limit curve fitting
by training the population in a 2 year window and then validating the result in another 2 year
window. When the best element has a better fitness score in the validation window, it is
saved, and otherwise it’s discarded. Optimization stops after a predefined number of
generations with no improvement in the validation window. S&P500 historical prices are
normalized, dividing them by the 250 day MA. The results of this configuration did not beat
the buy-and-hold strategy.
Majority vote: the solution obtained with each fitness function votes with a buy or a
sell and the final decision is given by the majority vote of all simulations with all the
objectives;
Normal: the capital is equally allocated to all the simulations.
19
weekly data gives worse results but still outperforms B&H. Using daily data the results are
equivalent to B&H.
2.4.1.3 Conclusions
The reviewed works on single asset optimization are summarized in Table 3. It’s difficult to
make direct comparisons between the methods used since they tend to use different assets,
different date intervals and sometimes different data time-scales. Nonetheless it’s possible to
make several observations useful to the present work and some will be discussed in 2.5.
Evolutionary algorithms are a good choice to solve the Portfolio Optimization problem since
they are flexible and can efficiently search a large solution space. An example can be found in
[32] where Lin and Gen use GA and follow a Mean-Variance approach, inspired in the work of
Markowitz. Each element in the population is represented by an array of weights, one for each
candidate stock to integrate the portfolio, Table 2.
Stock index 0 1 2 3 4 5 … N
Weight 0.02 0.04 0.00 0.11 0.02 0.09 … 0.04
Table 2 – Element from the population representing a possible portfolio.
20
Paper Optim. Operators / Fitness Time Assets Test
Ref Representation Results
date Method Genes function Frame used Period
24
More than 200 1997 -
[25] 2002 GA Array Several tested Daily French Beats B&H
predefined rules 1999
Stocks
Predefined genes with 2005 -
[26] 2012 GA Array Stirling Ratio Intraday FOREX Beats B&H
optimized parameters 2010
Several based on TA S&P500
[14] 2012 GA Array Cumulative returns Daily 2004 Beats B&H
and FA Stocks
ARITM, Logical, SMA, Excess return over 1936 -
[29] 1999 GP Tree Daily S&P500 Doesn’t beat B&H
MAX/MIN, IF, >, <, Lag B&H 1995
ROC, MA, Logical, >, <, 1990 -
[27] 2003 GP Tree N/A Monthly S&P500 Beats B&H
O,H,L,C, price levels 2002
AND, OR, NOT, >, <, Tree. Not mentioned - Some configurations
1954 -
[28] 2003 GP Monthly O, H, L and C, 2 includes complexity Monthly S&P500 beat B&H; Paired
2002
MA, ROC, S&R. chromosomes penalization Chrom. Perform better
Sets
AND, OR, >, <, MA, Daily,
Multiple from Monthly and Weekly
[17] 2010 GP ROC, “Price Resistance Tree Weekly, S&P500
components. 1960 to beat B&H.
Ind., “Trend Line Ind.” Monthly
2008
MA, ROC, MA
Daily,
GP Multi- Min/Max, Logical, >, <, S&P500 Multi-objective
[30] 2011 Tree Multiple Weekly, N/A
Objective O, H, L, C, V, Trend Line stocks superior. Beats B&H
Monthly
Indicator
Table 3 – Overview of different approaches for single asset optimization.
21
The fitness function is calculated dividing the total expected return by the risk of the portfolio.
No attempt is made to predict the future behavior of the assets by using TA or FA and the
main difference from the Markowitz approach is the use of GA to perform the optimization.
Aranha and Hitoshi proposed a different representation of the portfolio in [33], using a tree
structure as seen in Figure 9. The weight in a node refers to the left side of its sub tree and the
complement of that value is the weight for the right sub tree. The final weight of each asset is
given by the sum of the weights of all occurrences of that asset in the tree. For example, the
weight of a2 represented in Figure 9 is equal to [0.5 * 0.2 * (1 - 0.3)] + [(1 - 0.5) * (1 - 0.7)] =
0.07 + 0.15 = 0.22. The authors argue this representation allows the preservation of building
blocks which accelerates the optimization process. The results showed the portfolios
generated with this methodology have fewer assets compared with the ones generated with
the array of weights representation. Having fewer assets may have some benefits like reduced
transaction costs and better understandability of the portfolios.
Gorgulho et al. used GA to optimize a portfolio composed of stocks from the Dow Jones
Industrial Average (DJI) and S&P500. They represented the solutions as arrays of real numbers
which correspond to the weights given to 6 TA rules used in the scoring of stocks. The rules
are pre-defined and based on:
22
1) MACD Histogram crossing the 0 level;
2) Hull MA slope change;
3) EMA/Close crossing;
4) ROC crossing the 0 level;
5) RSI crossing overbought/oversold levels;
6) MA crossover.
The rules output is averaged according to the weights and used to score the stocks, as shown
in equations ( 1 ), ( 2 ) and ( 3 ).
𝑁
𝑆𝑐𝑜𝑟𝑒(𝑋) = ∑ 𝑤𝑖 ∙ 𝑆𝑐𝑜𝑟𝑒(𝑋, 𝑖) (1)
𝑖=0
(2)
0 ≤ 𝑤𝑖 ≤ 1
𝑁
0≤∑ 𝑤𝑖 ≤ 1 (3)
𝑖=0
Where:
All the stocks with a score higher than a pre-defined level are included in the portfolio whose
performance is evaluated using the ROI. Three simulations were performed: the first using the
optimization described above, the second using a Buy-and-Hold approach and the last using
random portfolios. The optimization with GA obtained the best results: ROI = 3.49% against
0.49% for the B&H and 0.90% for the Random approach.
Silva et al. proposed the use of GA with rules based on both TA and FA to optimize trading in
S&P500 stocks [35]. For a stock to be included in the portfolio, several conditions must be
verified: a TA rule must give a positive signal (crossover between price and a SMA) and be well
ranked among all the stocks. The ranking is done using a weighted average of several FA ratios.
The chromosomes include the weights for the FA ratios, the period of the SMA and several
23
other parameters like a minimum score to enter the portfolio, a Take Profit level, a Stop Loss
Level and the capital to allocate to each stock. The portfolios were evaluated using ROI and
the Variance. Several variations were tested in 2012-2013 and the results obtained beat the
index of reference (S&P 500) in the same period.
2.4.2.3 Conclusions
The works on portfolio optimization are summarized in Table 4 and again it’s difficult to make
direct comparisons since the assets and dates tested are different. Using the Markowitz Mean
Variance Model is the classic approach for portfolio optimization but there are indications that
using technical analysis to extract useful information may allow the creation of portfolios with
higher returns.
There are indications that using data from higher time scales (like weekly and monthly) tends
to increases returns [17], and this should be taken into account in the present work (more
than one time frame will be used simultaneously). Majority vote was used in some of the
works as a way to process multiple sources of information [26], [30] and is a path worth
exploring when creating more complex trading rules.
Genetic Programming with rules encoded in a tree structure allows for many combinations
which may easily generate curve-fitting and measures must be taken to address this problem,
similar to the ones employed in [27].
24
Paper Optim. Portfolio Operators / Chromosome Fitness Time Assets Test
Ref Results
date Method Analysis Genes structure function Frame used Period
Total return / Taiwanese
[32] 2007 GA Markowitz (Asset weights) Array Daily N/A Not specified
portfolio risk Stocks
Asset weights and Nasdaq and 2004 - Portfolios with
[33] 2008 GA Markowitz Tree structure Sharpe Ratio N/A
asset indexes NIKKEY stocks 2006 fewer assets
Arithmetic,
Boolean,
24 futures 1980 - No evidence of
[36] 2005 GP TA Inequalities, LAG, Tree structure Profitability Daily
contracts 2000 profitability
Min, Max, AVG and
others.
Information
Stocks from Higher returns
Variable Ratio, Omega 2006 -
[34] 2012 GA TA N/A Daily Dow Jones and higher risk
length array ratio, Sortino 2011
Industrial than DJI
Ratio
Weights of TA rules Stocks from Beats B&H and
2006 -
[37] 2009 GA TA used in the scoring Array ROI Daily DJI and Random
2007
function S&P500 portfolios
Parameters used Daily,
S&P500 2012 - Beats reference
[35] 2015 GA TA and FA for ranking and Array ROI, Variance Trimestral
Stocks 2013 index
trading. (FA)
Table 4 – Overview of different approaches on portfolio optimization.
25
26
3 Solution’s Architecture
This section explains the proposed solution, first in general lines and then in detail. The work
can be divided into two main components. The first consists in the optimization of trading
rules for individual assets (stock indices and commodities) and is the main focus of this work.
The second consists in simulating several portfolios using the trading rules optimized in the
first component and is a complement of the main objective.
The overall architecture of the solution can be seen in Figure 10. A user interface allows the
user to select which module to use and which configuration file should be loaded.
When the Optimization module is chosen it loads the global settings and asset specific settings
from files and also loads the historical data for the assets to be optimized. After this
initialization process, the optimization begins, optimizing one asset at a time independently
from all other assets. The optimization produces detailed results that can be later processed
and analyzed. It also produces results formatted for use by the Portfolio module.
When the Portfolio module is chosen, it loads settings from a configuration file and also loads
historical data from the assets it will use in the portfolio simulation. Several portfolios are then
created:
27
3.1 Optimization Module
The optimization module independently optimizes trading rules for several assets which is
done sequentially following a list specified in a configuration file. The solutions optimized have
a trend-following philosophy and combine two subsystems, one focusing on the medium term
and the other on the long term. The optimization is done with a sliding window approach, with
successive windows containing in-sample and out-of-sample data. The trading rules are
represented in tree structures and are evolved using genetic programing. Results are saved
for later visualization and processing and the trading signals of the best solutions for each
asset can also be later used to construct portfolios. Several aspects of the optimization process
are discussed in more detail in the following sections.
Optimization is first performed in an in-sample window and the best rules obtained are
applied to the corresponding out-of-sample window. Then the optimizer moves to the next
pair of in-sample/out-of-sample windows and the process is repeated until there is no more
data. A flowchart of this process is presented in Figure 12.
28
Figure 12 – Global flowchart of the optimization of an asset
The complete results of the optimization of an asset is composed of the results from the
several individual windows. Each out-of-sample window has associated a different set of rules
(best genome) that was obtained in the optimization of the corresponding in-sample window.
The complete trading results of an asset, are obtained after all windows have been optimized
and each out-of-sample window has its associated genome.
29
Figure 13 – Flowchart of the Genetic Programming optimization of one window
The elimination of similar genomes is not based on the tree structure of the chromosomes
since different chromosome trees can produce similar or equivalent results. The chosen
method is to look at the Mean Squared Error (MSE) of the vectors with trade signals produced
by the genomes, Figure 14. The algorithm starts at the most scored genome and compares its
signals with the ones from the less scored genomes until a distance of N genomes. If the MSE
is less than an arbitrary threshold, the less scored are eliminated. After having compared the
first genome with all the genomes below up to a distance of N, the second genome is
compared to the less scored ones in a similar fashion. The process is repeated with all genomes
until the end of the list.
The reason why genomes are not compared with all other genomes in the list is that it would
be computationally expensive because of the large number of Mean Squared Error
calculations. The presented solution seems to be a good compromise between efficacy in
removing similar genomes and the computational cost.
In this work the value chosen for N is 10 which demonstrated to be a good compromise
between efficacy in eliminating similar genomes and the computational cost associated with
the algorithm.
30
Figure 14 – Example of algorithm to eliminate similar genomes
Market trends are patterns that are well studied and exist in almost all markets and
exist since stock exchanges were created.
Literature shows it is possible to profit from trends in a consistent way using
mechanical trading rules.
It allows to strongly reduce the search space and avoid curve fitting. The optimizer will
be discouraged to profit from short term patterns that probably won’t be present in
the near future.
It can be hypothesized that in the market data, there is a signal corresponding to the
underlying factors that produce a trend, and there is noise corresponding to the contradictory
activity of all the market participants. In that case, the higher the time frame, the higher the
signal to noise ratio (SNR) one should expect since the noise produced by market activity has
more impact in the shorter time frames. With a higher SNR the trading rules will produce less
false signals and generate higher returns.
31
This effect observed in longer time frames (weekly and monthly) can be emulated with daily
data but using different parameters in the technical indicators that generate the trading
signals. For example, a simple moving average of 10 periods using weekly data, can be roughly
emulated in daily data using a period 5 times greater: 5 x 10 = 50.
In Figure 15 we have an example chart showing the S&P500 index with data that includes the
2008 violent bear market. In the top part of the chart we can see the signals generated by a
mechanical trading system which follows trends. This trading system has two components or
subsystems whose signals are then combined with equal weights to generate the global signal
of the system. The top line in green is the medium term subsystem and is the most responsive,
reacting to some S&P500 mild corrections. The middle signal line in blue corresponds to the
long-term subsystem which is much more stable and only changes position when the
movement is confirmed. The third line from top in red is the global output of the trading
system and is calculated giving weights of 0.5 to both subsystems. Each subsystem is either
LONG (1) or SHORT (-1) and when the two subsystems disagree, the global system is neutral.
The main chart background color changes according to the global system signal: green for
long, pink for short and yellow for neutral.
Figure 15 – S&P500 chart with the signals of a trend following trading system with two subsystems
32
This example shows it is possible to create trading rules which are stable and immune to daily
noise despite using daily data. This simplifies the optimization process by using data from only
one time frame. And at the same time allows the combination of several sub-systems sensitive
to trends in different time frames.
Preliminary studies show this type of system gives good results in stock indices and in some
commodities. There are in fact two trading systems trading in parallel, with different trade
profiles, different return profiles and different risk profiles, which average and complement
each other. The long-term subsystem tends to produce higher returns since it’s more stable
but tends to have higher drawdowns since it takes more time to react to market change. On
the other hand, the medium-term subsystem tends to have a lower return since it reverses
positions more frequently but, on the other hand will probably have lower drawdowns since
it reacts faster to market change. The system resulting from the combination of these two
components will tend to be more balanced and tend to have lower drawdowns than either of
the two subsystems alone. This approach will be studied, testing several configurations and
trying to find optimized solutions.
3.1.6.1 Representation
An element in the population is referred as a genome and it contains all the information
necessary to produce trading signals and trade an asset. A genome is composed of one or
more sub-systems as can be seen in Figure 16. A sub-system is a set of rules that produce
trading signals with a certain profile (ex: long-term trades) and contributes with a certain fixed
weight to the global output of the system. In the example of Figure 16 the genome is
composed of 3 sub-systems with weights 40%, 30% and 30%. The first two sub-systems have
a specific chromosome for long trades and another for short trades, but the third sub-system
only has one chromosome that generates signals for both long and short trades.
33
Figure 16 – Example of a genome structure
Chromosomes contain the actual rules organized in a tree structure and are integrated into a
sub-system, Figure 17. Each element in the tree is called a node or a gene. There are operator
genes (ellipses in Figure 17) which perform operations and deliver the result to its parent
node. And there are terminal genes which feed information to operator genes (rectangles in
Figure 17). Some terminal genes perform operations on pre-defined inputs so they don’t need
to be specified, ex: Relative Strength Index (RSI) which always uses the Close prices. Some
genes have parameters like the RSI and others don’t, like the operator AND.
Genes have inputs and outputs of type double or Boolean. For example, the Exponential
Moving Average (EMA) has an input double and output double; the operator AND has both
input and output of type Boolean; and the operator Greater than (>) has inputs double and
output Boolean.
34
3.1.6.2 Genes
Each node in a chromosome tree is a gene and can either perform an operation or be data
used by operators, Figure 18.
Gene group – genes from the same group share several characteristics. Ex: Arithmetic
operators or logic operators.
Type of Input data – if the gene has inputs (child nodes) the data it receives can be of
type Double or Boolean. For ex, the Simple Moving Average (SMA) gene has one input
of type Double.
Type of output data – The gene outputs data of type Double or Boolean. For example,
the SMA output is of type Double.
Number of inputs (number of child nodes). Ex: The SMA has one input which is a vector
of type Double from which the SMA is calculated.
Parameters – Some genes don’t have parameters and others have several. Ex: The
SMA has one parameter which is the period.
A set of possible genes is previously defined from which they are chosen when a chromosome
is constructed or mutated. Not all combinations are possible since parent-child genes have to
be compatible: a child gene must output same type of data used by the parent gene as input.
Genes are grouped according to their function and each group share several characteristics,
see Table 20 in Appendix B – Groups of genes.
35
A gene/node with inputs of type double must obviously have child nodes with outputs of type
double and the equivalent if the inputs are of type Boolean. The genes of type Technical
Primitive assume a very important role since they are complete trading rules and a
chromosome can have a single gene of this type. They can have complex rules coded inside
and output the corresponding signals. For example, the gene with id string “TMP_110112”
produces signals which are the result of the majority vote of 3 sub-rules. Some of these genes
are based on rules studied by other authors with good results and others are the result of
preliminary tests which showed promising results. Some of these Technical Primitive genes
don’t have parameters and therefore are not optimizable but, the chromosome and genome
still are.
Elitism – In each generation the genomes representing the 5% most scored are copied
unchanged to the next generation.
Crossover – Two parent genomes are randomly selected from the list of available
genomes ranked by decreasing fitness score. The probability of a genome being chosen
decreases exponentially with its decreasing position in the list. And the most scored
genome is 5 times more likely of being chosen than the last one. The crossover point
is chosen randomly in the first parent genome. The probability of a gene being chosen
as crossover point decreases exponentially from terminal node to root node and a
terminal node is 5 times more likely to be chosen than the root node. A compatible
sub-tree is then randomly chosen from the second parent and inserted in the crossover
point. Figure 7 shows an example of the crossover operator.
Soft Mutation – A single parent genome is randomly chosen as descried for the
crossover operator and only the parameters of the genes are mutated. The
chromosome tree structure remain unchanged. The probability of a particular gene in
a chromosome being mutated is proportional to the number of genes in the
chromosome. Each chromosome has the probability of having 1.0 genes mutated.
36
individually, and a ranked list created for each chromosome position. Then the best
chromosome of each list is picked and a new genome created.
Crossover, Hard Mutation and Soft Mutation are mutually exclusive and only one is chosen
with probabilities 0.5, 0.3, 0.2 respectively. In these 3 operators Genomes are reproduced
chromosome by chromosome and a chromosome from a particular position in the genome
never exchanges genes with a chromosome in a different position. In the Crossover and Hard
Mutation operators, not all chromosomes from a genome are reproduced and the probability
of a particular chromosome being reproduced is 0.1. If a chromosome is not reproduced, it is
copied intact from parent 1.
Asset data is also stored in vectors, one for each of the following fields:
When the rules in a chromosome are applied to a window of data (interpretation), the output
is a vector with the same dimension as the data window, containing the trading signals
produced by the chromosome. In this work chromosome outputs are always Boolean, being
either true of false.
Long only chromosome – true means buy and false means neutral.
Short only chromosome – true means Short and false means neutral.
Long and short chromosome – true means Long and false means Short.
Some trading rules can be implemented in a sub-system with a single chromosome that
produces trades in the long and short side, alternating between the two and never being
neutral, Table 5.
37
Long/Short chromosome Output
False SHORT
True LONG
Table 5 – Meaning of outputs from a chromosome that produces long and short signals
Other trading rules must be implemented in a sub-system with two chromosomes, one for
long trades and another for short trades. In this case the outputs are combined to give the
final output, Table 6.
A typical genome will have two subsystems, one of them will try to detect the long-term trend
of the market and usually needs only one chromosome, alternating long and short positions.
The second subsystem is usually medium-term, using one or two chromosomes depending on
the trading rules.
The method used for simulating a genome is to previously obtain a vector of trading signals
generated by the rules of the genome when applied to a particular asset. Each element in
these vectors corresponds to a trading day and contains the market position that the system
should assume. This market position ranges from -1.0 (100% short) to 1.0 (100% long), being
0 a neutral position.
Subsystem 0:
Chromosome k0:
[genes from chromosome k0]
[…]
39
Chromosome kn:
[genes from chromosome kn]
[…]
Subsystem N:
Chromosome p0:
[genes from chromosome p0]
[…]
Chromosome pn:
[genes from chromosome pn]
Each gene has an identification (id) string that identifies it, for example, the id of the Simple
Moving Average gene is “SMA”. Each chromosome is converted to string starting by the root
node (top) and going down the tree in a Depth-First-Search fashion: Tries to go down the tree
as much as possible, first by the left side. When a terminal gene is found, backtracks and tries
the branch to the right or backtracks again if there are no more branches. Repeats the process
until all genes are processed.
Each gene is printed in an independent line and the start position of the string is advanced
with TAB characters, as many as the depth level of the gene in the chromosome tree (0 for the
root gene). If the gene has parameters, they are inserted by order in front of the gene string,
inside round brackets.
For example, the chromosome in Figure 17 has 3 levels (0, 1 and 2) and would have the
following string representation:
AND
>
Close
EMA (200)
Close
<
RSI (14)
40.0
As can be seen, it’s easy to interpret the chromosome and understand its logic: To generate a
signal (root gene returns TRUE) two conditions must be verified (AND). The first is the close
must be greater (>) than the Exponential Moving Average of 200 periods of the close. And the
second condition is that the Relative Strength Index of 14 periods is smaller (<) than 40.
40
3.1.10 Enforcing Trend-following rules
Since the goal of this work is to study the optimization of trading rules that use a trend-
following philosophy, it’s necessary to implement measures to enforce that outcome. This is
done by limiting the search space, eliminating many combinations of genes that would
produce other trading patterns different from the desired one. In essence, we want rules that
are capable of detecting a consistent movement of prices in one direction and open a trade in
the same direction. The criteria to decide when a consistent movement/ trend is present and
what technical indicators to use, in what configuration and with which parameters, is what
the optimization will do.
In practice, we want to eliminate, for example, a rule that buys when the close price drops
below a moving average of the same close, because that means the price is dropping (the
moving average is slower that the close). The optimization setup presented in 5.2, with a large
search space, allow for such rules. In preliminary testes, it was determined that the best way
to proceed was to use simple chromosome trees, with few levels or even just one level and
use mainly technical primitive genes. These genes allow a greater control on the creativity of
the optimizer because the basic format of the trading rule is already defined and only the
parameters can be changed. Even with this restrictions, the optimizer has several genes to
choose from and several parameters to adjust.
Another important restriction that was adopted was to define allowed ranges for the gene
parameters. And to define different ranges for medium-term and long-term subsystems. What
this means is that the same genes will have different allowed ranges of parameters depending
on the subsystem on which they are used.
Since it was decided to combine several subsystems that respond to trends in different time
frames (ex: medium and long-term), it’s necessary to adapt the parameters to that behavior.
A possible option to achieve this result would be to use the same parameters but use data
with a different period, for example daily data for the medium term and weekly data for the
long-term. The solution used in this work was to emulate the behavior of the indicators with
weekly data by changing the allowed parameters and making them less reactive. In practice
the ranges were empirically chosen using some preliminary tests. For example, the gene with
id “C>SMA” which compares the Close with the Simple Moving Average of the Close allows a
period for the SMA in the range [50, 200] for medium term chromosomes and [100, 400] for
long term chromosomes.
A different approach to enforce the differentiation of trading rules of the several subsystems
(medium term and long term) was to enforce a range for the average number of trades
produced by each subsystem. When a genome is simulated with the asset data, its subsystems
are also individually simulated. From this simulation, many performance parameters are
obtained and one of them is the annualized average number of trades. If the subsystem has
an average number of trades than is outside the allowed range, the corresponding genome is
penalized in its fitness score. The greater the departure from the allowed range, the greater
41
the penalty. In this way, the offending genomes are not completely removed from the survival
race but are strongly discourage to go in that direction. The allowed range for the annualized
average number of trades is [3.0, 8.0] for the medium term and [0.0, 3.0] for the long term.
These empirically defined ranges are not critical and small variations have little impact on the
results.
𝑟𝑝 − 𝑟𝑓 (4)
𝑆=
𝜎𝑝
Where:
𝒓𝒑 is the expected portfolio return;
The Sharpe ratio is criticized because it doesn’t differentiate between positive and negative
volatility, and positive volatility shouldn’t be penalized. This lead to the creation of the Sortino
ratio [6] which is similar to the Sharpe ratio with the difference that in the denominator
42
appears an equivalent of the standard deviation that only accounts for the negative volatility:
Target Downside Deviation. The calculation can be seen in equation ( 5 ).
𝑅−𝑇 (5)
𝑆=
𝐷𝑇𝑇
Where:
R is the portfolio average realized return;
T is the target return, chosen by the investor as the reference;
TDD is the Target Downside Deviation.
The Target Downside Deviation is calculated as shown in equation ( 6 ).
𝑁
1 (6)
𝐷𝑇𝑇 = √ ∑(𝑀𝑖𝑛(0, 𝑋𝑖 − 𝑇))2
𝑁
𝑖=1
Where:
𝑿𝒊 is the 𝑖 𝑡ℎ return;
N is the total number of returns;
T is the target return.
𝑴𝑫𝑫 is the Maximum DrawDown in the period. The DrawDown on a particular day is
the difference between the peak return achieved in the past and the current return.
An account that had 1000 units as maximum and drops to 500 has a drawdown of 500
or 50%.
The Risk Return Ratio is similar to other ratios like the Calmar ratio, the Sterling ratio and the
Burke ratio.
43
3.1.11.3 Modified Risk Return Ratio
In some situations the Risk Return Ratio is not appropriate, for example, let’s imagine we use
the RRR as the fitness function of an evolutionary algorithm trying to evolve trading rules.
What can happen is that Maximum Drawdown (MDD) values close to zero will produce very
high RRR values. In the extreme, if the rules produce a single short term winning trade, the
MDD can be zero and the RRR will be infinite. In practice what this fitness function may
encourage are very low MDD because it’s an easier way of getting high fitness scores.
A modification is desired to prevent this behavior that was observed in preliminary tests
performed by the author. What is suggested is the addition of a constant in the denominator
of the RRR formula, making impossible for it to be zero and creating a balance between the
importance of high returns and low MDD, equation ( 8 ).
𝑅𝑒𝑡𝑢𝑟𝑛 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒) (8)
𝑅𝑅𝑅 =
𝐾 + 𝑀𝐷𝐷 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒)
Where K is a reference Drawdown. We suggest a value of 5 but it should depend on the size
of the test window amongst other things. In a larger window the MDD is expected to be higher.
44
The Lake Ratio is calculated dividing the Water area by the Land area, equation ( 9 ).
𝑊𝑎𝑡𝑒𝑟 (9)
𝐿𝑎𝑘𝑒 𝑅𝑎𝑡𝑖𝑜 =
𝐿𝑎𝑛𝑑
Sometimes the inverse of the Lake Ratio is used so that higher returns correspond to higher
values of the ratio. In this work the inverse of Lake Ratio will be used.
This ratio may suffer from the same problem mentioned regarding the Risk Return Ratio when
applied as fitness function of an optimizer. It may stimulate trading rules that perform few or
no trades in order to have a very low drawdown and achieve a good score. Another issue with
this ratio is that, as the window size increases, the Land area tends to increase at a faster pace
than the Water area. This prevents the comparison of the ratios between different windows.
𝑎𝑛𝑛𝑢𝑎𝑙𝑖𝑧𝑒𝑑 𝑅𝑒𝑡𝑢𝑟𝑛 % ( 10 )
𝐹𝑖𝑡𝑛𝑒𝑠𝑠 𝑠𝑐𝑜𝑟𝑒 = 𝐽𝑅4 =
5 + 𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝐿𝑜𝑠𝑠 %
Where:
The constant 5 in the denominator is empirical and not critical to the results. Its main
function is, as stated before, to discourage solutions that achieve a high fitness score by
making few trades resulting in a very small maximum drawdown.
45
synchronization described in 4.5.1. After all data and all vectors from all assets are
synchronized, a portfolio is simulated by trading all assets simultaneously using the
corresponding trading signals (using equal capital). Results are then compared with the
previously simulated reference portfolio.
Simple Moving Average (SMA) which is a Finite Impulse Response (FIR) filter where all
N values used in its calculation have the same weight.
Weighted Moving Average (WMA) also called Linearly Weighted Moving Average
(LWMA) which is a FIR filter where the N elements used in the calculation have weights
that decrease in an arithmetic progression.
Exponential Moving Average (EMA) which is an Infinite Impulse Response filter where
weights decrease exponentially never reaching zero. The exponential curve of the
weights is controlled by a parameter (alpha).
Figure 21 shows a chart of the S&P500 index with a 100-day Simple Moving Average (SMA)
superimposed in blue. An upper and lower band calculated from the SMA are also plotted in
red (SMA value +/- 3%). The moving average rule structure used in this work consists in
triggering a buy/short signal when the close price crosses above/below the MA value (light
blue ellipses in Figure 21). A drawback of this method is that the close price can oscillate
around the MA line and generate several undesirable contradictory signals. One way to limit
this problem is to use an upper and lower band based on the moving average that follow it at
a distance and filter the signals. In this band configuration, a buy signal is generated when the
close price crosses the upper band from below (green circles in Figure 21) and a short signal
is generated when the close price crosses the lower band from above (red circles in Figure 21).
46
The distance of the bands can be fixed by a percentage, or dynamic, based on the volatility of
the asset.
All types of moving averages can be used in this type of rules and obviously the period can
be optimized as can the distance of the bands when used. The genes that use this rule
structure have ID strings “C>SMA”, “C<SMA”, “C>WMA”, “C<WMA”, “C>EMA”, “C<EMA” for
simple crossover rules. And “C>SMAB” and “C<SMAB” for band crossover rules. See
Appendix C – Technical Primitive Genes.
Figure 21 – 100-day Exponential Moving Average applied to the close of the S&P500 index
47
Figure 22 – Price Channel applied on S&P500
3.3.3 MACD
The MACD technical indicator stands for Moving Average Convergence/Divergence and was
created by Gerald Appel [6] in the late 70’s, being one of the most popular technical indicators.
The complete MACD is composed of 3 values which can be used in different ways. Figure 23
shows the MACD applied to a chart with the S&P500 index.
The main line of the indicator is called MACD Line (blue line in Figure 23) and is the difference
between two exponential moving averages (EMA) with different periods, calculated from the
48
close prices (the commonly used periods are respectively 12 and 26). The formula of the MACD
line can be seen in equation ( 11 ).
Where:
𝑭𝒂𝒔𝒕𝑬𝑴𝑨(𝒄𝒍𝒐𝒔𝒆) is the EMA with the smallest period: fast_per (reacts faster).
Corresponds to the green line in Figure 23.
𝑺𝒍𝒐𝒘𝑬𝑴𝑨(𝒄𝒍𝒐𝒔𝒆) is the EMA with the longest period: slow_per (reacts slower).
Corresponds to the orange line in Figure 23.
The second MACD value is called the Signal Line and is an exponential moving average of the
MACD Line (red line in Figure 23). The formula is presented in equation ( 12 ). The most
commonly used period of the EMA is 9.
The Third MACD value is called MACD Histogram (gray bars in Figure 23) and is the difference
between the MACD Line and the Signal Line. The formula can be seen in equation ( 13 ).
The MACD indicator can be used in several ways, for example, the crossing of the MACD Line
and the Signal Line can signal a turning point and can be used to trade short term oscillations
(swing trading). It can also be used to detect trends by looking at the MACD Line that can be
above or below 0, signaling a possible up/down trend. In this work MACD rules will have this
later configuration and a buy signal will be generated when the MACD Line crosses the 0
level from below (green circles in Figure 23) and a short signal will be generated when the
MACD Line crosses the 0 level from above (red circles in Figure 23). The classic parameters of
12, 26 and 9 mentioned before can obviously be optimized. In this work, genes with this
MACD rule structure have the ID string “MACD>0” and “MACD<0” respectively for long and
short genes, see Appendix C – Technical Primitive Genes.
49
3.3.4 Relative Volatility Index
The Relative Volatility Index (RVI) was created by Donald Dorsey in 1993 and refined in 1995
[40]. It’s calculated in a similar way to the Relative Strength Index (RSI) but measures the
Standard Deviation of High and Low prices. It oscillates between 0 and 100, being 50 a neutral
value.
The RVI reacts fast when prices move with high volatility, signaling the probable start of a new
consistent movement. The calculations of the RVI can be consulted in the publication where
the indicator was presented [40]. Figure 24 shows the Relative Volatility Index in an S&P500
chart, in the configuration used in this work.
The figure shows three horizontal lines, a neutral one in the 50 level representing a neutral
situation, and upper threshold level (54 in this example) and a lower threshold level (46 in this
example). Instead of buy and short signals being generated by crossing of the neutral line,
which causes many undesirable false signals, the threshold levels are used instead. A buy
signal is generated when the RVI line crosses the upper threshold level from below (green
circles in Figure 24) and the system should remain long until a short signal is generated. A
short signal is generated when the RVI line crosses the lower threshold from above (red circles
in Figure 24) and the system should remain short until a buy signal is generated. This rules
produce a system that is constantly in the market, reversing positions, waiting for a longer and
stronger move. While the system waits for these longer movements, it’s likely it will sustain
several small losses.
The classic parameters for the RVI is a smoothing period of 14 and a standard deviation of 10
periods. These parameters can obviously be optimized, especially the smoothing period. In
50
this work, genes with this RVI rule structure have the ID string “RVI>X” and “RVI<X”
respectively for long and short genes, see Appendix C – Technical Primitive Genes.
The most known SAR indicator is the Parabolic Stop and Reverse [6] where the initial distance
between the close and the SAR line is a parameter and the SAR line tracks the close price at
an increasingly faster pace until a reverse happens.
Figure 25 shows a SAR Line applied to the S&P500 index, with buy signals in green circles and
short signals in red circles. The rules of this particular SAR can be consulted in [41] and they
will be the ones used in this work. The main difference from the Parabolic SAR is that the
Line doesn’t accelerate towards the price and it adjusts to the volatility of the asset. The
genes using this SAR line have the ID strings “SAR_S” and “SAR_L” for short and long genes
respectively, see Appendix C – Technical Primitive Genes.
51
3.3.6 Composite indicators
So far all the mentioned rules are based on a single indicator but it is possible to combine
several indicators into a composite indicator. Each of the single indicator rules focus on a
particular characteristic or behavior of the price, like momentum, volatility, price action and
when a trend forms, all give a signal, some sooner and some later. All of them will also produce
false signals some of the time that result is losses. The advantage of combining several
indicators into a single rule is to filter some of the individual false signals by waiting for the
consensus of at least the majority of the individual indicators.
A practical way to achieve this result is to use majority vote on the outputs of the single
indicator rules. For example, if we use 3 single indicator rules and apply majority vote, we have
the results observed in Table 7. Another option is to only validate a signal when all 3 inputs
have a signal, which corresponds to being more selective.
Single Indicator Rule 1 neutral signal neutral signal neutral signal neutral signal
Single Indicator Rule 2 neutral neutral signal signal neutral neutral signal signal
Single Indicator Rule 3 neutral neutral neutral neutral signal signal signal signal
Composite Output neutral neutral neutral signal neutral signal signal signal
Table 7 – Logical table of using Majority Vote on 3 single indicator rules
This majority vote operation is performed by genes with ID string “MVOTE3” and “MVOTE5”
corresponding to 3 and 5 inputs respectively. These genes have a single parameter which is
the number of active inputs necessary to validate the output.
There are two composite genes which use this majority vote technique. The first is a long-term
only gene without optimizable parameters, with ID string “TLP_100223_S” and
“TLP_100223_L” for the short and long versions. This composite rule is the result of previous
preliminary tests performed by the author. Internally, there are 3 single indicator rules:
Relative Volatility Index in the configuration presented in 3.3.4. The RVI period is 52
and the thresholds are 42 and 58.
Simple Moving Average with a band crossed by the close, as presented in 3.3.1. The
SMA period is 290 and the bands are separated by a distance equal to 2.5 times the
Average True Range [6] of period 60.
Based on the Aroon indicator [6]. Two 190-day SMAs are calculated from the Aroon up
line and from Aroon down line. The Aroon period is 16. Signals are generated when
the two SMAs cross.
The second composite gene is medium-term only, without optimizable parameters, with ID
string “TMP_110112_S” and “TMP_110112_L” for the short and long versions. This gene is the
result of preliminary tests done by the author. Internally it has 3 single indicator rules:
52
Relative Volatility Index in the configuration presented in 3.3.4. The RVI period is 35
and the thresholds are 48 and 52.
Simple Moving Average with a band crossed by the close, as presented in 3.3.1. The
SMA period is 60 and the bands are separated by a distance equal to 2.5 times the
Average True Range [6] of period 60.
Based on the Dynamic Momentum Index (DMI) [6] in an equivalent configuration used
for the RVI presented in 3.3.4. The DMI uses a period of 14 and the result is smoothed
by a SMA of period 30. The threshold levels used to generate signals are 48 and 52.
There was an important goal when the architecture of the proposed solution was developed,
which was to have a tool with great flexibility, allowing the simulation of many different types
of setups and not being limited by technical constraints. As the work later developed, practical
decisions were made relative to the solutions used and only a fraction of the potential of this
solution was actually used.
Several technical indicators employed in this work were addressed in this chapter as well as
some algorithms that can be created using those indicators, in order to generate trading
signals. Those algorithms detect trends and generate trading signals that allow profits to be
made by following the trends.
53
54
4 Implementation
In this section detailed information is given on the setups used to produce the results and how
they were implemented.
The trading rules were optimized using genetic programming coded in C++. All code was
developed with the help of the Integrated Development Environment (IDE) “Qt Creator”
version 3.3.0 (based on Qt version 5.5.0) [42]. Library MathGL version 2.3 [43] and library FLTK
1.3.3 [44] were used to produce charts with results and the Boost C++ library was used for
several auxiliary tasks [45]. The code was developed and run on a Linux machine with a core
i3 2350M CPU, 4GB of RAM and running Ubuntu Desktop 14.10 LTS 64 bits [46].
Depending on the setup used, the optimization of trading rules for the 30 assets took between
8 and 30 hours. The simulation of portfolios only takes a few seconds since all trading rules
have already been optimized and all the trading signals were saved to files.
4.1 Assets
It was decided to use classes of assets that tend to produce good results with trend following
strategies: stock indices and commodities. The selection method was composed of several
steps. Firstly, stock indices from all major markets with quality data available were pre-
selected as were all major commodities with quality data. A total of 34 stock indices and 18
commodities were obtained and from this group the final selection was made. The criteria
used for the stock indices was to select the major ones from each continent. From Europe, all
the EURONEXT indices (CAC40, BEL, AEX and PSI20), the British FTSE 100 and the German DAX.
In North America, the S&P500 and NASDAQ 100 from USA, the TSX from Canada and the IPC
from Mexico. In South America only the BOVESPA from Brazil and MERVAL from Argentina
were available and were selected. From Oceania the Australian AORD was chosen and from
ASIA 6 major indices were selected including the Japanese Nikkei. A total of 19 indices were
selected which are listed in Table 18 from Appendix A - Used Assets. To select the
commodities, the used criteria was the trade volume [47] amongst the ones available in the
pre-selection. The four most traded commodities from each of the following categories were
selected: Energy, Metals and Agriculture. (Only 3 options were available from the Energy
category). A total of 11 commodities were selected and are listed in Table 19 from Appendix
A - Used Assets.
The assets historical data was collected from several sources and include values for Open,
High, Low, Close and Volume. Volume wasn’t always available with quality and is not used in
the trading rules. The majority of data from stock indices was collected from “Yahoo Finance”
[48] using the software tool “ML Downloader” [49], saved in Comma Separated Value format
(csv). Data was visually inspected with software “MetaStock” [50] in order to detect possible
55
errors. When important errors were found and it was not possible to correct them, the asset
was discarded from the pre-selection.
All the Commodities historical data was obtained from the website “Quandl” [51] and consists
of continuous futures contracts created from individual contracts. The creation method used
was Backwards Ratio Adjusted, with Rollover on the first month. Some commodities had data
which wasn’t good enough in terms of quality (only close information in some periods) and
those commodities were discarded. Open interest was ignored.
4.2 Cache
Since many genomes have repeated chromosomes and genes, many calculations would be
repeated when the genes, chromosomes, sub-systems and genomes are interpreted. To
increase the efficiency of the optimization, a cache for vectors was implemented and is
described in Appendix D – Cache of vectors.
There is a third window not mentioned so far, the warm-up window, which corresponds to
asset data that must exist before the in-sample window. It is required by indicators that need
a certain amount of data to calculate its first value. For example, a simple moving average of
56
100 days, needs 100 days to calculate the first value. The warm-up window has a fixed size of
504 bars/days which corresponds to around 2 years in trading days. The existence of this
window doesn’t affect the qualitative comprehension of the presented solution and is a
technical detail.
A first condition that must be verified for the optimization to stop is a minimum number of
completed generations, which is set to 150. After this point, the optimization stops if the
57
increase in the fitness score is less than 5% in the last 150 generations. Both parameters were
obtained empirically based on repeated testing and are not critical to the results obtained.
These penalizations are not essential to any of the results obtained and are used mainly to
decrease the complexity of the solutions and decrease the computational effort.
The main parameters used by the simulation function are always the same (unless stated
otherwise) and will be discussed next.
To annualize some of the performance metrics, like the percentage return, it is considered
that one year has 252 trading days on average.
The start and end dates of the simulations differ depending on the window of data.
58
One counter-intuitive consequence of using a fixed capital is that the maximum drawdown,
when measured as a percentage of the invested capital, can be greater than 100%.
The fixed capital used in the simulations of individual assets is 1000000 monetary units.
Where:
𝑴𝒂𝒓𝒌𝒆𝒕 𝒑𝒐𝒔𝒊𝒕𝒊𝒐𝒏 Is the percentage of the capital to be invested, according to the trading
rules and the information is contained in the trading signal vector. Its values are in the range
[-1.0, 1.0].
𝑪𝒂𝒑𝒊𝒕𝒂𝒍 Is the fixed capital used in the simulation.
𝑨𝒔𝒔𝒆𝒕 𝑷𝒓𝒊𝒄𝒆 Is the price of the asset.
59
4.4.5 Position size recalculation
Since some trades can last for a long time (months or years) it is possible that the asset prices
change considerably in that period meaning the trade size no longer respects the desired
capital (since simulations use fixed capital). To minimize this effect, when prices increase or
decrease more than a threshold, the ideal size of the trade is recalculated using equation ( 15
) and the adjustment is made. The threshold used for this recalculation is 10%. This value
doesn’t have a critical impact in results.
60
Still, graphically visualizing a portfolio with equal weights (assets always bought) can help
understand what happened with the assets in the test period (from 2006 to 2014) and serve
as a reference to the tested portfolios. This equal weight portfolio is not technically a buy-and-
hold portfolio since simulations are performed with fixed capital and periodic readjustments
are made in the individual asset positions as described in 4.4.5.
61
62
5 Results
This section presents the results of the individual optimizations using several setups. These
results are then analyzed and interpreted. The solutions obtained by the best setups are then
simulated in portfolios will all assets traded simultaneously. The setups use the parameters
specified earlier unless stated otherwise.
aRet % - Is the annualized return in percentage of the invested capital (which is fixed).
Avg Trade % - Is the average gain or loss per trade in percentage.
Avg Bars – Is the average duration of trades in bars (days).
Avg W/L – Is the ratio between the average winning trade and the average losing trade.
Win % - Is the percentage of winning trades.
Avg Loss % - Is the average daily drawdown in percentage.
Max Loss % or ML% - Is the maximum percentage Loss as described in 4.4.2.
Daily StdDev – Is the daily Standard Deviation of daily returns.
Lake Ratio 2 – Is the inverse of the Lake Ratio.
RRR – Is the Risk Return Ratio.
Fitness or JR4 – Is the fitness score.
aRet% [year] – Is the annualized percentage return in a particular year.
In the bottom of the table, there is a row which averages the results of all assets and works as
a summary of the setup being simulated. There is also a bottom row which shows the standard
deviation of the results from the different assets and is a measure of the dispersion of these
results.
63
5.2 Setup with a large search space
For the first test, a setup that allows high optimization freedom was chosen. The range of
genes are all those in Table 20 with the exception that only the “AND” and “OR” logic
operators are used and genes specific to one time frame are prohibited in other time frames.
The genomes have two subsystems, one for the long-term time frame and another for the
medium-term. Each sub-systems contributes with a weight of 50% to the final output trading
signals. Each of these subsystems uses two chromosomes, one to generate short signals and
the other to generate long signals (buy). Each chromosome can have a maximum of 3 levels
and increasing penalizations are applied to genomes with chromosomes with depths greater
than 1 (according to the rules previously mentioned). The Results are presented in Table 8.
The first conclusion that can be drawn is that the overall results are very poor, with an average
annualized return of only 1.39% which means the trading signals are not very far from random.
There are several possible explanations for these results, one of them is that the big search
space allows for curve fitting, resulting in non-sense solutions that work well in the
optimization window (in-sample) but poorly in the test window (out-of-sample). This can be
exemplified with the following observed short chromosome:
IFB (Boolean IF operator)
C<EMA (143) (IF condition)
C<SMA (184) (the output if condition true)
>= (the output if condition false)
RVI (214)
CLOSE
The main problem with this chromosome is the comparison between the RVI and the close
price. Since the RVI oscillates between 0 and 100, it should never be compared with the close.
It’s therefore an undesirable non-sense chromosome.
Another issue with this setup is that it allows for solutions that are not trend-following, in
particular, mean-reversing solutions appear in some periods, meaning that the genome will
short when the price rises and buy when the price falls. This may produce good results in a
particular optimizing window but will probably fail in the corresponding test window.
64
5.3 Setup using only Boolean genes (3 levels)
Using genes with inputs/outputs of type Double allow for many undesirable rules like in the
example shown in 5.2. In this new setup, the allowed genes are restricted to the ones with
Boolean inputs and outputs. In particular the following genes were used, “AND”, “OR”,
“MVOTE3”, “IFB” and all the “Technical Primitives”. This setup should not only reduce the
search space, but also concentrate on trend-following solutions since the Technical Primitive
genes are already mini-systems with rules that favor that trading philosophy.
All the remaining parameters and formats are identical to the previous setup presented in 5.2.
Looking at the results presented in Table 9, we can observe that the average of the annualized
returns, 3.91%, is considerably higher than the value observed in the setup with a larger search
space (5.2). Still, the Boolean If gene (“IFB”) and the 3 levels permitted for the chromosome
tree, allow for complex solutions that may represent over fitting and degrade the out-of-
sample results. The path of further reducing the search space should therefore be explored.
Average of
3.91 0.59 25.29 1.97 44.9 -16.3 -44.9 0.00915 2.06 0.118 0.102
all assets results
Starting with a maximum tree depth of 2, the following setups were tested (only genes used
in long chromosomes are shown):
Setup 3 with genes: AND, OR, MVOTE3, C>EMA, C>SMA, C>WMA, C>SMAB, RVI>X,
MACD>0, SAR_L, CH_BRK_L, TLP_100223_L, TMP_110112_L, TMP_090212_L,
MT_OSC_L, RSI>X, STOCH_D_s>X.
65
Setup 4 with genes: AND, OR, MVOTE3, C>SMAB, RVI>X, MACD>0, SAR_L, CH_BRK_L,
TLP_100223_L, TMP_110112_L, TMP_090212_L, MT_OSC_L.
Setup 5 with genes: AND, OR, MVOTE3, RVI>X, MACD>0, SAR_L, TLP_100223_L,
TMP_110112_L, TMP_090212_L.
Table 10 presents the average results from all assets and as can be seen, the average returns
increase as the pool of genes is reduced to those which presented best results individually.
The increase in returns may be explained by the reduced search space and complexity of the
solutions, producing less curve-fitting. Another possible explanation is related with the
characteristics of the removed genes: some of them generate trades when there is a cross
between the price and a moving average (ex: C>SMA) or between an indicator and a threshold
(RSI>X). And since the close prices and indicators have noise, this generates successive trades
alternating long and short positions which generates losses.
In practice the structure of the chromosomes will either have a single technical primitive gene
or will have “AND”, “OR” or “MVOTE3” in the root level and technical primitives as terminals
in the second tree level. See example of a long chromosome:
The next setups tested use a maximum tree depth of only 1 level meaning that only technical
primitives exist in the gene pool. They approximately replicate the previous setups (3 to 6) in
terms of the technical primitive genes used. The following setups were tested (only genes used
in long chromosomes are shown):
Setup 7 with genes: C>EMA, C>SMA, C>WMA, C>SMAB, RVI>X, MACD>0, SAR_L,
CH_BRK_L, TLP_100223_L, TMP_110112_L, TMP_090212_L, RSI>X, STOCH_D_s>X.
66
Setup 8 with genes: C>SMAB, RVI>X, MACD>0, SAR_L, CH_BRK_L, TLP_100223_L,
TMP_110112_L, TMP_090212_L.
Table 11 present the average results from all assets and, as was observed with the setups of
Table 10, the average returns tend to increase as the worst performing technical primitive
genes are removed from the pool. There is an increase in the average number of bars per
trade corresponding to less trades and less reversals. This means the trading rules are more
stable and more immune to noise, changing position only when there is a stronger indication
of a trend change. The same progressive increase can be seen in the average trade return.
Given this results, a more detailed analysis will be made of setup 10.
Figure 26 shows the results of one optimization with setup 10 in the S&P500 index from 2006
to the end of 2014. We can observe the returns follow the price variation closely until mid-
2008, when the price starts to fall violently and the trading system profits by being short. This
results in a very positive performance in the 2008 bear market. Between 2010 and 2012
S&P500 had an upward bias but with some corrections which translated into the trading
system having difficulties, alternating periods of profits and losses. After 2012, the index
entered a period of steady gains and the trading system is able to consistently profit again. In
summary, the trading system was able to profit when the market had a clear direction and
was able to, not only prevent big losses in the 2008 bear market, but strongly profit in that
period. The average annual return of the trading system in the 9 year period was around 9%.
68
Avg Avg Avg Avg Max Daily Lake Fitness aRet% aRet% aRet% aRet% aRet% aRet% aRet% aRet% aRet%
Asset aRet % Trade% Bars W/L Win % Loss % Loss % StdDev Ratio2 RRR JR4 2006 2007 2008 2009 2010 2011 2012 2013 2014
CAC -0.82 -0.20 47.07 2.02 31.64 -38.55 -87.76 0.01307 0.608 -0.009 -0.009 15.61 -3.94 47.00 0.07 -31.86 -21.30 2.28 -1.39 -16.43
AEX 2.21 0.52 43.91 2.07 37.40 -38.23 -77.11 0.01253 0.733 0.029 0.027 11.10 -3.57 62.95 -6.83 -29.09 -17.44 5.94 5.77 -5.92
BEL 6.18 2.11 64.48 2.60 42.82 -23.66 -54.94 0.01160 2.150 0.117 0.107 21.16 -17.83 64.99 2.97 -15.71 -8.87 -7.02 4.91 1.05
PSI 7.96 1.85 40.15 2.27 44.12 -20.63 -55.23 0.01112 2.971 0.152 0.139 28.05 15.83 42.62 1.41 -19.57 -3.46 -7.16 0.87 7.32
DAX 4.70 1.29 51.07 2.18 41.58 -25.49 -61.38 0.01303 1.603 0.085 0.078 16.92 16.47 22.60 -10.57 1.24 -23.72 12.95 19.18 -15.83
FTSE -1.23 -0.33 49.97 1.22 40.63 -19.78 -41.49 0.01082 0.169 -0.027 -0.024 4.62 -4.14 3.13 9.31 -11.62 -13.65 0.60 5.34 -11.96
SPX 9.26 3.06 66.41 2.35 54.63 -7.43 -27.24 0.01210 6.067 0.342 0.289 11.13 -1.58 25.71 23.59 -0.78 -12.29 7.85 27.23 1.76
NDX 6.64 1.77 47.98 2.26 44.52 -11.31 -35.68 0.01204 1.686 0.187 0.164 -13.17 1.93 14.98 10.42 16.66 -18.30 13.20 28.39 0.71
TSX -0.42 -0.10 43.96 1.84 34.33 -19.30 -38.34 0.01023 -0.180 -0.009 -0.008 8.40 -28.74 14.78 -12.64 3.49 1.60 -5.45 4.92 2.52
IPC 7.03 1.86 44.17 3.21 36.17 -12.38 -34.98 0.01127 4.047 0.202 0.177 30.49 0.86 6.66 24.60 3.60 -15.03 6.64 -2.91 4.21
AORD 2.34 0.63 47.15 2.17 36.69 -24.54 -63.61 0.00992 1.318 0.041 0.038 10.87 2.05 40.28 16.83 -12.63 -31.17 -8.79 6.17 -2.36
NIKKEY -2.30 -0.52 38.78 2.07 28.74 -51.58 -105.29 0.01393 -0.271 -0.021 -0.020 2.28 0.42 25.34 -40.84 -9.89 -19.11 2.42 27.67 -2.36
HSI 4.62 1.21 41.67 2.70 35.77 -20.67 -49.24 0.01296 2.645 0.102 0.092 25.96 26.01 -2.89 18.71 0.54 -10.54 0.33 -8.23 -8.29
STI 8.45 1.75 37.36 3.53 37.27 -12.73 -30.17 0.01026 4.981 0.287 0.245 22.03 9.72 42.61 20.63 -6.20 -16.64 7.82 1.53 0.09
SENSEX 9.91 2.52 45.82 2.86 43.70 -20.91 -49.95 0.01338 3.216 0.203 0.184 38.75 41.23 1.88 16.50 4.02 -32.27 -5.91 2.42 24.25
JKSE 15.04 3.87 45.00 3.00 48.76 -12.14 -32.60 0.01235 7.975 0.463 0.401 29.61 18.00 19.73 42.75 29.35 -6.32 3.57 0.61 -4.73
TWII 11.15 2.70 42.87 4.85 35.30 -11.39 -34.01 0.01046 7.363 0.330 0.287 13.49 15.45 51.94 35.37 5.88 -11.83 -0.82 -11.82 -2.85
MERV 20.79 6.51 58.93 3.64 43.51 -25.10 -84.35 0.01768 3.098 0.265 0.249 23.54 0.51 36.88 42.19 20.38 -27.13 -13.87 51.00 68.21
BVSP 3.70 0.94 42.37 1.73 43.94 -31.00 -62.83 0.01487 1.243 0.061 0.057 16.62 33.41 3.97 7.63 -3.70 -7.88 -11.63 0.54 3.83
B1_FR 25.82 6.29 47.22 4.42 46.87 -12.96 -56.66 0.01809 9.299 0.456 0.419 4.34 42.60 102.95 18.07 -25.24 11.95 21.27 5.70 61.14
HO1_FR 16.15 5.07 60.85 3.43 41.53 -37.29 -79.26 0.01690 2.297 0.206 0.194 4.77 15.59 126.22 -38.04 15.34 -3.71 12.62 -19.23 41.60
NG1_FR 25.52 7.71 55.58 3.37 46.68 -30.91 -89.68 0.02828 5.536 0.298 0.281 43.54 23.22 93.81 57.83 4.05 15.61 6.10 -28.45 -8.16
GC1_FR 1.15 0.32 50.51 1.45 43.71 -33.89 -61.23 0.01166 0.301 0.022 0.020 9.68 19.10 -25.40 -12.76 18.06 2.74 -12.85 16.11 -3.38
XAGUSD 11.27 3.02 47.31 3.16 38.97 -35.41 -66.67 0.01992 1.554 0.170 0.158 16.59 -0.35 -6.77 17.69 55.38 -11.03 -3.69 15.84 11.25
PL1_FR 4.50 1.17 51.54 2.91 33.26 -25.70 -70.87 0.01330 1.915 0.064 0.059 2.94 23.14 40.33 11.26 12.21 -38.73 -6.27 -1.87 -3.29
PA1_FR -2.76 -0.59 36.36 2.90 23.57 -54.83 -114.19 0.01747 0.073 -0.022 -0.022 -12.95 -2.76 6.73 40.12 57.05 -46.77 -33.19 -11.61 -5.00
S1_FR 3.18 0.70 37.36 2.67 31.30 -17.85 -44.18 0.01403 1.689 0.073 0.066 10.78 13.38 14.50 10.90 20.77 -17.48 21.66 -5.98 -19.78
SM1_FR 15.27 4.03 52.21 2.92 46.35 -9.91 -43.49 0.01618 7.494 0.355 0.318 -7.82 29.87 23.16 38.50 3.54 3.84 43.79 17.10 -6.07
W1_FR 3.14 0.78 40.80 2.15 35.58 -29.42 -72.23 0.01956 0.785 0.046 0.042 -16.34 10.56 43.44 -14.97 -7.95 9.93 0.79 13.12 -12.10
SB1_FR 23.99 8.44 71.06 3.29 56.88 -13.39 -37.60 0.01968 8.707 0.650 0.571 31.00 3.39 24.85 22.00 99.08 -6.62 -6.22 21.63 29.48
Average 8.08 2.28 48.33 2.71 40.21 -24.28 -58.74 0.01429 3.036 0.171 0.153 13.47 9.99 32.30 11.76 6.55 -12.52 1.57 6.15 4.30
Std Dev 8.12 2.43 8.82 0.81 7.27 12.04 22.67 0.00401 2.837 0.169 0.150 14.77 16.16 33.26 22.81 27.10 14.42 13.72 15.92 20.96
Table 12 – Results from optimization with setup 10 – best genes with tree depth 1
69
Figure 27 shows the graphical results of one optimization in Brent, which is as asset that
produces very good results with the trading rules generated with this method and setup. We
can observe the returns follow the price change until mid-2008 where Brent initiates a very
sharp fall. The system quickly reverses to a short position and strongly profits from the fall
that ends in the beginning of 2009. After that, Brent has a more erratic behavior despite having
an upward bias, and the trading system only manages a small gain in that period, until mid-
2014. After mid-2014 Brent starts a very strong fall and the system is again able to strongly
profit from the movement. In average, the trading system was able to gain 27% per year. In
this case it becomes very clear that the system specializes in following strong trends and earns
most of its profits from those movements, resulting in a return curve that can be almost flat
for several years and then have sudden jumps. This means it’s wise to use these type of
systems in a portfolio in order to have a smoother return curve and less risk.
Figure 28 shows the example of Palladium, which has a chart that is harder to be traded by
the trading systems generated by this method/setup. The system is able to profit from the
strong trend that starts in 2009 and ends in the beginning of 2011 but, in the remaining years
Palladium trades in a price range with no defined trend and the returns fall consistently. In the
entire period, the return is close to 0 and palladium is a case of an asset which was not
successfully traded by a system generated by this method/setup.
70
Figure 28 – Results for Palladium
As was previously mentioned, setup 10 has two subsystems, one focusing on the long term
time frame (0) and the other on the medium term time frame (1). Each subsystem has a single
chromosome which generates both long and short signals. Chromosome indices are unique
and can have any order depending on what is defined in the configuration files (in this case
the chromosomes 1 and 3 are used). Each chromosome has a tree with a single technical
primitive gene in the root node. In the long term chromosome, signals are generated by a
MACD Line when it crosses the 0 line. The MACD line is calculated from Exponential Moving
Averages (EMA) with periods 186 and 400. In the medium term chromosome, signals are
71
generated by a Relative Volatility Index (RVI) with period 37 when its value crosses the upper
threshold line of 53.7321 or the lower threshold line of 48.7321.
Figure 29 shows an equal weight portfolio simulated from the beginning of 2006 to the end of
2014. As can be seen, there is an upward bias but in 2008 there is a sharp fall followed by a
sharp rise. If an investor had used this strategy in this period, he would have had a reasonable
final return but would have sustained a very high decrease in his capital in 2008 (around 60%).
Table 13 shows a summary of the results obtained by this portfolio.
aRet% Win % Avg Loss % ML% ML Date Std Dev LR2 RRR JR4
5.21 76.67 -9.27 -59.94 21/11/2008 0.00936 0.035 0.087 5.614
Table 13 – Results summary of an equal weight portfolio
72
5.7 Setup 10 portfolio
This portfolio uses the optimizations obtained with setup 10 whose individual results were
presented in 5.5. Each asset receives an equal share of the portfolio capital and is traded
according to the trading signals generated by the best genomes optimized for that asset.
Figure 30 shows the returns obtained with this portfolio (red line) and the reference line in
black is the all-bought portfolio discussed in 5.6.
Table 14 shows the summary of results and Table 15 presents the annualized percentage
returns by year. As can be observed, this portfolio has a much better performance in the 2008
bear market than the reference. Not only it didn’t lose money but was able to strongly profit
from the fall observed in the majority of assets. The Maximum loss was 16.69% of the invested
capital and happened in the 4th of November, 2008. Analyzing the results by year, we can see
that the results were very good until the end of 2011. The next year, 2012 was negative and
the following years were positive but with lower returns. In fact, the last 3 years were not able
to recover from the 2011 loss, which may be explained by a different phase of the market with
more erratic movements from the majority of the assets.
We can conclude that this portfolio had an interesting annualized average return in the 9 years
of the simulation with a relatively low risk, but the returns were heterogeneously distributed
by the 9 years. There is no disadvantage in having very high performing years but there is a
disadvantage in having negative or poorly performing years.
73
aRet % Win % Avg Loss% ML% ML Date Std. Dev. LR2 RRR JR4
7.81 40.04 -5.96 -16.69 04/11/2008 0.00631 0.088 0.468 8.705
Table 14 – Results summary of portfolio with setup 10 optimizations
Table 16 compares the results from three portfolios: the one presented in 5.7 (with medium
and long term components) and the equivalent ones with only medium term (MT) and only
long term (LT) components. As can be seen, combining both time frames increases the winning
74
percentage and decreases the Maximum loss. Each time frame produces a different trade
profile, a different drawdown profile (maximum losses occurred in different dates) and
combining these two subsystems tend to balance each other and reduce risk. In practice, when
an asset has contradictory trends in the two time frames, the corresponding trading signals
are contradictory and the global system remains neutral. Only when both time frames agree
will the global system assume a position.
Table 17 shows the annualized percentage returns of the same three portfolio simulations
detailed by year.
Portfolio aRet % Win % Avg Loss% ML% ML Date Std Dev LR2 RRR JR4
MT 6.28 33.68 -8.44 -19.49 31/12/2014 0.00678 0.058 0.322 6.816
LT 7.85 36.33 -7.41 -21.21 18/05/2009 0.00689 0.070 0.370 8.592
MT + LT 7.81 40.04 -5.96 -16.69 04/11/2008 0.00631 0.088 0.468 8.705
Table 16 – Results summary comparing portfolios with different time frames
Portfolio 2006 2007 2008 2009 2010 2011 2012 2013 2014
MT 13.16 11.22 34.74 11.67 2.15 -15.53 -0.15 3.52 -2.24
LT 19.35 16.04 21.23 9.28 8.60 -12.86 -4.41 12.31 3.57
MT + LT 13.10 9.66 33.89 12.26 7.05 -11.91 1.64 6.25 0.81
Table 17 – Annualized percentage returns from portfolios with different time frames
75
5.9 Chapter Conclusions
Preliminary simulations showed that having a large search space is counter-productive since
it generates solutions with curve-fitting that performs poorly in out-of-sample data. As the
search space was restricted by decreasing the maximum depth of the chromosome trees and
by limiting the allowed genes (and consequently the complexity of rules), the out-of-sample
results gradually improved. The best setup used a single level in the chromosome trees and
only technical primitive genes were employed (setup 10). This appears to indicate that since
the data available for the optimization (in-sample) is limited, the search space and complexity
of the solutions must also be small in order to produce robust solutions.
A possible method to increase the complexity of the solutions without producing curve fitting,
is to use more complex rules encapsulated in genes that perform a simple task. In this work
this is done with Technical Primitive genes using majority vote internally to manage several
sub-rules.
The trading systems obtained with setup 10 produces very good results in some assets and
the portfolio constructed with the individual optimized trading signals produce interesting
results, with a good average annualized return and a relatively low risk. The trading systems
perform well in trending markets but poorly in lateral markets, which maybe can be improved
in subsequent works. Despite the portfolio overall positive results, it shows an irregular
distribution of returns, which means that the portfolio may produce high returns in some
phases and struggle in other phases. This may possibly be mitigated combining more
subsystems with different profiles, making more probable that the several different
subsystems balance each other.
76
6 Conclusions and Future Work
This work presents a methodology to develop trading rules for trading individual assets. The
resulting rules use a trend-following approach and combined medium-term and long-term
components in an attempt to increase the robustness of the solution. Genetic programming
was used as the optimization algorithm and the solutions are represented with chromosomes
organized in a tree structure. Several setups were tested in order to find the best configuration
and the portfolio simulations were performed to observe the combined result of trading
several assets with optimized trading rules. Several measures were employed in order to
reduce curve fitting and obtain robust solutions.
Several positive results were obtained, some limitations were identified and many research
paths remain open for further work.
6.1 Conclusions
The results obtained confirm that trend patterns in markets can be successfully identified and
explored in order to obtain profits. The portfolio simulation using the best setup produced an
annualized percentage return of around 8% with a maximum loss percentage of around 16%
in the period 2006 – 2014 which must be considered as positive. This results also confirm that
technical analysis can be used to predict the future behavior of the markets to some extent.
It also confirms that technical indicators can be used to extract useful information from asset
historical prices in order to generate successful trading signals.
This work combined rules with two time frames, medium-term and long-term and
demonstrated the improvement that can be obtained by such approach. The returns of the
combined rules are similar to the returns obtained by the single time frame components but,
the maximum loss is significantly lower. This is the result of having two sets of rules with
different trade profiles and different loss profiles, which tend to balance each other and
reduce risk. It’s a form of diversification: trading strategy diversification.
It was possible to use two time frames but without using data of different scales (i.e. daily,
weekly or monthly). Only daily data was used to produce rules with different time frames by
defining allowed parameter ranges for the technical indicators. It’s a method that can have
benefits over using several sets of data. To further differentiate the rules of each time frame
the following method was successfully employed: The rules of each time frame must generate
an annualized number of trades inside a certain allowed range. Solutions not respecting this
are penalized in the fitness function.
Several measures were successfully employed to reduce curve-fitting of the solutions and
improve robustness. This was done by reducing the search space, eliminating solutions which
didn’t use a trend following philosophy. In practice this was achieved by only using genes that
77
produced trend-following rules. The complexity of the solutions was also reduced by limiting
the chromosome trees maximum depth.
To prevent successful solutions to dominate the population with many similar copies, similar
solutions should be deleted in order to prevent a drastic decrease in the diversity of solutions
and an increase in the likelihood of the optimization getting stuck in local maxima. To
accomplish this, a novel algorithm was used which doesn’t look at the chromosome tree
similarities. Instead, it compares the Mean Squared Error (MSE) of the vectors with trading
signals generated by the solutions. If two solutions have a MSE lower than an empirical
threshold, the one with less fitness score is eliminated.
A special type of complex gene was proposed (technical primitive) which combines output
simplicity with internal complexity. The signals generated by these genes are simple in the
sense that they are binary and follow a very simple logic of detecting and following a trend.
But the way to achieve that end is more complex than common trading rules since they include
several sub-rules which are combined by majority vote. This approach has the advantage of
not increasing the search space since the gene doesn’t necessarily have many optimizable
parameters and, allows for more sophisticated rules. Majority vote also tends to suppress
false signals generated by the sub-rules and the output signals tend to be more stable, with
some immunity to noisy price action.
Alongside the several mentioned successful points, some limitations were observed and are
enumerated next.
78
Chande, the Dynamic Momentum Index [6], The Directional Movement [6], Moving
averages which adapt to market volatility like the VIDYA [6].
Test complex trading rules encapsulated in technical primitive genes. Use several sub-
rules and apply techniques like majority vote to filter false signals and increase the
resistance to noise.
Experiment with genes that generate signals in the direction of the main trend but also
wait for a correction in that trend. That approach would probably give better results in
periods of less clear trends.
Test other ways to articulate the individual optimization phase with the portfolio
phase. Some assets could be excluded from the portfolio using a criteria that checks if
their contribution would likely be negative. Alternatively an algorithm could be used
to give different weights to different assets.
Try using different setups for different asset types (i.e. stock indices, commodities,
etc.). This was technically possible in this work but wasn’t tested because of time
restrictions.
Experiment with setups that use genomes with more subsystems, with different
profiles, in order to have greater system diversification. More subsystems with
different profiles tend to balance each other and reduce risk.
Experiment the optimization of trading rules with other types of assets, like stocks,
currencies (FOREX) and financial assets.
79
80
References
[1] E. Fama, "Random Walks in Stock Market Prices," Financial Analysts Journal, vol. 21,
no. 5, pp. 55-59, 1965.
[2] B. Malkiel, A Random Walk Down Wall Street : The Time Tested Strategy for Successful
Investing, New York: WW Norton & Company, 2011.
[3] C. M. Andrew Lo, A Non-Random Walk Down Wall Street, Princeton University Press,
1999.
[4] C.-H. Park and S. Irwin, "The Profitability of Technical Analysis: A Review, AgMAS
Project Research Report 2004-04," University of Illinois at Urbana-Champaign, 2004.
[5] J. Murphy, Technical analysis of the financial markets, New York Institute of Finance,
1999.
[6] P. Kaufman, Trading systems and methods, John Wiley & Sons, 2013.
[7] J. Machado, R. Neves and N. Horta, "Developing Multi-Time Frame Trading Rules with a
Trend Following Strategy, using GA," in Genetic and Evolutionary Computation
Conference - GECCO, Madrid, 2015.
[10] B. Graham and J. Zweig, The Intelligent Investor: The Definitive Book on Value
Investing, HarperBusiness, 2006.
[11] B. Baumohl, The Secrets of Economic Indicators: Hidden Clues to Future Economic
Trends and Investment Opportunities (3rd Ed.), FT Press, 2012.
[13] J. Bettman, S. Sault and E. Schultz, "Fundamental and Technical Analysis: Substitutes or
Complements?," Accounting & Finance, vol. 49, no. 1, pp. 21-36, 2009.
81
[15] W. Brock, J. Lakonishok and B. LeBaron, "Simple Technical Trading Rules and the
Stochastic Properties of Stock Returns," The Journal of Finance, vol. 47, no. 5, p. 1731–
1764, 1992.
[19] H. Markowitz, "Portfolio Selection," The Journal of Finance, vol. 7, no. 1, pp. 77-91,
1952.
[21] S.-H. Chen, Genetic Algorithms and Genetic Programming in Computational Finance,
Springer, 2002.
[22] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs (3rd Ed.),
Springer, 1996.
[24] R. Poli, W. Langdon and N. McPhee, A Field Guide to Genetic Programming, Lulu
Enterprises, 2008.
[25] J. Korczak and P. Roger, "Stock timing using genetic algorithms," Applied Stochastic
Models in Business and Industry, vol. 18, no. 2, p. 121–134, 2002.
[26] L. Mendes, P. Godinho and J. Dias, "A Forex trading system based on a genetic
algorithm," Journal of Heuristics, vol. 18, no. 4, pp. 627-656, 2012.
82
[29] F. Allen and R. Karjalainen, "Using genetic algorithms to find technical trading rules,"
Journal of Financial Economics, vol. 51, no. 2, pp. 245-271, 1999.
[32] C.-M. Lin and M. Gen, "An effective decision-based genetic algorithm approach to
multiobjective portfolio optimization problem," Applied Mathematical Sciences, vol. 1,
no. 5, pp. 201 - 210, 2007.
[33] C. Aranha and I. Hitoshi, "A tree-based GA representation for the portfolio optimization
problem," in Genetic and Evolutionary Computation Conference (GECCO), Atlanta,
2008.
[35] A. Silva, R. Neves and N. Horta, "A hybrid approach to portfolio composition based on
fundamental and technical indicators," Expert Systems with Applications, vol. 42, no. 4,
p. 2036–2048, 2015.
[36] M. C. Roberts, "Technical analysis and genetic programming: Constructing and testing a
commodity portfolio," Journal of Futures Markets, vol. 25, no. 7, p. 643–660, 2005.
[37] A. Gorgulho, R. Neves and N. Horta, "Using GAs to balance technical indicators on stock
picking for financial portfolio composition," in Proceedings of the GECCO'09, 2041-
2046, Montreal, Canada, 2009.
[40] D. Dorsey, "Refining The Relative Volatility Index," Technical Analisys of Stocks &
Commodities, pp. 388-391, September 1995.
83
[42] "The Qt Company," [Online]. Available: http://www.qt.io/. [Accessed December 2014].
[47] "2014 FIA Annual Volume Survey – Charts and Tables," Futures Industry Association,
2014.
84
Appendix A- Used Assets
Table 18 lists all the stock indices used in this work and Table 19 lists all the commodities used.
85
86
Appendix B – Groups of genes
Operator /
Index ID String Input Output Description Genes
Terminal
0 O_ARTM Operator double double Arithmetic operators +, -, /, *, Max, Min
1 O_MATH1 Operator double double Math operators Pow, SQRT, LN, LOG10
2 O_MATH2 Operator double double Math operators 2 Round, Floor, Ceil, Abs
3 O_COMP Operator double Boolean Comparison operators >, <, >=, <=, ==, !=
4 O_BOOL Operator Boolean Boolean Boolean operators AND, OR, NAND, NOR, XOR, XNOR, NOT
EMA (Exponential Moving Average),
5 O_TECH Operator double double Technical operators SMA (Simple Moving Average),
WMA (Weighted Moving Average)
6 O_MVOTE Operator Boolean Boolean Majority vote Majority Vote 3 inputs, Majority Vote 5 inputs
IF with inputs of type double (* condition is
7 O_IF Operator double* double IF with double inputs
Boolean)
8 O_IFB Operator Boolean Boolean IF with Boolean inputs IF with inputs of type Boolean
9 T_CONST Terminal - double Constant series Constant double value (ex: 80.5)
The vectors Open, High, Low, Close and Volume
10 T_FIXED Terminal - double Fixed series
from the asset data
Technical primitives (auto- See
11 T_PRIM Terminal - Boolean
sufficient mini subsystems) Appendix C – Technical Primitive Genes
Stochastic Indicator %K, Stochastic Indicator %D,
Technical indicators with fixed
12 T_TECH Terminal - double Relative Volatility Index (RVI), Relative Strength
parameters (no child nodes).
Index (RSI)
Table 20 – Gene groups and characteristics
87
88
Appendix C – Technical Primitive Genes
Technical Primitive genes are terminal genes with Boolean output that encapsulate more
complex rules which are auto-sufficient in generating buy and/or short signals. The internal
algorithm may be relative simple or more complex. In some cases the output is the result of a
majority vote of several sub-rules. They may or may not receive parameters. Its use in this
work is fundamental since tests showed the best results are obtained when the chromosome
trees have a single technical primitive gene.
Table 21 shows the list of all Long Technical Primitive genes, with its Identification string and
a description. Equivalent short genes exist but are not shown.
89
Medium-term composite indicator described in section 3.3.6. A buy signal
TMP_110112_L is generated when the majority vote of its 3 components give a buy signal
and the equivalent for the sell signal.
Medium-term gene which has some internal complexity and was
previously created by the author. It uses a volatility calculation based on
the difference between an upper and lower price bands. The upper band
TMP_090212_L
is calculated with the past high peak values and the lower band is
calculated with past low peak values. A buy/sell signal is generated when
the present volatility line crosses a past volatility reference.
Medium-term gene which generates a buy signal when 3 conditions are
verified. 1) An up-trend is detected with an RVI indicator; 2) A correction
MT_OSC_L on that up-trend exists; 3) Price gives an indication that the up-trend will
resume. This was an experimental algorithm with limited impact on the
results.
A buy signal is generated when the Relative Strength Index (RSI) [6]
RSI>X crosses a threshold level from below and a sell signal is generated when
the RSI crosses the threshold line from above.
A buy signal is generated when the Slow Stochastic D Index [6] crosses a
STOCH_D_s>X threshold level from below and a sell signal is generated when the
Stochastic index crosses the threshold line from above.
Table 21 – Description of LONG Technical Primitive genes
90
Appendix D – Cache of vectors
The optimization module works with vectors of data in which the vector elements correspond
to trading days. There are many types of data and all corresponding vectors must be
synchronized with each other. Data from those vectors is accessed using an integer index
(associated with a trading day) and a particular vector position must correspond to the same
trading day in all vectors.
When a chromosome is interpreted for a particular asset and converted into trading signals,
the process is completed for all trading days, resulting in a vector with the trading signals for
that chromosome for that particular asset. Intermediary steps in the interpretation of the
chromosome (gene interpretation) also produce vectors. The asset Open, High, Low and Close
data is also hold in vectors with the same characteristics.
When the optimization module starts, it creates the cache with a pre-defined number of
vectors, which are encapsulated in a C++ class. If later the program determines the need to
have more cache vectors, they are created. A vector from this cache are generic vectors than
can be used and reused for any task that requires a vector.
Each vector in the cache will be referred as a slot and each slot contains the following
information:
The cache is managed in a C++ class which contains two structures with the previous
enumerated information:
A C++ unordered map structure which associates the ID string with the pointer to the
slot object;
A bidirectional map defined in the C++ library Boost (the 2 fields can act as keys). This
structure relates the ID string with the age of the series in the cache.
91
Find a specific vector and return the pointer to the slot: Before any calculation, the
program searches the cache to see if the result is already there. For this is calculates
the ID string of the resulting vector. The cache either returns the pointer of the slot
object containing the vector or returns NULL indicating the vector is not in the cache;
Get a slot: If the program needs a vector to hold some calculation, it requests a free
slot. The cache manager will locate an unused slot or will free the oldest used slot and
return the pointer to it. The cache function receives the new ID string as parameter
and associates it with the slot.
When the information on a particular vector is no longer needed, the pointer to it, received
from the cache, can be deleted and that slot/vector will later be reused for another purpose
in another part of the program. The use of this cache system has a high hit rate (requests for
vectors that are in the cache) that depends of the types of genomes, genes,
crossover/mutation rates and other parameters. In some cases the hit rate is higher than 80%
which represents a large saving in terms of the time required to run the
optimizations/simulations. The structures and organization of the cache can be seen in Figure
33.
92
Appendix E – Performance metrics
Each simulation of a genome, sub-system or chromosome produces performance parameters
that can be used to plot results and analyze results.
Table 22 shows a list of performance metrics calculated 3 versions: for long trades, for short
trades and for all trades (there are 3 versions of each item).
93
max_dd_date maximum drawdown date (string)
return_p percentage return
annual_ret_weighted weighted annualized rate of return
annual_return annualized percentage return
equity vector with daily returns (equity)
ddown vector with daily drawdown
Table 22 – Performance metrics calculated for Long, Short, and Global trades
Table 23 shows performance metrics that are calculated only for the global trades and not
for the special cases of long and short trades.
94