We have talked about the dangers of optimizing trading systems (forcing trading systems to conform to historical data), but, one of the subtle forms of optimization happens with portfolio selection. Market selection curve fitting happens when trading systems only get tested across a handful of markets (or sometimes just one market).
The problem is that what usually gets done is that most all the available markets get tested, and then only those that performed the best get shown in the portfolio. Doing this is an enormous mistake because the markets that performed best historically are rarely the ones that continue to be the best. What traders end up with is something that only worked well historically.
Trading Systems Solution
To avoid this tendency, we believe that the most robust way to see a system test is across ALL the available markets. Some will argue that different markets should get traded different ways, and to that we say NONSENSE. Markets are always changing, and a market that traded like market XYZ today will trade like market ZYX tomorrow. Unless a system is robust enough to trade every market, it is likely a useless curve fit of the data.
For us in the commodities markets, we use test roughly 80 markets. There are over 100 commodity contracts that trade, but we do limit the selection to those that are liquid enough (have enough trading volume) to trade.
Testing this way does cause one problem. The problem is that traders can be trading many markets in the same sector at a given time. Investors will need to have some sector risk control mechanisms in place. We like to be certain that the risk in each sector does not exceed about 5% of the account equity.
If someone creates a system that can trade nearly EVERY commodity market and uses the same rules for each market and gets tested over a long period, he may be on to something. Just remember, the next time someone shows the results of backtesting remember to ask “How many markets does this test include?” If the answer (for commodities) is less than about 70 or 80, then be suspicious that this may be curve fit results. Once again, curve fitting tends to produce systems that ONLY perform well historically.
Our Approach to Trading Systems.
We test all the systems we release across nearly every tradable commodity market. We could easily improve performance (historically) by cherry picking only the best markets, but we know this is misleading data.
[/et_pb_text][/et_pb_column][/et_pb_row][/et_pb_section]
By: Dean Hoffman
FUTURES TRADING IS NOT SUITABLE FOR EVERYONE AND PAST PERFORMANCE IS NOT NECESSARILY INDICATIVE OF FUTURE RESULTS. THERE IS RISK OF SUBSTANTIAL LOSS IN FUTURES TRADING OR WITH ANY TRADING SYSTEM OR PROGRAM. CAREFUL EVALUATION OF YOUR PERSONAL FINANCIAL SITUATION MUST BE DONE PRIOR TO DECIDING TO TRADE IN THE FUTURES MARKETS OR ANY GIVEN TRADING SYSTEM OR METHODOLOGY.
