• Introduction: What are Backtesting and Optimization?
• Backtesting: Using the Backtest Wizard to set up a backtesting strategy
  • Determine your Strategy Entry and Exit goals
  • Create a new Backtest page
  • Changing your Entry and Exit Strategy settings
  • Selection of ticker(s) to backtest
  • Performing the Backtest
  • How to read the Backtest Report
• Optimization: Comparing Backtest Strategies using different parameter values
• Adding variables to your formula for optimization
• Starting an Optimization for a series of Backtest strategies
• Curve fitting
• How to read the Optimization Report

Backtesting provides a way for you to test your trade strategies on historical data. For example, if you have a strategy that enters a trade when the price crosses a 20-bar moving average, you can use Backtesting to test it. Backtesting will start as far back as you would like (10 years, for example) and test your trading strategy bar-by-bar. It will place trades when your strategy triggers an entry, and exit trades when your strategy triggers an exit. Backtesting does this on every bar of historical data, and then generates a report for you that tells you how profitable the strategy was, and also lists the exact trade entry and exit conditions (price and date). By reviewing the backtesting report, you can determine whether your strategy would have been profitable if you had been trading it for the last 10 years. This lets you adjust your strategies to find conditions that have worked well in the past. Of course, just because a strategy has worked in the past, does not mean that the strategy will continue to work.

Optimization takes backtesting further by performing a series of Backtests using different settings. After you have selected a strategy formula that you would like to use, you can use the Optimization feature to test the strategy with different parameters. For example, if your strategy is "enter a trade when the price crosses a Simple Moving Average", you might like to find out what SMA values work the best. Often, the 50-SMA daily is used as support or resistance for institutional investors. But you might want to determine whether the 20SMA or 30SMA or 40SMA or 50SMA gives you a better profit when backtesting. The Optimization will run a Backtest on each of these conditions, and then give you a summary report that you can use to evaluate which of them works best for your particular situation.

Backtesting

To set up a useful Backtest, you need to identify two things: Trade Entry conditions and Trade Exit conditions.

Trade Entry conditions include:

• Trade Direction (short or long)
• Trade setup conditions (what is it that tells you that the conditions are good for a trade?)
• Trade placement conditions (assuming setup is good, what stop or limit conditions are appropriate to take the trade, or do you just take the trade at market price?)

• Profit Exit - at what point has your profit goal been reached? This may be a percentage gain, or a dollar amount.
• StopLoss Exit - at what point do you decide that the trade is a losing trade, and exit? This may be a percentage loss, or a dollar amount.
• Breakeven Exit - how much profit do you need before you move the StopLoss to the trade entry price? This may be a percentage gain, or a dollar amount.
• Trailing Exit - what is the most amount of profit, in dollars, that you are willing to give back before exiting the trade?
• Time Exit - how many bars will you stay in the trade with no other Exit conditions triggering?
• Reversal Exit - Assuming you test with simultaneous Short and Long strategies, when you hold a position, and the Trade Entry conditions signal for you to take a trade in the opposite direction, HamScan does a reversal - that is, it closes the original trade and enters a new trade in the new direction.

• Take a Long trade when the Closing price crosses up over the 20 period Simple Moving Average.
• Take a Short trade when the Closing price crosses down over the 20 period Simple Moving Average.
• Enter the trade at the Opening market price on the next bar
• Exit the trade with either a 10% profit or 2% loss, or if the trade lasts for more than 10 bars

Using the above Example trade Entry and Exit conditions, here are the steps to have the Backtest Wizard in HamScan create the Backtest for you:

• Start HamScan
• From the HamScan menu select Backtest / New Backtest Page and name it "BackTest" (the default)
• HamScan pops up the window below, with the defaults already set for our Example trade conditions
• Verify that each of the conditions are correct (details on all of the fields, below)
• Click the Save and exit button
• HamScan creates a new Page for you with each of these conditions. HamScan allocates one column for Trade Setup (a true/false formula), one column for Trade Entry (a stop/limit price or "at market open" condition), one column for Trade Exit Above (which is the Profit exit for Long trades, or Stop exit for Short trades), one column for Trade Exit Below (which is the Profit exit for Short trades, or Stop exit for Long trades), and one column for Duration Stop (which is a number or calculation giving the number of bars). Long and Short each get these five columns. Advanced users may edit these columns directly.

• Take Long(Short) Trades - this is only used on the initial creation of a Backtest Page. Uncheck one of these if you ONLY want to test Short(Long) trades. There is no harm in creating both Long and Short conditions, because when you run the Backtest, you are again given the choice to test Long and/or Short trades.
• Setup Conditions - This is a standard HamScan Formula which returns a true/false value. The default condition (as in the Example) is "Take a Long trade when the Closing price crosses up over the 20 period Simple Moving Average." The formula to do this compares the current Close with the current SMA20 to make sure that the Close is higher and compares the previous Close with the previous SMA20 to make sure that the previous Close is lower. You may edit the formula directly in this window, or you can click the FormulaBuilder button to bring up the HamScan Formula Builder window which gives you access to all of the functions, variables, context-help, watchlists, and system criteria. NOTE - the formula should return a True/False value - this is a "yes or no" condition - if the condition is yes (true), then HamScan checks the Trade Entry to determine whether a trade can be put on; if the condition is no (false), then no trade is attempted.
• Trade Entry - Assuming that the Setup Conditions are "true", this field is used to try and enter the trade. There are three ways to enter a trade: At Market, a Limit trade, or a Stop trade.
  
  If you select At Market Open then the trade will be taken at the Open price (and the Stop/Limit Price is not used).
  
  If you select Limit (or Lower) for a Long trade, then the trade will be taken only if the price during the bar gets to the Stop/Limit Price value or lower. The Stop/Limit Price might be set to "C1-.1" for example - to cause the trade to be entered only if the price drops 10 cents below the previous Close price.
  
  If you select Stop (or Higher) for a Long trade, then the trade will be taken only if the price during the bar gets to the Stop/Limit Price value or higher. In this case, you might set the Stop/Limit Price to "C1+.1", for example. Trade Entry for Short trades is the same, except that a Stop trade entry is "or Lower" and a Limit trade entry is "or higher". The easy way to remember Stop and Limit order labelling for trade entry is: Limit means "at this price or better" - that is, for a Long trade, you would like to enter at the specified price or at a better price (lower). Stop means "at this price or worse" - that is, for a Long trade, you would like to enter at the specified price or at a worse price (where you pay more - higher). Stop entry prices are often used for breakouts. You can place a Stop Entry trade at a price that is above (for Long trades) a channel or band, for example, and then when the price breaks out of the channel, your order gets filled.
• Profit Exit - determines when the trade will be exited with a profit. If the Percent is checked, then this value is in percent profit based on the price the trade was entered. If Percent is unchecked, then this value is a dollar amount. For example, a 10% Profit Exit on a stock that was purchased (Long) at $40 would be $44. A dollar Profit Exit of 2 would exit with a profit at $42.
• Stop Loss Exit - determines when the trade will be exited with a loss. If the Percent is checked, then this value is in percent loss based on the price the trade was entered. If Percent is unchecked, then this value is a dollar amount. For example, a 2% Stop Loss Exit on a stock that was purchased (Long) at $40 would be $39.20 ($40 minus 2% or $0.80). A dollar Stop Loss Exit of 1 would exit with a loss at $39.
• Breakeven Stop - determines at what point the Stop Loss Exit will be moved to the entry price of the trade, that is, how much profit is required before a retrace to the entry price will cause an exit. If the Percent is checked, then this value is in percent profit based on the price the trade was entered. If Percent is unchecked, then this value is a dollar amount. For example, a 2% Breakeven Stop on a stock that was purchased (Long) at $40 would be $40.80 ($40 plus 2% or $0.80). If the prices reaches $40.80, and then retraces to $40, the trade would be exited. A dollar Breakeven Stop of 1 would exit if the price reached $41 and then retraced to $40.
• Trailing Stop - determines how much profit can be retraced before exiting the trade. This value is in dollars. For example, if the Trailing Stop is set to 1, and the (Long) trade was entered at $40, then if the price ever went down $1.00 from the highest high since the trade was entered, the Trailing Stop would cause the trade to be exited. However, if the price of the stock trends up (with pullbacks of 50 cents or 75 cents), the Trailing Stop would continue to be moved up, but the trade would not exit. Use of the Trailing Stop can allow a trade to go much further than a specified Profit Exit because it does not cause the trade to exit until a large enough pullback (or start of a reversal) has occurred.
• Duration Stop - determines how many full bars a trade can be active before deciding that the trade has lasted too long, and should be exited. This is often used to identify trades in a stock that is moving sideways - and to allow you to reallocate your money to another trade that is more active. For example, a value of 10 means that if the trade has not exited for another Stop or Profit reason by the time the trade has been active for 10 complete bars, then the trade will be terminated on the next bar.
• Save and exit - Causes all of the settings entered above to be saved into the HamScan Page in their respective columns.
• Cancel - Causes changes to be ignored. This button is normally used when editing an existing Backtest Page when you decide that you do not want to make any changes.

After creating a Backtest Page using the Backtest Wizard, you may want to make some changes to the settings. To do this, make sure that you have the BackTest Page selected in the Page drop-down, then select from the HamScan menu Backtest / Backtest Wizard. This will bring up the above Backtest Wizard window with your current settings. You can modify the settings, and click the Save and exit button to change them or click Cancel if you change your mind.

Depending on the speed of your computer, the amount of memory available, the number of Optimization combinations, and the number of bars of history you select to test, a Backtest may take anywhere from a few seconds to several hours. Generally, if you are Backtesting a particular formula, you would like to have it work across multiple stocks or markets. If you test or Optimize a particular formula for a particular stock, you should be careful to not be overly optimistic of the results due to possible curve fitting. In testing hundreds of TradeStation Strategies over the last two years, I have found that careful selection of tickers for backtesting is essential. If you trade one particular sector, you might choose two or three of the most liquid (highest trading volume) stocks in that sector. Another good approach is to test the strategy on the sector index (SOX, BTK, etc.), or even on a broader index like the Dow 30, Nasdaq 100, or S&P 500.

In general, you should select a very small number of representative tickers for Backtesting when you first start testing a formula. Then as you improve the formula, you may want to drill down to more specific sectors or stocks that interest you.

You may use any of your TCNet/TC2000 watchlists as a starting point. From there, right click on tickers to "hide" them (Backtesting and Optimization only calculate and report on visible tickers in the watchlist). Narrow your list of tickers to one or a very small number. Note that you can also save the visible tickers of a watchlist as a new watchlist (right click again). The easiest way that I have found is to create a new small watchlist specifically designed for Backtesting. Also note that HamScan version 03.00.01 has a faster way of selecting watchlists than from the drop-down list. You may now select them from the hierarchical list from the HamScan menu Data / Choose WatchList.

To start a Backtest after selecting a Backtesting Page and choosing a list of ticker symbols, select Backtest / Test Strategy from the HamScan menu. The Backtest Settings window will appear:

To do a Backtest, you may ignore the Optimization Variable Settings - they should all be left blank.

Determine how far back you would like to test. For example, if you would like to test the strategy over the past two years, set Number of bars back to test to 500 (at approximately 250 trading days per year). You may choose to use a shorter timeframe to test the strategy over a shorter term bear or bull market - or you may choose a much longer timeframe to test the strategy over a combination of markets.

Next, select whether you would like to test Long or Short trades - or both. You may select either Long or Short to test only trades in that direction. If you select both, then Long and Short trades are allowed to reverse each other; that is, a Short trade signal may cause an exit from a current Long trade (before any Stops or Profit exit is triggered).

Finally, you may change the name of the file to which the report will be written. If you are only viewing the report, you do not need to change this. If you are saving different reports for future examination, then change the Save results to file name (click Browse to change file names or folders). HamScan supports both HTML and Excel spreadsheet outputs. If you output to Excel, then each individual report is written to a separate WorkSheet, and each Optimization report also is placed on it's own WorkSheet.

Click the Start backtest button to have HamScan begin the Backtesting. The progress of the Backtest will be displayed in the Status Message box in the upper right corner of the main screen - and the Stop button will allow you to interrupt the Backtesting process in between individual tests (each ticker or optimization will complete, however, so the Stop signal may take several seconds before it halts the backtesting). When HamScan has finished the backtest, it will automatically display the web page containing the Backtest Report. This web page is saved in the file you selected with Save results to, so that you can go back later to examine it.

There are three main sections to the Backtest Report: Trade Summary, Backtest Settings, and Trade Statistics. Examples of each of the three sections are shown below, with a descriptions of the contents.

Optimization

The HamScan Optimization feature allows you to test the same formula multiple times, with different parameter values, and then compare the results. To do this, replace the number or calculation in your Formula with one of the Optimization Variables (X, Y, or Z). For example, if you want to test the formula "C>Avg(C,20) AND C1<Avg(C,20,1)" (which looks for the price to cross up over the 20-period Simple Moving Average), but you would like to determine whether 20 is the best period setting to use, replace the "20" with "X": "C>Avg(C,X) AND C1<Avg(C,X,1)". In the next step, when you start the Backtest / Optimization, you will be given the opportunity to select the value(s) that X can take on.

After altering your Trade Setup formulas by adding Optimization Variables, you are ready to run an Optimization Backtest. To start an Optimization Backtest, select Backtest / Test Strategy from the HamScan menu. The Backtest Settings window will appear:

With the above example formula where the value "20" was replaced with "X" to test for the optimal value for an X-period Simple Moving Average, enter values for each variable used in the formulas. In this example, we would like to test the Simple Moving Average for periods of 10, 20, 30, 40, and 50. To do this, we set the value of "X" with a Start Value of 10 (the first test we want to try). We also set the End Value to 50 (the final test we want to perform). Finally, we set the Increment value to 10, which means that each successive test will add 10 to the previous value of X. So, X will start at 10, then be incremented to 20, 30, 40, and 50 - and then stop after the Backtest with a value of 50 is completed.

You can set ranges for each of three variables (X, Y, and Z). However, each Backtest may take one second to several minutes, depending on your settings, and how much historical data you have chosen to test. Therefore, the most efficient technique is to use larger increment values to narrow your Optimization down, and then retest with a smaller range. For example, if the 10 to 50 test (10 at a time) indicates that 30 looks promising, you may then want to run another Optimization from 25 to 35 with an increment of 2 (25, 27, 29, 31, 33, 35). Also, if you use more than one variable (X, Y, Z), multiply the number of choices for each variable to determine how many total tests there will be. If you vary X from 10 to 50 by 10's, (5 values total) *and* you vary Y from 1 to 8 by 1's (8 values total), HamScan Optimization will backtest 5 x 8 = 40 different combinations. HamScan has no problem doing extensive Backtests, but you should be aware that the larger number of combinations you select, the more time will be required to finish the Optimization report. Note that you may click the Stop button at any time during an Optimization, but the current ticker and X,Y,Z setting will be completed before the Optimization is stopped.

The Optimization Report includes all of the Backtest reports for each of the X,Y,Z combinations. This allows examination of the detail of the conditions and trades for each of the Backtests. In addition, the Optimization Report includes a summary of all of the Backtest reports to help to quickly identify the most profitable variable combinations. If a particular combination works well, then you can narrow the search by selecting a smaller range (and smaller increment) for each variable. Finally, when the best combination has been identified, replace the X, Y, Z variables in the original formula with the values determined to give the best results. Below is an example Optimization Report which shows the settings and results for each Backtest, one per line:

• X - the value of the "X" variable, if used
• Y - the value of the "Y" variable, if used
• Z - the value of the "Z" variable, if used
• Number of Trades - the total number of (round-trip) trades taken
• Net Profit/Loss - the amount of money gained or lost after taking all of the trades, using a default trade size of 100 shares
• Percent Profitable - the percentage of winning trades divided by total number of trades

Strategy Backtesting and Optimization are very powerful features - allowing you to test multiple combinations of formulas and determine whether these formulas have performed well or poorly in the past (based on historical data). There are two major flaws in the whole idea of Backtesting and Optimization. They do not necessarily invalidate the results - but they must be taken into consideration when evaluating the results.

The first issue, as mentioned in the disclaimer of most trading strategies, is that "past results may not reflect future profitability" - that is, if a strategy has been successful in the past, there is no guarantee that it will continue to work. To help validate that a particular formula works (and to help strengthen the premise that it will continue to work), the best method generally is to broaden the scope of the test. If a particular strategy works on a single stock, there is not any reason to believe that it will work on another one. However, if a strategy works on a group of stocks, then you may feel more confidence that it will work for the stocks in that group. For example, if a strategy works on KLAC, it may or not work well on other Semiconductor stocks. However, if the strategy works for the SOX index, then you may be able to have more confidence using the strategy on individual stocks (particularly the larger component stocks of the index) in the Semiconductor Sector. If a strategy works on broader based indices (Dow Jones 30, Nasdaq 100, S&P 500, etc.), then you may have more confidence in using it on individual stocks. Of course, each stock should be backtested to verify that past historical data does not break your model. Again, Past performance does not indicate that the strategy will be guaranteed to work in the future - you're just looking for a good probability that the strategy generally reflects the market.

The second issue, which is often abused when doing Strategy Optimization, is called Curve Fitting. Curve fitting is a mathematical technique that allows you to take an arbitrary set of data and generate a formula that approximates that data. When doing strategy Optimization, it is important to *not* try to get too detailed in your formula description, because if you do, then you are probably identifying particular conditions in *past* historical data, and improving the formula for them - which does *not* help for any future usage of the calculation. Again, if you are able to define a more generic formula (one that works on multiple stocks, or broader markets or indices), then you are more likely to have a valid test. However, it you choose a Simple Moving Average period of 43, but testing of values 41, 42, 44, and 45 produce inferior results, then you should be very cautious about believing the results. In general, if a particular value (like SMA period) in your strategy seems to work well, then it's also likely that values near that will work reasonably - be VERY suspicious if a single value works but others near it do not.