Backtester Logic (Part 3)

I included a code snippet in Part 2 to explain how the wait_until_next_day flag is used in the early stages of the backtesting program. I mentioned that once the short option is addressed (either via finding long/short branches or updating long/short branches), the program should skip ahead in the data file to the next historical date.* This begs the question as to where in the program current_date gets assigned. Today I will take a closer look to better understand this.

The find_long branch (see control_flag here) assigns a value to trade_date. Do I need this in addition to current_date? I created trade_date to mark trade entry dates and on trade entry dates, the two are indeed identical. I could possibly use just current_date, eliminate trade_date, and add some lines to do right here with the former what I ultimately do with the latter. This is worthy of consideration and would eliminate what may be problematic below.

The find_short branch has two conditional blocks—the first of which checks to see if historical date differs from trade_date. This could only occur if one leg of the spread is found, which implies a flaw in the data file. In this case, trade_date is added to missing_s_s_p_dict, control_flag (branch) is changed to find_long, and a continue statement returns the program to the top of the loop where next iteration (row of data file) begins. The process of finding a new trade commences immediately because wait_until_next_day remains False.

The other conditional block in the find_short branch includes a check to make sure historical date equals trade_date. This seems redundant since the first conditional block establishes them to be different. I could probably do one if-elif block instead of two if blocks since the logic surrounding date is mutually exclusive. This may or may not be more efficient, but this second conditional block also has some additional logic designed to locate the short option—additional logic that, as above, could pose a problem if the data file is flawed.

At the end of this second conditional block, wait_until_next_day is set to True but current_date is not assigned. This seems problematic because L66 (see Part 2) will be bypassed, wait_until_next_day will be reset to False, and the flag will basically fail as rows with the same historical date will be subsequently evaluated rather than skipped. Remember, for the sake of efficiency, every trading day should include either finding or updating: not both. How much does this matter?

I will continue next time.

*—Historical date is a column of the data file corresponding to a particular trading day.

Backtester Logic (Part 2)

Last time, I discussed a plan to iterate through data files in [what I think is] an efficient manner by encoding elements only when proper criteria are met. Today I will continue discussing general mechanics of the backtesting program.

While the logic for trade selection depends on strategy details, the overall approach is dictated by the data files. The program should ideally go through the data files just once. Since the files are primarily sorted by historical date and secondarily by DTE, in going from top to bottom the backtester needs to progress through trades from beginning to end and through positions from highest to lowest DTE. This means addressing the long leg of a time spread first as it has a higher DTE.

Finding the long and short options are separate tasks from updating long and short options once selected. When finding, the backtester needs to follow trade entry guidelines. When updating, the backtester needs to locate previously-selected options and evaluate exit criteria to determine whether the trade will continue. I therefore have four branches of program flow: find_long, find_short, update_long, and update_short.

Going back to my post on modular programming, I could conceivably create functions out of the four branches of program flow. Two out of these four code snippets are executed on each historical date because opening and updating a trade are mutually exclusive. I do not see a reason to make functions for this since the program lacks repetitive code.*

Whether finding or updating, once the short option has been addressed the backtester can move forward with calculating position parameters and printing to the results file. For example, in order to calculate P_price = L_price – S_price (see key here), the program needs to complete the find_long / find_short or update_long / update_short branches since the position is a sum of two parts. One row will then be printed to the results file showing trade_status (reviewed in previous link), parameters, and trade statistics.

Since every trading day involves either finding or updating, once the short option has been addressed any remaining rows in the data file with the same historical date may be skipped. For that purpose, I added the wait_until_next_day flag. This is initially set to False and gets toggled to True as needed. Near the beginning of the program, I have:

If the flag is True, then the continue statement will return the program to the top where the next iteration (row of the .csv file) begins. This will repeat until the date has advanced at which point the flag will be reset to False.

To complete our understanding of this matter, I need to analyze where current_date is assigned.

I will continue with that next time.

*—Resetting variables is the one exception I still need to discuss.

Backtester Logic (Part 1)

Now that I have gone over the modules and variables used by the backtester, I will discuss processing of data files.

My partner (second paragraph here) has done some preprocessing by removing rows pertaining to calls, weekly options, deepest ITM and deepest OTM puts, and by removing other unused columns. Each row is a quote for one option with greeks and IV. I tend to forget that I am not looking at complete data files.

As the backtester scans down rows, option quotes are processed from chronologically ordered dates. Each date is organized from highest to lowest DTE, and each DTE is organized from lowest to highest strike price.

My partner had given some thought to building a database but decided to stick with the .csv files. Although I did take some DataCamp mini-courses on databases and SQL, I feel this is mostly over my head. I trust his judgment.

I believe the program will be most efficient (fastest) if it collects all necessary data from each row in a single pass. I can imagine other approaches that would require multiple runs through the data files (or portions therein). As backtester development progresses (e.g. overlapping positions), I will probably have to give this further thought.

The first row and column of the .csv file are not used. The backtester iterates down the rows and encodes data from respective columns when necessary. The first column is never called. Dealing with the first row—a string header with no numbers—requires an explicit bypass. It could raise an exception since the program evaluates numbers. The bypass is accomplished by the first_bar flag (initially set to True) and an outer loop condition:

The first iteration will set the flag to False and skip everything between L58 and L208. The bulk of the program will then apply to the second iteration and beyond.

To maximize efficiency, the backtester should encode as little as possible because most rows are extraneous. Backtesting a time spread strategy that is always in the market with nonoverlapping positions requires monitoring 2 options/day * ~252 trading days/year = 504 rows. My 2015 data file has ~120,000 rows, which means just over 0.4% of the rows are needed.

I will therefore avoid encoding option price, IV, and greeks data unless all criteria are met to identify needed rows.

A row is needed if it has a target strike price and expiration month on a particular date but once again, in the interest of efficiency the backtester need not encode three elements for every row. It can first encode and evaluate the date, then encode and evaluate the expiration month, and finally encode and evaluate the strike price. Each piece of information need be encoded only if the previous is a match. Nested loops seem like a natural fit.

I will continue next time.

Backtester Variables

Last time I discussed modules including those used by an early version of my backtester. Today I introduce the variables.

For any seasoned Python programmer, this post is probably unnecessary. Not only can you understand the variables just by looking at the program, the names themselves make logical sense. This post is really for me.

Without further ado:

• first_bar is Boolean to direct the outer loop. This could be omitted by manually removing the data file headers.
• spread_count is numeric* to count number of trades.
• profit_tgt and max_loss are numerics. These could be made customizable or set as ranges for optimization.
• missing_s_s_p_dict is a dictionary for cases where long strike is found but short strike is not.
• control_flag is string to direct program flow (‘find_long’, ‘find_short’, ‘update_long’, or ‘update_short’).
• wait_until_next_day is Boolean to direct program flow.
• trade_date and current_date are self-explanatory with format “number of days since Jan 1, 1970.”
• P_price, P_delta, and P_theta are self-explanatory with P meaning “position.”
• H_skew and H_skew_orig are self-explanatory with H meaning “horizontal.”
• L_iv_orig and S_iv_orig are long (L) and short (S) original (at trade inception) option IV.
• ROI_current is current (or trade just closed) trade ROI.
• trade_status is a string: ‘INCEPTION’, ‘IN_TRADE’, ‘WINNER’, or ‘LOSER’.
• L_dte_orig and S_dte_orig are days to expiration for long and short options at trade inception, respectively.
• L_strike and S_strike are strike prices (possibly redundant as these should be equal).
• L_exp and S_exp are expiration dates with format “number of days since Jan 1, 1970.”
• S_exp_mo and L_exp_mo are expiration months in 3-letter format.
• L_price_orig and S_price_orig are option prices at trade inception.
• L_delta_orig and S_delta_orig are delta values at trade inception.
• L_theta and S_theta are current theta values.
• spread_width is number of days between long- and short-option expiration dates.
• P_t_d_orig and P_t_d are original and current TD ratio, respectively.
• PnL is trade pnl.
• test_counter counts number of times program reads current line (debugging).
• trade_list is list of trade entry dates (string format).
• P_price_orig_all is list of trade inception spread prices (for graphing purposes).
• P_theta_orig_price (name needs clarification) is position theta normalized for spread price at trade inception.
• P_theta_orig_price_all is list of position theta values normalized for spread price at trade inception (graphing).
• P_theta_all is list of position theta values at trade inception (graphing).
• feed_dir is path for data files.
• strike_file is the results file.
• column_names is first row of the results file.
• btstats is dataframe containing the results [file data].
• mte is numeric input for minimum number of months until long option expiration.
• width is numeric input for number of expiration months between spread legs.
• file is an iterator for data files.
• barfile is an open data file.
• line is an iterator for barfile.
• add_btstats is a row of results to be added to dataframe.
• realized_pnl is self-explanatory numeric used to calculate cumulative pnl.
• trade_dates_in_datetime is list of string (rather than “days since Jan 1, 1970”) trade entry dates.
• marker_list is list of market symbols ( ‘ ‘ or ‘d’).
• xp, yp, and m are iterators to unpack dataframe elements for plotting.
• ticks_array_raw creates tick array for x-axis.
• ticks_to_use determines tick labels for x-axis.

I will have further variables as I continue with program development and I can always follow-up or update as needed.

*—Exercise: write a code overlay that will print out all variable names and respective data types in a program.