Building a fully automated trading system that is profitable and stable is the first challenge in this field. There are 9 steps to develop a trading algorithm as follows.
1. Forming Algorithmic Hypotheses
Traders begin to develop an algorithm according to familiar investment strategies or market assumptions that will likely bring profits. This step is to break down a trading strategy into a set of statements that execute trading logic to make buy or sell orders. These orders have key information like the order type, price, and volume.
After suggesting the trading algorithm, it’s possible to move on to real trading. However, it would be extremely risky and can be a game of chance. To limit the risks, testing phases are required especially with large accounts.
2. Data Collection
In most cases, basic data would satisfy the needs of algorithm development. However, for complex algorithms that may require tick data, financial statements, macroeconomic indices, commodities, or ETF indicators, traders need to collect data before implementing the algorithm.
Around the world, data collection has gone so far as using satellite images to estimate cars in a parking lot, or to estimate the number of sheep on a farm to predict business outcomes. This goes hand in hand with algorithm development to seek profits.
3. Data Processing
This stage requires lots of effort to ensure that collected data is not duplicated. It also removes data outliers and structural errors to ensure sufficient quality data to develop algorithms.
Suppose investors in Vietnam would like to calculate the correlation between PVD shares and crude oil prices. After collecting oil price data, it is necessary to synchronize the time of world oil data with Vietnam time, adjusted for public holidays and any time zone difference. This is a simple example of the data processing operation.
4. Backtesting With In-Sample Data
Backtesting evaluates whether the trading algorithm is profitable. A correct method of testing on the historical in-sample data will give a reasonable expectation of its performance. It also helps understand the algorithm risk in the future.
To test past data, the first step is to divide this dataset into two parts, 70% and 30% of the dataset. The former is called in-sample data and the latter is out-of-sample historical data.
The in-sample data is used to evaluate the trading profile of the algorithm. The core metrics include profitability, maximum drawdown, and Sharpe ratio. Note that an algorithm requires at least 300 transactions to ensure these metrics are statistically significant. Testing on past data should also include taxes, fees, slippage, and any other trading costs incurred.
After estimating the parameter, if the performance is unsatisfactory, return to step 1 to propose a different hypothesis. Otherwise, the process can move to the next step
5. Optimization
In this stage, investors make necessary adjustments to algorithm parameters to find the values that optimize the algorithm’s efficiency in the targeted market. It is necessary to test hundreds of different sets of parameters until a stable area is found. It is important to avoid overfitting, that is the set of parameters only yields good returns in the past but will unlikely be the same in real trading.
In Vietnam, many focus on finding a set of parameters with optimal profit results in past data to attract and sell trading algorithms to customers. Investors who buy and trade according to these algorithms have no idea they are in a highly risky game.
6. Backtesting on Out-Of-Sample Past Data
After the optimization phase, traders continue to test the remaining 30% out-of-sample data from step 4. This process tests whether the post-optimization algorithm can generate stable profits. If the performance is similar to that of step 5, it’s likely a good algorithm with strong potentials. Otherwise, investors should be cautious if they want to proceed.
7. Paper Testing
This stage tests the algorithm on completely unknown market data. The data comes from real data at the present until a certain point in time in the future. It is to objectively evaluate the algorithm’s performance. If it is similar to that of step 6, investors can move to the next stage of testing on small accounts.
8. Small Account Testing
After the 7 stages above, it’s now possible to deploy the algorithm on a small account to test the performance, as well as the algorithm hypothesis during real trading. This stage also finds possible technical bugs and compares the result with the paper trading stage to make any adjustments if necessary.
This period usually lasts 2 months to have a large enough number of transactions and a long enough time to assess the stability of the algorithmic trading system. If the performance profile is still consistent with that of previous stages, investors now have confidence and scientific grounds to apply to real trades.
9. Real Trading
At this point, investors can confidently let the algorithm trade at maximum capacity. In the early times, it’s necessary to monitor the operation and fix any technical bugs that may arise. After 3 months, investors can officially launch a new trading algorithm if the operation proves stable.
Don’t forget to get a cool and catchy name for your brand-new trading algorithm!