Many of the algorithm being used to predict future price of a financial instrument such as: simple statistics, linear regression, time series, reinforcement learning, sentiment, machine learning and artificial intelligence contain four basic assumptions: 

The Four Basic Assumptions

1. Independent and identically distributed data (IID) data. This essentially means that you assume that there is some true but unknown data distribution from which each of your training and test points are drawn independently.

2. The true function mapping inputs to outputs is smooth almost everywhere. Essentially, you assume that there is some true but unknown function that maps the inputs to the outputs. The goal is to approximate this true function, so that when given a new input, you can use this function to predict the output. You expect the predicted output to be reasonable because hopefully the model has seen similar inputs in training data, and the function can approximate the output by combining the information from those training data points. If the true function is smooth, this works. If not, then looking at similar inputs in training data tell you very little about this new data point.

3. Signal to noise ratio is large. The noise in input features and labels is assumed to be low.

4. Unmodelled features have negligible effects. You represent any input as a set of features, e.g., a price and volume as past performance metrics. Now, of course, there are other variables in the environment that you ignore in the process. If the other parameters accounted for more than 80% of the effect, then there will be no true model.

Do These Assumptions Hold True in Algorithmic Trading?

1. Independent and identically distributed data (IID) data. This is the same as past performance indicates future performance which should not be true in the financial market. Perhaps, there are some traits in the data set that are identical but for the whole data set in the financial market to be identical distributed in the past as well as the future seems very unlikely. 

2. The true function mapping inputs to outputs is smooth almost everywhere. This is the same way to say there is a true holy grail in the financial market which can predict the outcomes in the future with 100% accuracy. If this assumption holds true then there will be an absolute winner. By empirical evidence of no huge successful big firm with more than 1000% return annually and Warren Buffet with 23% annually return still the most admired investor, we may think that there is no such true function. 

3. Signal to noise ratio is large. It may be the opposite in financial data, in which the majority of data is noise or randomness.  

4. Unmodelled features have negligible effects. It is very well known that stock market is the leading indicator of the social economy, and the social is under effect of a lot of unknown and unexpected features such as natural disaster, pandemic, war. 

This discussion, even not yet sophisticated proven, may give a hunch that the approach in the algorithmic trading industry should not be identical as some clear rules game like chess.    

Applying these thought models, one may come up with the current status of the algorithmic industry, using statistics for finding features that long lasting in the future to achieve superior return after utilizing the law of large numbers. In case of alpha decay, with or without reasoning, the solution is to find another alpha to maintain the trading system performance.   

This approach, which has been applied by many well known quantitative trading firms, signals it is good enough to survive the test of time. To further improve our thought model, we may take a look at the idea of market evolution. 

Market Evolution

Market, as an entity, combines many factors. Some have a huge impact on market movement, while others have minimal impact. By quantifying any amount of impactful factors, algorithmic traders may find a good model such as smart beta strategy. Or, by naturalizing a lot of factors to reduce noise, also help enhance model accuracy even with weak prediction factors such as market neutral strategy. Or, focusing on a time frame that most factors cannot affect the market to reduce noise and increase accuracy such as scalping trading strategy.

Regardless of the approach, the market will change as its core pillar changes overtime. Some core pillars in each market may be named as: cooperation aggregation, regulation, environment and human nature.  

Once a significant change is made or a combination of small changes is too impactful, a market may reach a tipping point and transform itself into a new state. We may call it market evolution.

Market evolution is an event that shifts the market nature permanently from state A to state B and redefines a set of impactful factors in the process.

When market evolution happens, factors that impact the market will change accordingly. Some that are very impactful may no longer hold prediction value. 

The dilemma is, in light of simplicity, regardless of what change in the nature of the market, it represents no changes in terms of price and volume as hundreds of years. To cut it straight, the market has evolved to a completely new state and without notice, algorithmic traders will leverage obsolete models. This should not result in anything but randomness. 

So the point here, the market does evolve. And once it does, there is no way back just like the nature of evolution. 

Question Regarding Market Evolution?

At Algotrade, we are trying to embrace and nurture the ideas though the result is still very vague. Following is a list of questions we are trying to solve.

• Which factor will mark the market evolution? Is it a new law, a pandemic or any tech disruption can make it happen?

• How long does it take to transition completely from the old state to the new one?

• Does previous data have less value after market evolution?

• After the market evolution, there is little to no data in the new state, how can we build models with very minimal data?   

• Can social prediction models be employed in the financial market? 

We are still at the very beginning of discovering this track. We found the topic of complex systems may aid. Here is the resource if you want to tag along: https://www.santafe.edu/