The topic below describes typical techniques for trading system creation. Everybody who is interested may Google terms mentioned for better understanding:

* Trading models should be checked for statistical validity. In statistics ability of model to adjust to data sample without predicting power is know as degrees-of-freedom (equals to data sample). The formula exists in deriving impact of optimization inflated correlation (R) given the true correlation (RC) for model with P number of parameters and data sample N

R = SQRT(1-((N-P)/(N-1))*(1-RC^2))

Such model is not precises for trading system model with non linear regression or classification models, but brigs the picture how over optimization may negatively impacted if system has large number of parameters and rules and small data set.

* Besides analytical approach in estimating trading system degradation because of increased degrees-of-freedom the practical data mining validation approach is used for checking system performance with out-of-sample usage the principle is also know as walk-forward testing.

* Even if model's Degrees Of Freedom is constrained, it is tested with Out Of Sample data sets and shows positive results the sufficient number of trades should be collected for estimating average behavior of model outcomes (results from trades). The statistical term is know as The Law Of Large Numbers. e.g. The confidence in estimated profitability figures of trading system with > 400 trades are much greater comparing to 20 trades.

* The output of model should be critically evaluated by applying simulation of system trading with warring parameters. The method is known as Monte Carlo method.

* Even simple model optimization with huge amount using brute force algorithms requires lot of computational resources. The global optimization algorithms such as Genetic Algorithms provides the better way of achieving better results with less computational resources.

* Besides checking characteristics of each system individually, the portfolio of systems is validated. The Modern portfolio theory can be applied for combining different trading systems into portfolio. The leverage of different systems may be combined to get efficient frontier (optimal for testing set risk adjusted reward) portfolio. The portfolio is created from small number of systems with outcomes negative and/or very low correlations. Such approach is supported both by analytical analysis and practical production ability in monitoring stability of small covariation matrix over long period.


Thanks for reading,
pranasblk

Translations and notes are welcome!

P.S. The post is based on my promise to Spekuliantai group stake holders. I guess everybody who want to achieve some results in this particular are should know English