Testing the performance of technical trading rules in the Chinese market

Technical trading rules have been widely used to detect the market trends for financial practitioners. In academia, numerous studies have been conducted to try to answer the question whether technical strategies are useful through applying trading rules to different financial markets, but instead give conflicting conclusions. On the one hand, some scholars advocate that technical rules do have predictive ability to earn excess profits. Treynor and Ferguson found that it is possible to get the abnormal profits with the past prices and other valuable information [1]. Brock et al showed that in the U.S. market, technical trading rules could reveal some certain return patterns when applied to the Dow Jones Industrial Average (DJIA) [2]. In the European monetary system, Neely et al reported that technical trading rules could be useful even with the use of out-of-sample test [3]. On the other hand, technical rules are found to be useless by other researchers. Lucke carried out a test on the effectiveness of head-and-shoulder trading rules on foreign exchange markets and found that returns to head-and-shoulder trading rules were not significantly positive [4]. Anderson and Faff found that in futures markets, the profit of technical trading rules was not significantly obtainable [5]. To investigate the efficiency of technical trading rules, Kung focused on profits produced by buying signals and selling signals, using the Taiwan Stock Exchange Weighted Index and found that for these rules, returns from buy signals are higher than those from sell signals. The predictive power became less effective over 1997-2007 when compared with periods 1975-1985 and 1986-1996, indicating an improvement of efficiency of the Taiwan stock market. [6]. Fang used the out-of-sample test, and found that no technical trading rule had the predictive ability [7]. Obviously, the profitability of technical trading strategies depends on the states of the market and the choice of specific trading rules. Yu et al argued that the predictive ability of technical trading rules appears only when the market is less efficient [8]. Hudson et al tested the possibility of earning excess returns by using technical analysis in the UK market and found that although the technical trading rules do have predictive ability, it considerably weakens when one takes into account trading costs [9].

The effectiveness of technical trading rules is further in doubt due to the data-snooping effect, which occurs when the same data is used more than once for the purpose of inference or model selection. If the best model is obtained by an extensive specification search, it will be highly possible that this good model is chosen by luck rather than its actual forecasting ability [10]. The existence of dependence between all the tested trading rules imply that the data-snooping effect would inevitably amplify the significance levels of conventional hypothesis tests. Jensen and Bennington demonstrated that data-snooping on the performance of technical trading rules was a selection bias [11]. Lo and Mac Kinlay showed that effects of data-snooping were substantial by using analytical calculation and simulations [12]. Brock et al also recognized the existence of data-snooping and evaluate by fitting several models to the raw data and created new sample series by resampling the residuals [2]. They applied 26 simple trading rules, including moving average and range break-out rules, to the US stock markets and found that technical analysis was indeed capable of providing significant economic contents.

In order to correct the biased effects induced by data-snooping, a statistical test, called “White’s Reality Check” (abbreviated as WRC), was proposed to examine whether the profit of technical trading rules is true or just from luck [10]. Sullivan el al also applied this test to check whether the technical strategies ($7846$ trading rules) had the predictive power in DJIA and S&P 500 index and found that the superior performance of the best trading rule could beat the benchmark, but could not repeat in the out-of-sample experiment [13]. Chen et al tested the same trading strategies as in Ref. [13] in Asian stock markets and found that the WRC $p$-values of different markets were not the same. They also found that the predictive ability in Asian stock markets was not as good as that in the US.

Due to the deficiency of WRC tests whose $p$-value would increase when bad trading rules produced negative performance measurements, the Superior Predictive Ability (SPA) test was proposed to check the existence of predictive ability of a superior model [14]. Hsu and Kuan reexamined the profitability of technical trading rule set in four indexes by using both WRC and SPA tests and found that the trading rules could only earn profits in young markets (NASDAQ Composite and Russell) rather than in developed markets (DJIA and S&P 500). Hsu et al further extended the WRC and SPA test into a stepwise SPA test and a stepwise Reality Check test. With these two extended tests, they examined the predictive ability of technical trading rules in emerging markets, and demonstrated that technical trading rules do have significant predictive abilities. Park et al used both WRC and SPA tests to investigate the profitability of technical trading rules in U.S futures markets and found that the best trading rules generated statistically significant economic profits only for some futures contracts [15]. This suggests that technical trading rules do not generally have the profitable ability in U.S. futures markets when take into account data-snooping biases. Shynkevich applied the technical trading rules to the growth and small cap segments of the US equity market, and found that mechanical trading strategies had lost their predictive ability when data-snooping was considered [16].

In this paper, we will apply the 7846 trading rules into Shanghai Securities Composite Index (SSCI) and Shanghai Shenzhen 300 Index (SHSZ 300) in the Chinese stock markets to study whether any of these technical trading rules is capable of making profits under the consideration of the data-snooping effect. This paper is arranged as follows. In Section 2, we introduce the data information. All the trading rules and the benchmark are presented in Section 3. In Section 4, we will describe details of the SPA test. In Section 5, we will present the empirical results. Section 6 is the conclusion.

We apply the trading rules in our strategy pools on two important indexes (SSCI and SHSZ 300) in Chinese stock markets to investigate which technical strategy has the best performance by means of the SPA tests. Both daily indexes are provided by the financial data company RESSET and China Investment Security database. The SSCI covers a period from May 21, 1992 through June 30, 2013, which leads to 5144 data points. SHSZ 300 spans a period ranging from April 8, 2005 and June 30, 2013, which results in 1996 data points. We estimate the daily logarithmic difference return and find that the largest daily return is $28.86\%$ for SSCI and $8.93\%$ for SHSZ 300. The smallest return for SSCI and SHSZ 300 is $-17.91\%$ and $-9.70\%$. We also find that there are 32 returns whose values are larger than $10\%$ or less than $-10\%$ for SSCI. All these returns occurred before Dec 16, 1996 when the price limit rule was launched in Chinese markets. We also estimate the skewness and kurtosis of daily returns for both indexes. The skewness of the SSCI returns is 1.18, which indicates that the return distribution is right skewed. For SHSZ 300, the return distribution is a little bit left skewed and the corresponding skewness is -0.37. The kurtosis values of the returns for both indexes are greater than 3, which is a typical value of normal distributions. For the SSCI, the kurtosis is 22.07, which is about four times greater than the kurtosis 5.71 for SHSZ 300. This means that the return distribution possesses the characteristic of leptokurtic.

To begin with, it is necessary to specify all the technical trading rules which will be tested. Brock et al have used 26 simple trading rules to test the performance of technical trading strategies in the U.S. market [2]. Sullivan et al. extended these simple trading rules to a larger universal technical analysis space [13], which were also tested in Ref. [17]. Our analysis is also based on this universal technical analysis space, which comprises 7846 trading rules from five main technical analysis catalogs, namely filter rules, moving averages, support and resistance, channel breakouts, and on-balance volume averages.

In most of the existing work in the literature on technical trading rules, the most profitable trading rules are usually obtained by applying a pool of trading rules to past data. As a result, some of the trading rules may be selected as useful just by chance rather than their actual forecasting ability, which is mostly a result due to the data-snooping effect. The most popular testing methods taking into account to correct the data-snooping effect are WRC tests [10] and SPA tests [14]. Note that SPA tests are modified from WRC tests to offer more reliable calculations. In order to test whether the technical rules have the capacity to predict and produce profits after accounting for the effects of data-snooping in Chinese stock markets, we here choose the SPA test to perform our statistical tests.

To check whether the best trading rule in the strategy pool can provide significant performance statistics (return or Sharpe ratio) under the consideration of the data snooping effect, we perform back tests on two important indexes (SSCI and SHSZ 300) in Chinese stock markets. We first evaluate the performance of the trading rules on the two indexes in the whole sample period. Figure 1 (a) illustrates the performance statistics based on the returns for each trading rule on SSCI. The dots represent the performance statistic $\langle f^{r}\rangle$ for each trading rule. We simply set the performance statistic value to be zero for the trading rules that generate no signal. Note that the value of $\langle f^{r}\rangle$ is annualized by multiplying $252$. The solid line stands for the maximum value of performance statistics achieved by testing the trading rules one by one. One can observe that the final maximum value is $26.13\%$ generated by the $2$ and $20$-day moving averages. Figure 1 (b) illustrates the performance statistics based on the Sharpe ratio for each trading rule on SSCI. The performance statistics $\langle f^{s}\rangle$ are plotted as dots for each trading rule. Note that the trading rules with zero return performance statistics are excluded. We also do not show 22 Sharpe ratio performance statistics whose values are less than $-0.2$, since we are more interested in the large and position values. The maximum values $\langle f^{s}\rangle$ can be achieved as $0.0622$, generated by the channel breakouts rule with the number of days of forming a channel, the band and the number of holding days to be 20, 10% and 5 respectively. Figure  1 (c, d) illustrates the performance statistics based on the returns and Sharpe ratios of the trading rules on the Shanghai Shenzhen 300 Index. For return statistics, the highest value is $41.69\%$. For Sharpe ratio statistics, the largest value is 0.09, and the smallest value is $-0.47$. Both values are generated by the on-balance volume rule with the $5$ and $20$-day on-balance volume average and $5$ signal-valid days.

During the procedure of back tests on technical trading strategies, the data-snooping effect that may occur for the testing series is repeatedly used. To correct for this data-snooping effect, we adopt the SPA test to check whether the predictive ability of the best trading rule in the strategy pool is true or just from luck. Figure 2 illustrates the $p$-value of SPA tests for performance measurements on both indexes with the block shuffling parameter to be $q=0.1$. The shadow areas correspond to the upper and lower bounds of the $p$-value. The dash lines represent the $p$-value of the tested trading rules which were picked up one by one from the strategy pool. Plots of other values of $q$ exhibit very similar patterns and we do not show them here. We can see that at the significance level of 0.1, the tests of both performance measurements on SSCI can be definitely rejected, indicating that technical trading rules have predictive ability for SSCI. This result is further consolidated by the $p$-values listed in the first row of Panel A in Table 1 and 2. However, the rejection of the tests on SHSZ 300 is failed for both performance measurements, as shown in Fig. 2 (b) and the first row of Panel B in Table 1 and 2, which means the same trading strategies are ineffective for SHSZ 300.

In order to understand why the same trading strategies give absolutely different results by applying on SSCI and SHSZ 300, which are highly correlated and share very similar behaviors, we first check whether these consequences are attributed to the differences between the data spanning periods. By separating the SSCI indexes at the time point April 8, 2005, which results in two sub-series and the latter series spans the same period as SHSZ 300, the same trading rule testing procedures are carried out on both sub-series. For the performance measurement based on returns, we find that there are 3 $p$-values (corresponding to $q=0.01$, $0.02$ and $0.05$) less than 0.1 for the first sub-series, as listed in the second column of Panel A in Table 1 and there is only one $p$-value (corresponding to $q=0.01$) less than 0.1 for the second subseries, as in the third column of Panel A in Table 1. For the performance measurement based the Sharpe ratio, one can observe that only the $p$-value of $q=0.01$ are less than 0.1 for the first subseries, as reported in the second column of Panel A in Table 2 and the $p$-values of $q=0.01$ and $q=0.02$ are less than 0.1 for the second sub series, as shown in the third column of Panel A in Table 2. In some sense, the results of the second sub-series are consistent with those of SHSZ 300. As our benchmark is out of market, we conjecture that the trading rules can make profits only when the price trajectories are in the rising and declining regions. Compared with the whole series of SSCI, both sub-series do not exhibit such prominent up trend behaviors. This is why the testing results two sub-series of SSCI and SHSZ 300 are not significant for all the values of $q$.

To further check the conjecture that our trading rules can only earn money in the market with prominent trends, we perform the trading rule tests on both indexes with a five-year moving window. As shown in Tables 1 and 2, all of the series in the windows of 2005-2009 pass the SPA tests based on the return and the Sharpe ratio performance measurements at the 0.1 significance level for different values of $q$. During that period, SSCI rose from 998 to 6124 and a bubble was diagnosed from mid-2005 to October 2007 [20]. For the other periods, no such obviously rising trends are observed, which gives rise to the fact that not all $p$-values for the different values of $q$ are less than 0.1. From our testing results, we also find that the values of the block shuffling parameter $q$ does have influences on the significance of the SPA tests. By increasing $q$ from 0.01 to 1, the number of tests (including all time windows for both indexes) whose hypothesises are rejected exhibit a first decreasing then increasing pattern, where the lowest number is located at around $q=0.1$.

In this paper, we use the SPA test [14] to demonstrate how to correct for the influence of the data-snooping effect when one tries to look for profitable trading rules from a pool of trading strategies by repeatedly using a given financial price series. Taking SSCI and SHSZ 300 as examples, we find that technical trading rules can make reliable profits in SSCI while losing the profitable ability in SHSZ 300. This contradicting consequence can be attributed to the fact that there is a more clear price trend in SSCI, which is the base of technical trading rules. By applying the trading rules on the sub-series of SSCI with the same spanning period as that of SHSZ 300, we find that the efficiency of the technical trading rules is greatly reduced. From the back tests in each moving window with a size of five years, we find that periods which contain extreme upward or downward trends such as financial bubbles, offer good opportunities for investors to make profits by the adoption of technical trading strategies.

This work was partially supported by the National Natural Science Foundation of China (11075054 and 71131007), Shanghai “Chen Guang” Project (2012CG34), Program for Changjiang Scholars and Innovative Research Team in University (IRT1028), and the Fundamental Research Funds for the Central Universities.