The Bartels test returns a value that gives the measure of the likelihood of genuineness of a cycle: values range from 0 up to 1, and the lower the value, the less likely is that this cycle is due to chance, or random.

The test considers both the consistency and the persistence of a given cycle within the data set it is applied to. To make it more human readable as we are looking for an easily readable indication if the cycle is genuine, we just convert the raw Bartels value into a percentage that indicates how likely the cycle is genuine by using the conversion formula:**Cycle Score Genuine % = (1 - Bartels Score) \* 100**

It gives us a value between 0% (random) and 100% (genuine). This test helps us now to filter out possible cycles that might have been detected in the cycle detection step (Step 2), but had only been in the data series for a short or random period and should therefore not be considered as dominant cycles in the underlying original data series. As we have a final percentage score, we just need to define an individual threshold below which the cycles should be skipped. We recommend using a threshold of >49% and hence cycles with a Bartels genuine percentage value below 49% should be skipped by any cycle forecasting or analysis techniques that follow. Further Reference: [\[1\]](#_ftnref1) C. E. Armstrong, Cycles Magazine, October 1973, p. 231ff, "Part 25: Testing Cycles for Statistical Significance" (see pdf attachment) # Ranking An important final step in making sense of the cyclic information is to establish a measurement for the strength of a cycle. Once ranking and sorting for detected cycles is completed, we have cycles that are dominant (based on their amplitude) and genuine (considering their driving force in the financial market). For trading purposes, this does not suffice. The price influence of a cycle per bar on the trading chart is the most crucial information. Let me give you some examples by comparing two cycles. One cycle has a wavelength of 110 bars and an amplitude of 300. The other cycle has a wavelength of 60 bars and an amplitude of only 200. So, if we apply the “standard” method for determining the dominant cycle, namely selecting the cycle with the highest amplitude, we would select the cycle with the wavelength of 110 and the amplitude of 300. But let us look at the following information - the force of the cycle per bar: - Length 110 / Amplitude 300 = Strength per bar: 300 / 110 = 2.7 - Length 60 / Amplitude 200 = Strength per bar: 200 / 60 = 3.3 For trading, it is more important to know which cycle has the biggest influence to drive the price per bar, and not only which cycle has the highest amplitude! That is the reason I am introducing the measurement value “**Cycle Strength**.” The Cycle Scanner automatically calculates this value. That said, to build a ranking based on the cycles left, we recommend sorting these cycles based on their "influence" per bar. As we are looking for the most dominant cycles, these are the cycles that influence the movement of the data-series the most per single bar. Sort the outcome according to the calculated cycle strength score. Now we have a top-to-bottom list of cycles having the highest influence on price movements per bar. #### What is the dominant cycle? After the cycle scanner engine has completed all steps (detrend, detect, validate, rank), the cycle at the top of the list (with the highest cycle strength score) will provide us the information on the ***dominant cycle***. In fact, the wavelength of this cycle is the dominant market vibration, which is very useful for cycle prediction and forecasting. However, not only is the result limited to the cycle length (we not only have the dominant cycle length) but we also know—and this is very important—the current phase status of this cycle (Important: not the averaged phase over the full data set). This allows us to provide more valid cycle projections on the "right" side of the chart for trading instead of using the normally used "averaged" phase status over the full data set for this cycle. # Spectral Averaging #### Cycles are not static, so what? Most cycle analysts have seen the moment-to-moment fluctuations, often referred to as shifts in the length and phase of a cycle, which are common in continuous measurements of a supposedly steady spectrum of financial records. But wait, we should know that cycles are not static in real life. We can see these variations in size and phase for each cycle length in the updated spectrogram after a new data point/bar is added to the data set. Although small variations are certainly no cause for concern, we should remember that we do not live in an ideal, noise-free world, but we are always looking for ways to reduce the phase shifts and variations in dominant cycles. Therefore, instead of additional smoothing of the input data, we will apply averaging to the received cycle spectrum. The basic idea of averaging to reduce spectral noise is the same as averaging - or smoothing - the input signal. However, touching the raw input signal with the averaging reduces data accuracy and results in a delay of the signal. This is not what we need, especially because the end of the data set - the current time - is of the utmost importance when working with cycles for financial time series data. Therefore, averaging the input and adding a delay to our series of interest is a bad idea for cycle analysis in data sets where the most recent current data points are more important than data points from long ago. #### Dont smooth the input - average the spectrum! Averaging a spectrum can reduce fluctuations in cycle measurements, making it an important part of spectrum measurements without changing the input signal or adding delay. Spectral averaging offers a different approach and is a kind of ensemble averaging, meaning that the "sample" and "average" are both cycle spectra. The "average" spectrum is obtained by averaging the "sample" spectra. However, due to the nature of the spectra, this is not as simple as this. If you apply a discrete Fourier transform routine to a set of real-world samples to find a spectrum, the output is a set of complex numbers representing the magnitude and phase of that spectrum. Calculating an average spectrum involves averaging over common frequencies in several spectra.**Spectral averaging eliminates the effect of phase noise**.

The size of the spectrum is independent of time shifts in the input signal, but the phase can change with each data set. By averaging the power spectra and taking the square root of the result, we eliminate the effect of phase variation. **Reducing the noise variance helps us to distinguish small real cycles from the largest noisy peaks.** The result of spectral averaging is an estimate of the spectrum containing the same amount of energy as the source. Although the noise energy is preserved, the variance (noise fluctuation) in the spectrum is reduced. This reduction can help us reduce the detection of "false cycles" that are the result of single, one-time "noise" peaks. Lets illustrate the concept of spectral averaging with a simple example. This method is a derivative of the so-called Bartlett method, if no other window than a rectangular window is applied to the data sections. #### Cycle detection example We use a constructed, simplified data-set which consists of 2 cycles, noise and different trends. The two cycles have a length of 45 and 80 days: [![image-1597074936323.png](https://docs.cycle.tools/uploads/images/gallery/2020-08/scaled-1680-/image-1597074936323.png)](https://docs.cycle.tools/uploads/images/gallery/2020-08/image-1597074936323.png) Raw input signal consisting of cycles, noise and trends Let us have a look at the different spectrogram results *without* (1) and *with* (2) spectral averaging. The first diagram shows the basic amplitude spectrum of the signal without spectral averaging. [![image-1597075174397.png](https://docs.cycle.tools/uploads/images/gallery/2020-08/scaled-1680-/image-1597075174397.png)](https://docs.cycle.tools/uploads/images/gallery/2020-08/image-1597075174397.png) Chart 1: Cycle spectrum without spectrum averaging It clearly shows the cycles with a amplitude peak at 45 and 80 days, but you can identify lower peaks in the spectrum that have nothing in common with real cycles of the original dataset. These smaller amplitude peaks are "false cycle" - noise only. We want to avoid using these cycles in cycle forecasting models. Since it is important to identify and separate peaks based on noise - we will see if the method of averaging in the spectrum offers some added value. The next diagram uses the same input signal, but runs not just once to generate the spectrum, but several spectra to form the spectral average. In our case, the window only changes the beginning of the series and always uses the same end point for each spectrum. This ensures that we have overlapping windows, always using the last available closer/bar and changing only the beginning of the series. [![image-1597075260299.png](https://docs.cycle.tools/uploads/images/gallery/2020-08/scaled-1680-/image-1597075260299.png)](https://docs.cycle.tools/uploads/images/gallery/2020-08/image-1597075260299.png) Chart 2: Cycle spectrum with spectrum averaging The result illustrates that the peak values for 45 and 80 days are still present. But now the "noise" level has become much lower, and due to the averaging of the spectral window analysis, we have a lower number of "false cycle peaks". This will help us distinguish between important and unimportant cycles for future cycle prediction modeling techniques. References: - (2008) "The Spectral Analysis of Random Signals", in: Digital Signal Processing. Signals and Communication Technology. Springer, London. https://doi.org/10.1007/978-1-84800-119-0\_7 - Oppenheim and Schafer (2009): "Discrete-Time Signal Processing", Chapter 10, Prentice Hall, 3rd edition. - Bartlett (1950): "PERIODOGRAM ANALYSIS AND CONTINUOUS SPECTRA", Biometrika, Volume 37, Issue 1-2, June 1950, Pages 1–16, https://doi.org/10.1093/biomet/37.1-2.1 - Wikipedia: "Bartlett Method", [https://en.wikipedia.org/wiki/Bartlett%27s\_method](https://en.wikipedia.org/wiki/Bartlett%27s_method) - Shearman (2018): "Take Control of Noise with Spectral Averaging", https://www.dsprelated.com/showarticle/1159.php # Endpoint flattening The digital signal processing (e.g. Discrete Fourier Fransform) assumes that the time domain dataset is periodic and repeats. Suppose a price series starts at 3200 and toggles and wobbles for 800 data samples and ends at the value 2400. The DFT assumes that the price series starts at zero, suddenly jumps to 3200, goes to 2400, and suddenly jumps back to zero and then repeats. The DFT has to create all sorts of different frequencies in the frequency domain to try to achieve this kind of behavior. These false frequencies, generated to match the jumps and the high average price, mask the amplitudes of the true frequencies and make them look like noise.Fortunately, this effect can be nearly eliminated by a simple technique called **endpoint flattening**.

##### Example The following chart shows an example data series (green) and the de-trended data at the bottom panel (gold) without endpoint flattening: [![image-1626255996599.png](https://docs.cycle.tools/uploads/images/gallery/2021-07/scaled-1680-/image-1626255996599.png)](https://docs.cycle.tools/uploads/images/gallery/2021-07/image-1626255996599.png) The next example shows the same data series now with endpoint flattening applied to the detrended series: [![image-1626256103968.png](https://docs.cycle.tools/uploads/images/gallery/2021-07/scaled-1680-/image-1626256103968.png)](https://docs.cycle.tools/uploads/images/gallery/2021-07/image-1626256103968.png) The difference is only visible at the beginning and the end on both de-trended series. While the first one starts below 0 and ends well above 0, the second chart shows that the de-trended series starts and ends at zero. ##### Math formula Calculating the coefficients for endpoint flattening is simple: Taking n closing prices. If x(1) represents the first price in the sampled data series, x(n) represents the last point in the data series, and x