6.2 Random Number Generator Failure Experiments
6.2.6 Failure Point 2 Experiment: External Frequency Interference
94 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
0 500 1000 1500 2000 2500 3000 3500 4000
data 2
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
0 500 1000 1500 2000 2500 3000 3500 4000
data 1
a) Single frequency example. b) Wide frequency group example.
Figure 6.19:Frequency spectrum of single frequency and wide frequency group example.
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 95 Single Frequency Wide Frequency Group
Frequency 4000 [3800 3850 3900 3950 4000 4050 4100 4150 4200]
Magnitude 1.0 [0.2 0.4 0.6 0.8 1.0 0.8 0.6 0.4 0.2]
Table 6.3:Experimental settings for the single and wide frequency group used with the freqaddRNG function.
sample random sequences, the frequencies to be added and the magnitude for each of the new frequencies.
The experiment parameters testing are found in Table 6.3. Using these settings the freqad-dRNGfunction adds the interference signal at different ratios. For these experiments the ratios used are 0%, 10%, 30%, 50%, 70%, and 90%. Only the 50% and 90% levels are to be discussed in this thesis to show the effect the interference has on the random data.
Results for the Frequency Addition (Single Frequency)
Thefirst results examined come from the single frequency interference experiment. Many of the observations for the addition of a single frequency component also apply to the addition of the wide frequency group to the random signal. The results for both error types have been graphed.
It is impossible to determine before hand the strength of the interference, so only the trend of how the random number generator tests react at different interference levels can be examined.
The results of the signal at no interference (Matlab RNG results), 50% interference, and 90%
interference are studied here.
The single frequency component results can be seen in Figures 6.22 to 6.24. One of the methods used to determine if this type of error is present in a RNG is to perform the spectral analysis. If present, a frequency component is clearly apparent, and a significance level can be set where any frequency component passing this level indicates a defective RNG. Currently, it is not possible to implement an efficient spectral test on the smart card due to the complexity of required operations; for example, the fast Fourier transform.
Figure 6.20 displays an example sequence using 5 points/cycle and a sample length of 50 points, where the x-axis shows the point count and the y-axis the random value between -1 and 1. It is clear the influence the sine wave signal has on the random data. In this particular case the random data is concentrated in the lower region (−0.50 to 0.50) where more zeros are likely to occur. The addition of the sine wave also adds a pattern to the data where every 2.5 cycles the chance of ones increases. For the 50% case, the basic pattern for the random data is still evident;
however, for the 90% ratio experiment (ie. extreme interference) the random data only has a minor influence on the output data, which shows a lot of regularity (see Figure 6.20b).
The conclusion drawn from analyzing the sample data is that the data generated by this RNG is not acceptable for cryptographic applications. Thefirst experimental sequence to be studied in detail is the 50% single frequency generator. Figure 6.21 reveals that not all of the sequences
96 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 10 20 30 40 50 60
−1
−0.8
−0.6
−0.4
−0.2 0 0.2 0.4 0.6 0.8
1 Sine Wave
Random Signal Combined Signal
a) 50% interference
0 5 10 15 20 25 30 35 40 45 50
−1
−0.8
−0.6
−0.4
−0.2 0 0.2 0.4 0.6 0.8 1
Sine Wave Random Signal Combined Signal
b) 90% interference
Figure 6.20:Example sine wave interference with random data.
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 97
0 50 100 150 200 250 300 350 400 450 500 550
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
Countof"Pass"Sequences(max.500)
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial FIPS @ 20000
Figure 6.21:Single test “pass” count for the Frequency Add Narrow generator with 50% interference.
have been rejected by the FIPS standard. Approximately 70% of the sequences have been re-jected. The FIPS standard has rejected more sequences than any single test; therefore, the test combinations should provide better matching than the single tests. It should also be noted that the FIPS test group is used as the standard measurement for generator randomness. Therefore, the higher rejection at longer sequences for the poker, frequency and serial tests will show up as a lower matching value. Another observation from this graph is that the tests only start to reject the sequences at the 5000 bit length mark.
Figure 6.21 shows some of the same observations that Figure 6.22 displays. The constant matching sits approximately at 30% at the lower bit sample lengths, because the FIPS standard only rejects about 70% of the sequences at the 20000 bit mark. The lower sequence lengths passed all the sequences, so the starting mark for a poor test for the generator is set at 30%. The three tests that start to catch the failure in the generator are the poker, frequency and the serial test. They steadily improve until the 20000 bit point after which they decrease in matching. As has been previously explained, the loss in matching is due to the particular tests labeling more sequences as fail than the FIPS, which is a deviation from the standard laid out at the beginning of this chapter.
The best test, the frequency test, does not achieve 100% matching with the FIPS standard. It only has a 92% success rate; whereas, the next two tests, serial and poker tests, have a 82% and a 79% maximum success rate, respectively. The highest matching percentage occurs only at the 20000 bit mark.
98 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
PercentMatching
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial Test
Figure 6.22:Single test percent matching with FIPS 140-2 results for the Frequency Add Narrow gener-ator with 50% interference.
After looking at the single test results, it is hoped that the combination tests provide better re-sults. As has been mentioned in previous paragraphs, the indication from the “pass” count graph is that the test do not fully overlap when rejecting the various sample sequences. This should show up in the combination test results with better matching for some combinations. The results are presented in Figure 6.23 and 6.24. Here it is seen that the frequency and poker test achieve 100% matching; however, this occurs only at 20000 bits. The rest of the test combinations only show a slight or no improvement (0 to 1%) over the single tests.
Before drawing anyfinal conclusions about the addition of a single frequency to a random bit stream experiment, an extreme case of interference is examined. The single RNG test results and the total “pass” counts are shown in Figures 6.25 and 6.26.
Even with the extreme sinusoidal interference, the RNG tests are not able to recognize that there is a failure occurring with the RNG until the test sequence is at least 2500 bits. Only at a test sequence length of 15000 bits does one test, the poker test, achieve 100% matching. The poker test is by far the most sensitive test for this type of failure. Looking at the results from the number of “passes” given to the sample sequences, the FIPS test rejects all the samples, as does the poker test. However, three of the eight tests do not detect any failure with one test only able to slightly detect the failure. This can be explained by the regular swing in the sine wave interference, which is hard to detect for the frequency type tests. In this case, the average value from the sine wave is zero; therefore, it swings between the maximum and minimum value, but the number of zeros and ones is approximately equal. Refer to Figure 6.27 for an example binary
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 99
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Lengths
PercentMatching
F_R_FIPS F_L_FIPS F_P_FIPS F_T_FIPS F_A_FIPS F_FB_FIPS F_S_FIPS R_L_FIPS R_P_FIPS R_T_FIPS R_A_FIPS R_FB_FIPS R_S_FIPS L_P_FIPS
Figure 6.23:Test combination percent matching with FIPS 140-2 results for the Frequency Add Nar-row generator with 50% interference showing the combinations Frequency/Runs to Longest Runs/Poker.
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Lengths
PercentMatching
L_T_FIPS L_A_FIPS L_FB_FIPS L_S_FIPS P_T_FIPS P_A_FIPS P_FB_FIPS P_S_FIPS T_A_FIPS T_FB_FIPS T_S_FIPS A_FB_FIPS A_S_FIPS FB_S_FIPS
Figure 6.24:Test combination percent matching with FIPS 140-2 results for the Frequency Add Narrow generator with 50% interference showing the combinations Longest Runs/Turning Point to Frequency Block/Serial.
100 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 100
00 15000
20000 30000
50000 100000 Sequence Length
PercentMatching
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial Test
Figure 6.25:Single test percent matching with FIPS 140-2 results for the Frequency Add Narrow gener-ator with 90% interference.
0 50 100 150 200 250 300 350 400 450 500 550
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
Countof"Pass"Sequences(max.500)
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial FIPS @ 20000
Figure 6.26:Single test “pass” count for the Frequency Add Narrow generator with 90% interference.
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 101 Example:
Sampled data: 100 101 110 111 110 101 100 011 010 001 000 001 010 011 Poker test data: 10 01 01 11 01 11 11 01 01 10 00 11 01 00 01 00 00 01 01 00 11
Sampled input sine wave:
Number of 1s: 21 Number of 0s: 21
Pattern Count
00 5
01 9
10 2
11 4
Figure 6.27:Binary analysis for a sine wave
breakdown of a sine wave. There it is apparent that the frequencies of the 01 and 00 patterns are not similar to what a random sequence would produce where all the dual bit patterns should occur approximately the same number of times.
The graphical results from the 90% interference generator combination tests are not pub-lished in this chapter, since none of the combinations show a significant increase in sensitivity to the FIPS standard. They are, however, included in the appendix if the reader is interested (see Figures 9.1 on page 131 and 9.2 on page 132).
Conclusion for the Frequency Addition (Single Frequency)
The interference from the external frequencies and, in particular, the addition of the single fre-quency is a challenge for the selected RNG tests to detect. Normal procedure is to include a spectral test where the error can readily be seen; however, as mentioned in the results discussion, this type of test is not possible to implement on a smart card processor at this time. Extreme interference, for example, a 90% sine wave addition, is detectable with the poker test starting at 2500 bits, but only at 10000 bits is it achieving the FIPS standard level of accuracy. The less extreme interference, with a sine wave addition of 50%, is significantly harder to detect. Perfect FIPS matching is only achieved with the frequency and poker test combination at 20000 bits.
102 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 50 100 150 200 250 300 350 400 450 500 550
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
Countof"Pass"Sequences(max.500)
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial FIPS @ 20000
Figure 6.28:Single test “pass” count for the Frequency Add Wide generator with 50% interference.
This indicates that the FIPS standard is basing its rejection on the combination of the frequency and poker tests. The other two tests in the FIPS group do either not catch the failure or overlap with thefirst two tests.
The recommendation for the addition of a single frequency detection is very hard to set, since it is up to the manufacturer to decide the sensitivity of the test unit. However, if designing for the worst case situation is required, the detection of both types of signal addition, then the test combination with the poker and frequency tests at 20000 bits is recommended. Reducing the test units detection sensitivity allows the testing sequence length to be reduced to 10000 bits with only the poker test. This reduction in quality still allows for the detection of extreme interference.
Results for the Frequency Addition (Wide Group Frequency)
The last sections investigated the detection quality or sensitivity of the RNG tests for single frequency type interference. The results studied in this section deal with interference that has a main component and some neighbouring falloff components. This type of interference is more likely to occur in a natural environment. The experimental results for the selected interference settings, see Section 6.2.6 for the parameters, can be seen in Figures 6.29 to 6.33. As with the single frequency study, multiple levels of interference have been tested, but only the 50% and 90% interference levels are analyzed in detail.
The analysis begins with the 50% levels by examining the results found in Figures 6.28 and 6.29. The addition of more frequencies into the random signal has the effect of making the
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 103
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
PercentMatching
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial Test
Figure 6.29:Single test percent matching with FIPS 140-2 results for the Frequency Add Wide generator with 50% interference.
error more detectable when compared to the single frequency interference. The three tests that show the greatest sensitivity to the FIPS standard are the poker, frequency, and the serial tests.
These three tests are the same tests that show high sensitivity for the single frequency interference (see Figure 6.22). However, the sensitivity of each is switched around for this experiment. In the single frequency trial the poker test is the least sensitive of the top three, but here it is the first to have 100% FIPS matching. For this experiment, the poker test requires only 10000 bits to achieve 100% FIPS matching, whereas the other two tests require at least 20000 bits. Looking at Figure 6.28 reveals that the FIPS standard rejects all the sample sequences. Therefore, as with the other experiments where the FIPS rejects all the sample sequences, the single tests are being measured for their ability to reject the generator. The results from the FIPS matching graph (Figure 6.29) are easily seen in thisfigure as well, since 100% matching is equal to zero tests passed.
After analyzing the the results from the single RNG tests, a closer look at the poker data reveals the type of data present. A sample sequence has been divided into bits of four and converted into decimal values. The examination of of one sample sequence reveals that the sequences produces zeros with greater probability than it does ones. In Table 6.4 it is visible that the subsequences with more ones than zeros (–) appear less often than the subsequences that have more zeros than ones (**). The reason for this phenomenon is the sine interference moves the data around the the zero mark. This can be seen if the data is reorganized as shown in the second part of Table 6.4). The data around the average(0000)occurs more often than the outer
104 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
Table 6.4:Sample data examining using poker test.
values (0111 and 1111). This shows the effect the sine wave interference has on the data, and the more sinusoidal interference there is the easier it is to detect the error.
The test combinations are again examined to checked if any improvement in quality is achieved.
The poker test is able to achieve 100% FIPS matching with a test sequence of 10000 bits. The question is is it possible to get 100% FIPS matching at 5000 bits or smaller? From Figures 6.30 and 6.31 it is evident that this is not possible. The frequency-poker and frequency-serial tests both show improvement over the single poker test; however, this improvement is only in the range of 3 to 4%. The test combinations do not achieve 100% matching at the 5000 test bit length.
As with the single frequency interference, this experiment has been tested at an extreme in-terference level (90% level). The results for the single tests and “pass” count have been included in Figures 6.32 and 6.33, with the combination test results included in Appendix 9 in Figures 9.3 and 9.4 on pages 132 and 133.
The results from Figure 6.33 show that the FIPS standard rejects all the sample sequences.
Therefore, the percent matching graph is the measure of how quickly the test rejects all the sequences, and the results from Figure 6.33 should matching closely with Figure 6.32.
The results in Figure 6.32 show a significant jump between the 500 and 1000 bit test lengths.
Initially, the two best tests are the serial and the poker test; however, the frequency test catches
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 105
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
PercentMatching
F_R_FIPS F_L_FIPS F_P_FIPS F_T_FIPS F_A_FIPS F_FB_FIPS F_S_FIPS R_L_FIPS R_P_FIPS R_T_FIPS R_A_FIPS R_FB_FIPS R_S_FIPS L_P_FIPS
Figure 6.30:Test combination percent matching with FIPS 140-2 results for the Frequency Add Wide generator with 50% interference showing the combinations Frequency/Runs to Longest Runs/Poker.
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
PercentMatching
L_T_FIPS L_A_FIPS L_FB_FIPS L_S_FIPS P_T_FIPS P_A_FIPS P_FB_FIPS P_S_FIPS T_A_FIPS T_FB_FIPS T_S_FIPS A_FB_FIPS A_S_FIPS FB_S_FIPS
Figure 6.31:Test combination percent matching with FIPS 140-2 results for the Frequency Add Wide generator with 50% interference showing the combinations Longest Runs/Turning Point to Frequency Block/Serial.
106 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
0 10 20 30 40 50 60 70 80 90 100
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
PercentMatching
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial Test
Figure 6.32:Single test percent matching with FIPS 140-2 results for the Frequency Add Wide generator with 90% interference.
0 50 100 150 200 250 300 350 400 450 500 550
25 50 75 100
250 500
1000 2500
5000 10000
15000 20000
30000 50000
100000 Sequence Length
Countof"Pass"Sequences(max.500)
Frequency Runs Longest Runs Poker Turning Point Autocorrelation Frequency Block Serial FIPS
Figure 6.33:Single test “pass” count for the Frequency Add Wide generator with 90% interference.
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 107 up, and all three tests achieve perfect matching with the FIPS standard at the 2500 bit length.
The runs test is not far behind, and it also reaches perfect FIPS matching, but with a test length of 5000 bits.
The extreme interference example is a lot easier to detect than the 50% interference RNG sample. The tests are able to achieve 100% FIPS matching at greatly reduced test lengths (2500 bits and 10000 bits). In the 50% interference experiment only the poker test at 10000 test bit length reaches perfect matching with FIPS; whereas, the rest require at least 20000 bits if they reach perfect FIPS matching at all. In this study the extreme interference is detectable by more tests and at lower test lengths; therefore, it stands to reason that the effects discussed in the 50%
interference case are only more pronounced allowing the tests to detect a more definite error.
Conclusion for the Frequency Addition (Wide Group Frequency)
This interference type is similar to the single frequency interference in the type of results shown;
the best test for detecting single frequency interference is also the best test to detect wide group interference. The effect the added frequencies has on the random sequence is to average the RNG data even more than is present in the single frequency experiments. This allows the poker, serial, and frequency tests to be used to detect this type of error instead of requiring the spectral test.
The recommended test for the wide group frequency interference is the poker test with a 10000 bit sample sequence. This test is able to catch both the 50% and 90% interference and reject the samples as not being good for cryptographic applications.
108 CHAPTER 6. EMPIRICAL TEST QUALITY MEASUREMENT
Implementation of the Frequency Addition (Pink Noise or 1f Noise)
The two frequency interference models discussed previously are for simple types of signal in-terference. Another type of frequency interference found in many unexpected sources is the 1f frequency interference (also called pink orflicker noise). This type of noise has the characteristic of having equal power per decade of frequency or a spectrum proportional to 1f. In comparison, white noise has the same distribution of power for all frequencies. One cause of this noise is the recombination effects at defects along the semiconductor’s border, material surface or in the volume itself. Figure 6.34 is an example of the frequency spectrum of 1f noise.
0 100 200 300 400 500 600 700 800 900 1000
−0.2
−0.15
−0.1
−0.05 0 0.05 0.1 0.15
0 100 200 300 400 500 600 700 800 900 1000
0 2 4 6 8 10 12
Time Frequency
Figure 6.34: Random 1f noise sample in the time and the frequency domain.
It is possible to empirically describe the 1f spectrum using the following formula [Sis02]:
C1 f = α
N ·1
f (6.1)
whereN is the total number of moving charges in a device. The variableα is a material charac-teristic called the Hooge-Parameter. Another form of Equation 6.1 is:
C1
f =KF·IAF fB
withKF,AF,Bthe model parameters. This form is more common in simulation programs.
There is another method for modeling 1f noise, filtering white noise with afilter that has an amplitude responseG(ω)proportional to √1ω. The filter should roll at -3 dB per octave in the frequency domain. One type of 1f filter proposed in [Whi99] uses afilter with poles and zeros set
6.2. RANDOM NUMBER GENERATOR FAILURE EXPERIMENTS 109 at:
Pole Zero 0.99516 0.98223
0.9438 0.83265
0.5559 0.107980 (6.2)
Another method is to use weighted sums offirst orderfilters. The followingfilter is found in [Whi99] on page 3:
b0 = 0.99886∗b0+white∗0.0555179;
b1 = 0.99332∗b1+white∗0.0750759;
b2 = 0.96900∗b2+white∗0.1538520;
b3 = 0.86650∗b3+white∗0.3104856;
b4 = 0.55000∗b4+white∗0.5329522;
b5 = −0.7616∗b5−white∗0.0168980;
pink = b0+b1+b2+b3+b4+b5+b6+white∗0.5362;
b6 = white∗0.115926; (6.3)
For the 1f noise generator used in the simulator, thefirstfilter has been implemented. It has been programmed in Matlab where thefilter function is employed on random numbers between -1 and 1. The filter data is then scaled by the maximum value of eachfilter (see Figure 6.35), after which the data is converted to a 20 bit fractional binary number. The binary number is stored the process is repeated until a 100000 bit sample sequence is created.
Results for the Frequency Addition (Pink Noise or 1f Noise)
The results from the 1f noise experiment are found in Figures 6.37 to 6.39. Thefirst step in the 1f noise analysis is checking to see how many sequences “passed” the FIPS standard. Figure 6.36 reveals that at 20000 bits the FIPS standard rejects all the sample sequences. As with the previous experiments that have the FIPS standard reject all the sequences, the sensitivity or quality is a measure of how quickly the individual test or test group rejects each of the samples.
The 1f noise interference has been described previously and can be looked at as low frequency interference. It is a more extreme frequency interference case than the previous two experiments.
It has been included in the experiment due to the common occurrence of this type of interference in normal usage.
The addition of even more frequencies than what is present in the wide group frequency experiment should show a result where the tests are able to detect the failure at a smaller test