Variation and Replication

Microarrays have been proposed as a technique to measure transcript abundances.

As a large-scale measurement technique, microarrays will almost certainly not be able to deliver exact quantifications of transcript abundances, but will be prone to some measurement error. Since its introduction, microarray technology has

under-gone many advances. These advances have been made in the different technological platforms, as well as in the available statistical data analysis methods, where some have been designed specifically for microarrays. “However”, as Shields (2006) puts it in his Editorial text – titled “MIAME², we have a problem” – in Trends in Genetics, “no amount of statistical or algorithmic knowledge can compensate for deficiencies in the technology itself”.

For this reason and for a well founded research, it is necessary to perform studies about the reliability and reproducibility of any method. Microarrays, in particular due to the complex laboratory work-flows involved, suffer from a variety of influ-ences, that may result in a multitude of deviations. This is particularly important, as many researchers prospect the use of microarrays for drug discovery or even as a diagnostic tool in clinical disease classification (see for example Wang et al., 2005;

van de Vijver, 2005; van de Vijver et al., 2002).

Unfortunately, the independent validation of microarray data is hampered by the lack of standards to compare its results with. For studies concerned with clinical prediction of malignancies, independent validation of their results should be possible in principle, because samples can be labeled by human experts and results from studies on a similar topic can be compared. The importance of the validation of studies has often been stressed, but “Validation is still an analysis and can be manipulated as can any analysis” (Ioannidis, 2005).

Ntzani and Ioannidis (2003) have carried out an evaluation study on previously published prediction studies of clinical diagnosis based on microarrays. Only a minority of these studies were found to comprise sufficient validation of the findings.

Michiels and coworkers performed a re-analysis of seven large studies predicting prognosis of cancer patients based on microarray data (Michiels et al., 2005). It was found, that the list of predictor genes was highly inconsistent between studies and that five out of seven studies did not perform better than random predictions.

In fact, the above mentioned methods make far-reaching assumptions about the data, in particular, that the expression profiles of the tissues contain information about cancer types or prognosis and that these changes in expression profiles are predominant over changes from other sources. This is not necessarily the case and is also hard to validate with the rather small number of cases selected for the classification studies. Hence, the variability of the resulting classifications might be caused by the statistically unsound foundation of the experiments and not by the technology.

In contrast to these rather frustrating findings, there are also more positive results when the measured quantitative value of mRNA is directly controlled by comparison with other methods, not indirectly by making strong (possibly unjustified) assump-tions. The Microarray Data Quality Control (MAQC) project is concerned with reproducibility and quality assessment of microarray data. Within this project, measurements from several microarray platforms have been compared with each other and with other measurement techniques such a quantitative RT-PCR. MAQC

2Minimum Information About a Microarray Experiment (MIAME), see Section 4.1.1

3.3. Properties Of Microarray Data 25 members have compared the quantitative measurements of RNA levels and have found that the correlation between different measurement techniques was generally high (Patterson et al., 2006; Consortium et al., 2006; Canales et al., 2006).

Despite these findings, microarray studies have been criticized for exhibiting vari-ation and beeing hard to reproduce. This immediately gives rise to the question, which reasons for variability can be identified. It can be partially answered by the fact that all measurement techniques, not only microarrays, are subject to a certain level of measurement error. Variation can be classified by its origin, and it is common to differentiate technical and biological variation.

This differentiation might seems a bit arbitrary when looking at data from a real experiment, because variation observed in an experiment cannot be directly assigned to its source. The usual strategy to assess the level of technical and biological variation is to perform the measurements repeatedly, yielding replicated measurements, so called replicates. It can be deduced from statistical inference theory that the higher the variability of the data, the more replicates are required to achieve a significant result. On the other hand this implies experimenters can respond to higher variability by simply adding more replications. In any case, to have a closer look on the sources of variation can be helpful to reduce them in further analysis steps.

Technical Variation Deviations in the technical process can have a large influ-ence on the results of microarray experiments. The extend of this influinflu-ences depend on the applied technology and microarray platform and can be observed in repli-cated experiments in the same laboratory and also between laboratories. Primary causes are variations in the applications of protocols. Further influences stem from the microarray production process such as variation of feature sizes and concentra-tions (Bammler et al., 2005).

Some studies have also shown the large impact of differential reporter sequences as a source of cross-platform variation. Other technical problems include scanner settings, as well as image segmentation and quantification (Repsilber and Ziegler, 2005; Yauk et al., 2004, 2005).

Failed PCR reactions, contamination of spotting solution with other DNA, or even false reporters from wrongly assembled microtiter plates can be sources of errors, which often cause constant deviations and are sometimes hard to detect.

In microarray experiments, technical variation can be assessed by using technical replicates. To achieve this, experimenters can use the same sample and perform the whole process of a microarray measurement. Material stemming from the same sample can then be hybridized to multiple microarrays of the same or of differ-ent platforms. Also, microarrays can carry replicate spots of the same reporter or different reporter sequences but for the same genes. Replicate spots can also be interpreted as technical replicates, but intra-array variation could underestimate global technical variability. A large number of technical replicates has been rec-ommended for quality control studies or evaluation studies of technologies (Allison et al., 2006).

In principle, technical variation can be reduced by conducting the experiment carefully, following standardized protocols rigidly and by improvements in the mea-surement techniques. Quantitative RT-PCR and other methods show less technical variation while lacking the high-throughput abilities of microarrays. These methods are often used to verify microarray data from few genes.

Biological Variation Assume that all kind of technical variation during measure-ment of a quantity could be reduced to zero, which is of course infeasible, there can still be variation in the measured quantities. This is in fact true for the process of gene transcription. Many authors have described the process of gene expression as a stochastic one; meaning that identical cells exposed to identical conditions will exhibit large fluctuations in gene expression. The effects of natural fluctuation in regulation is not interpreted as a general disadvantage, but seem to be beneficial or even necessary for the cell (see for example Rao et al. (2002); Blake et al. (2003);

Kaern et al. (2005)).

As a consequence, there can exist no ‘true’ measurement of transcript abundance, even under the most rigid experimental control. Further influences can be associ-ated with measurements of multiple cells. The amount of RNA extracted from a single cell is insufficient, so that cell cultures or tissue sample must be used. Not all cells in the sample are necessarily genetically identical, they can even belong to different cell lines or strains. Synchronization of cells with respect to their growth state is also an issue; cells of identical type can be in very different developmental stages. This is especially an issue when experiments are repeated with different organisms or patients (Baldi and Hatfield, 2002).

The level of biological variation is assessed by performing the experiment multiple times under the same conditions and by harvesting samples repeatedly. In general, the biological variation seems to be a bigger issue than the technical variation for assessment of significant results. Some publications recommend to prefer biological replicates over technical replicates in many cases (Allison et al., 2006). Biologi-cal replicates will contain influence from both techniBiologi-cal and biologiBiologi-cal variation, thereby serving to assess the overall variability of the experiment. If the number of replications exceeds the number of available microarrays, pooling is often used to generate a mixture of samples.

It is important to note that every sufficient measurement pipeline, not only those using microarrays, will exhibit biological variation. Any reasonable analysis should therefore contain biological replicates. To stress the effects of variation in further studies seems to be an important issue.