Permutations, Bootstraps and Jackknives

Zmapqtl allows for permutation tests and bootstrap or jackknife resamplings. The former is a way to determine experimentwise significance levels and comparisonwise probabilities [Churchill and DoergeChurchill and Doerge1994,Doerge and ChurchillDoerge and Churchill1996]. Phenotypes are shuffled against genotypes and the analyses are redone. For each test position, the comparisonwise probability or p-value is the proportion of permuted datasets that have test statistics less than the observed data set test statistic. It should correspond to the probability of the observed test statistic assuming a $\chi^2$ distribution with one degree of freedom. For the experimentwise significance level, the highest test statistic in each permutation is recorded, and these are ordered at the end of the permutations. The 90, 95, 97.5 and 99th percentile values are then the experimentwise significance levels at $\alpha = 0.1, 0.05, 0.025 {\rm and} 0.01$, respectively. Permutation tests are done for interval mapping within Zmapqtl, and interim results are stored in the files qtlcart.z3c and qtlcart.z3e. There are two distinct ways to perform the permutation test in QTL Cartographer. The first is simply to have Zmapqtl do the permuting and analysis: You would then use -r with the number of permutations to perform. If you choose to do the permutation test entirely within Zmapqtl, you must set the number permutations to a value larger than number of permutations already completed. In this way, if you started a permutation test and your machine crashed before the test was complete, you can restart Zmapqtl and finish it from where it left off.

An alternative way to do the permutation test is in a batch file. For composite interval mapping, one might want to reselect the background markers with SRmapqtl in each permutation. To this end, one would need to permute the traits, reselect the background markers and then run the composite interval mapping. The pseudo code example in Section 2.4.2 shows how to do this without the SRmapqtl step.

In the bootstrap, new datasets are created from the original by sampling with replacement. New datasets are the same size as the original. The statistics are redone and printed out. See the section Prune as to how to do bootstrapping.

Table 3.5: Examples of Interim Files for Model 6

Interim file	Created during	Contains
qtlcart.z6e	permutation test	Experimentwise state
qtlcart.z6c	permutation test	Comparisonwise state
qtlcart.z6a	bootstrap resampling	Iteration $i$ bootstrap
qtlcart.z6b	bootstrap resampling	Iteration $i+1$ bootstrap
qtlcart.z6i	jackknife resampling	Iteration $i$ jackknife
qtlcart.z6j	jackknife resampling	Iteration $i+1$ jackknife

Jackknife resampling is performed by calculating $n$ (the sample size) new estimates of the parameters: The $i$th estimate is calculated by deleting individual $i$ from the dataset. The standard deviation over these $n$ new estimates provides an estimate of the standard deviation for the test statistic and additive and dominance effects. You invoke the Jackknife by setting the number of bootstraps to 2. Zmapqtl uses two interim files to perform the jackknife. If you are using Model 6 in Zmapqtl and your filename stem is qtlcart, then these files will be called qtlcart.z6i and qtlcart.z6j. These files contain the sum and sum of squares up to the previous and current iteration, as Zmapqtl runs. Initially, the qtlcart.z6i file contains columns of zeros: This is the sum before any iterations are performed. Subsequently, qtlcart.z6j will contain the interim state after each odd-numbered iteration, while qtlcart.z6i will contain the state after each even-numbered iteration. If individual $i$ has no trait data, then the $i$th iteration will be skipped. For this reason, one cannot be sure that the file ending in ``j'' is the last iteration for odd sample sizes. It is best to look at both files at the conclusion of a jackknife experiment, and rename the interim file with the greater number of iterations to qtlcart.z6i. It this is done, then Eqtl will recognize it and calculate the means and sample standard deviations of the test statistic and effects.

To clarify the interim file names, we consider an example using Model 6 in Zmapqtl and the default filename stem ``qtlcart''. Table 3.5 lists the interim file names. Eqtl automatically looks for files named qtlcart.z6e, qtlcart.z6a and qtlcart.z6i. These files will be processed and the appropriate calculations done. Eqtl will overwrite the qtlcart.z6b and qtlcart.z6j files after completing its calculations, so if you want to save them, do so before running Eqtl. If you chose to use another model (say model 3), then the ``6'' in the filenames of Table 3.5 would be a ``3''.