Advanced Biostatistics: Logistic Regression & GLMM For this...

Advanced Biostatistics: Logistic Regression & GLMM

For this lab, we will use data from Vicente et al. (2005), who looked at the distribution of the first-stage larvae (L1) of the parasite Elaphostrongylus cervi in red deer across Spain. The researchers sampled individual deer from a number of farms, recorded the length and sex of each deer, and recorded whether or not E. cervi were present. Here, we will focus on the effect of length, sex, and farm identity on the probability that E. cervi are present. First, input the dataset DeerEcervi.txt and call it DeerEcervi. Create a column fSex that treats Sex as a factor and a column fFarm that treats Farm as a factor. Now, copy the column that contains numbers of E. cervi per deer and re-code as a binary variable (1 = E. cervi present; 0 = E. cervi absent) using the following code:
> DeerEcervi$Ecervi.01 <- DeerEcervi$Ecervi
> DeerEcervi$Ecervi.01[DeerEcervi$Ecervi>0]<-1
The second line of code assigns the number 1 to every place where E. cervi is > 0. Finally, it is useful to center the variable length by subtracting the mean length from each individual measurement of length. If we did not do this, then the intercept of our model would represent the probability that a deer of length 0 has the parasite. This is not very useful, since there cannot be a deer of length 0. By centering length, we get a model where the intercept represents the probability that a deer of average length has the parasite. Centering data can also be useful in avoiding numerical problems in parameter estimation. Center length using the following code:
> DeerEcervi$CLength <- DeerEcervi$Length - mean(DeerEcervi$Length)
Let’s start by using a GLM so that you can see what a simple multiple logistic regression model looks like. We can use the glm function and specify family = binomial to perform a logistic regression. Because we are sampling individual deer, glm will automatically use the Bernoulli distribution. If the data were proportions of infected deer among samples of multiple deer, glm would automatically apply the Binomial distribution. We can model presence of E. cervi as a function of deer length, sex, length * sex interaction, and individual farm. The drop1 function compares the deviance of the specified model with that of nested models (nested models are those in which one or more variables in the specified model are absent or, in other words, constrained to be 0). The difference between the deviances of a full and nested model is Chi-square distributed, and can be used to determine whether or not the full model is significantly better than the nested model. Because the main terms length and sex are required while the interaction term between them remains in the model, the drop1 function does not drop either of these main effects.
> DE.glm<-glm(Ecervi.01 ~ CLength * fSex + fFarm, data =
DeerEcervi,family = binomial)
> drop1(DE.glm, test = "Chi")
The first line of output shows the deviance for the model in which no term is dropped. Its AIC is 799.50. By dropping the nominal variable farm from the model, the deviance increases to 1011.72, a change of 258.22. The change in deviance for a binomial GLM is Logistic Regression & GLMM
Chi-square distributed with (in this case) 23 degrees of freedom. The change of 258.22 is associated with a P-value that is much smaller than 0.001, which means that the effect of farm is highly significant. The interaction term is also significant, yielding a P-value of
0.0016. Using the summary(DE.glm) command reveals that the glm function has estimated the coefficient for every single farm. Note: pay no attention to the P-values in the summary output; they result merely from comparing each estimate to the baseline.
Validation of logistic regression models is difficult because the residuals are not expected to be normally distributed. You can get some idea of what’s going on, and especially check for outliers, by using the plot command to obtain residual plots:
> par(mfrow=c(2,2))
> plot(DE.glm)
We obviously have an outlier at point 196, which corresponds to row 196 of the dataframe. If you take a look at row 196, it corresponds to a farm with only one observation (or possibly to a row where the farm was mislabeled). Either way, it is fine to omit this datapoint, since we do not have a large enough sample to say anything useful about this one farm. You can drop the offending datapoint and re-run the model and the residual plots by using these lines of code:
> DeerEcervi.fixed<-DeerEcervi[-196,]
> DEfixed.glm<-glm(Ecervi.01 ~ CLength * fSex+fFarm, data =
DeerEcervi.fixed,family = binomial)
> par(mfrow=c(2,2)) > plot(DEfixed.glm)
This is now looking much better. The odd shapes to the lefthand graphs are typical of binary data and very hard to interpret, though they do show outliers in length. Most importantly, there are no longer any datapoints with disturbingly large Cook’s distances. As with a Poisson regression, we can get some sense of how well the model fits by looking at the relationship between the null and residual deviances. First, the explained deviance (sometimes called the pseudo R2 value) is calculated as 100 x (null deviance – residual deviance)/null deviance = (1070.18-743.26)/ 1070 = 30.55. This means that the model explains 30.55% of the variation in E. cervi presence. Not fabulous, but enough to make the model useful. We can also test whether or not the model is better than a null model with just an intercept by using a Chi-squared test on the difference in deviance values:
> 1-pchisq(1070.18-743.26, 823-798)
This gives a P-value << 0.00001, indicating that our model is indeed much better than the null model. We can also take a look at the residual deviance by itself. The smaller the residual deviance, the better the model. Some statisticians recommend testing to see whether the residual deviance is significantly greater than expected by chance, using a Chi-square test of residual deviance with the corresponding degrees of freedom:
> 1-pchisq(743.11,797)
Logistic Regression & GLMM
The high P-value indicates that this model is a good fit, producing low residual deviance. This should give us some confidence in moving forward with a logistic regression.
(Note: because I discovered the outlier and point 196 only at the end of writing this lab, in the wee hours of the morning, we will proceed with the rest of the lab using the original dataset DeerECervi). To obtain insight into what the model is doing, is it extremely helpful to visualize the predicted values. Because we have multiple explanatory variables, this is not trivial. Let’s start with just data for sex = 1 (female). We can look at the males later to see how they differ. Now, we can plot the predicted probability of E. cervi presence as a function of length, with a separate line to denote each farm. First, plot the observed presence and absence values of the parasite vs. deer length:
> plot(DeerEcervi$CLength,DeerEcervi$Ecervi.01, xlab="Length",ylab="Probability of \presence of E. cervi",main="Male data")
Next, create a vector that contains the order of each deer in terms of its length, from the shortest to the longest deer, and call it I. In other words, if the first row in DeerECervi contains data from the 516th shortest deer, the first element of I will be 516. If the last row of DeerECervi contains data from the 2nd shortest deer, the last element of I will be 2. This step will set you up to create a smooth plot when you use the “lines” command to connect each datapoint. (If you do not do something to order your data, you will wind up with a “spaghetti plot” of lines going in every direction). Use the order command to create I:
> I<-order(DeerEcervi$CLength)
Next, create a vector that contains all unique values in the column “Farm” (i.e., a vector that contains all the farm names). Also, set the variable x equal to the number of different farms:
> AllFarms<-unique(DeerEcervi$Farm)
> x<-length(AllFarms)
In the first line of the next chunk of code, you will use the “for” command to loop through all the different farms. This means that R will first set the variable j = 1 as it executes the next 3 lines of code. The second line of code pulls out just the data for females (fSEX=1) and the first farm (fFarm = AllFarms[1]). The third line of code uses the model parameters from DE.glm to predict the probability of presence of E. cervi for each deer in farm 1 based on the deer’s length. The fourth line of code plots the probabilities for each farm by drawing a line between the predictions from the previous line of code. Next, R will set the variable j = 2 and repeat the next 3 lines of code, resulting in a plot of the predicted values for the second population. It will keep looping through all 24 populations, until the predicted values from each one are on the plot. FYI: “for loops” are a standard and extremely useful construct in computer programming, no matter the language.
Logistic Regression & GLMM
> for (j in seq(1,x)){ mydata<-data.frame(CLength=DeerEcervi$CLength,fSex="1", fFarm=AllFarms[j])
P.DE2<-predict(DE.glm,mydata, type="response")   lines(mydata$CLength[I],P.DE2[I])}
So how does the predict function get us from the output of glm to a plot of probabilities of E. cervi presence? Go back to the results of summary(DE.glm). Here, you see the intercept and slope of the lines that describe the relationship between length and logit(presence) for each farm. Sex 1 (female) and farm AL are used as a baseline. Therefore, the log odds ratio associated with a female deer from farm AL having the parasite is given by logit(pi) = -1.796 + 0.04062 * Lengthi
the log odds ratio associated with a female deer from farm AU having the parasite is given by logit(pi) = (-1.796 + 3.340) + (0.04062 * Lengthi)
and the log odds ratio associated with a male deer from farm AL having the parasite is given by logit(pi) = (-1.796 + 0.6280) + (0.04062 + 0.03618) * Lengthi
To get the probability that a female from farm AL has the parasite, the predict function uses the formula
(pi) = (e-1.796 + 0.04062 * Lengthi)/(1+ e-1.796 + 0.04062 * Lengthi)
This formula is the “link” between the mean Yi and the systematic part of the generalized linear model. The logit link is the default link for binomial regression. However, other links are possible, including the probit link and clog-log link, which each produce a slightly different shape in the fitted line. The logit and probit links assume that you have approximately equal numbers of zeros and ones. The clog-log link may be an option if you have many more zeros than ones, or vice-versa, because the sigmoidal curve is asymmetrical. To use a different link, you just need to modify the code in the family option inside the glm command to family = binomial(link = “probit”), for example.
Open a word document. Create a plot like the one you created above, but for male deer. Paste this plot into your word document. In approximately 1 sentence, describe the differences between female and male deer in terms of their probability for hosting E. cervi.
Logistic Regression & GLMM
So far so good, except that estimating separate intercepts for the variable Farm consumes 23 degrees of freedom, and we are not even interested in this effect—we just want to know how length and sex are generally related to probability of parasite presence! We can’t drop this variable, however, since it is clearly highly significant and dropping it would lead to pseudoreplication by ignoring the fact that we do not have a random sample of all deer—we have a collection of samples from particular farms. In addition, this GLM model only allows us to make predictions for particular farms, when we would really rather make a general prediction that could be applied to any farm. As you have already guessed (I hope!) it would be better to treat farm as a random effect. This will require using a generalized linear mixed-model—a technique on the frontier of statistical research. The model now becomes
Yij ~ Binomial(1,pij) logit(pij) = α + β1 * Length + β2 * Sexij + β3 * Length * Sexij + ai ai ~ N(0,σ2a)
Yij is 1 if animal j on farm i has E. cervi and 0 otherwise. The random intercept ai is assumed to be normally distributed with mean 0 and variance σ2a. If this variance is small, then all farms will tend to have a similar logistic curve. If this variance is large, then different farms will tend to have very different intercepts. Regardless, we now have to estimate only one variance parameter ai instead of 23 different intercepts. In addition, we will be able to make general predictions that apply to all farms, including farms that were not sampled in this study.
There are several functions that can be used for generalized linear mixed models, including glmmPQL from the MASS package, lmer from the lme4 package, and glmmML from the glmmML package. They all give similar estimates and standard errors for regression parameters. However, details of the output differ. We will use lmer because it provides an AIC, BIC, log likelihood values, and a deviance, making model comparison easier. Download, install, and load the lme4 package. Now specify the appropriate mixed-model using:
> library(lme4)
> DE1.lme4<-lmer(Ecervi.01 ~ CLength * fSex +(1|fFarm), family = binomial, data = DeerEcervi)
The probability of presence of the parasite is modeled as a function of length, sex, and their interaction. The random effect of farm is adding a random term to the intercept.   
Take a look at the results of the summary command:
> summary(DE1.lme4)
The random intercept ai has a variance of 2.3929 and a standard deviation of 1.5469, representing the variation among different farms. In your word document, provide an equation for the probability of parasite presence on a female deer (Sex = 1) with length = Lengthi, based on the output of the summary command. Also provide an equation for the probability of parasite presence on a male deer (Sex = 2) with length = Lengthi, based on the output of the summary command.
Logistic Regression & GLMM
The random intercept ai is assumed to be normally distributed with mean 0 and standard deviation 1.5469. This means that the majority of the values (95% to be more exact) of ai are between -1.96 * 1.5469 and 1.96 * 1.5469. Let’s create a figure that shows the probability of a female deer from any farm having E. cervi as a function of centered length, with 95% confidence intervals around that prediction. Unfortunately, there is no method for the handy function “predict” for an object of class mer, such as the output from lmer, so we will have to do some manual calculations. First, we will create a variable g that represents logit(pi) for a female deer of length i. Then, we define
p.averageFarm (the probability of E. cervi presence on a female deer from an average farm) according to the logit link function:
> g <- 0.939676 + 0.038978 * DeerEcervi$CLength
> p.averageFarm<-exp(g)/(1+exp(g))
Then, we plot the actual presence/absence of E. cervi on each deer vs. the centered length of the deer:
> plot(DeerEcervi$CLength,DeerEcervi$Ecervi.01,xlab="Length", ylab="Probability of > presence of E. cervi")
Next, we get the order of deer length, in order to avoid a spaghetti plot, and add a dark line that connects the predicted probability of presence in each deer, based on that deer’s length:
> I1<-order(DeerEcervi$CLength
> lines(DeerEcervi$CLength[I1],p.averageFarm[I],lwd=3)
Finally, we calculate the upper and lower 95% confidence bands and plot lighter lines representing these bands:
> p.Upp<-exp(g+1.96*1.5469)/(1+exp(g+1.96*1.5469))
> p.Low<-exp(g-1.96*1.5469)/(1+exp(g-1.96*1.5469)) > lines(DeerEcervi$CLength[I1],p.Upp[I1])
> lines(DeerEcervi$CLength[I1],p.Low[I1])
In this plot, the thick line now shows predicted probabilities for females on a typical farm. (Typical in this case means that ai = 0). The light lines are created by adding or subtracting 1.96 x 1.462 to the predictor function. Hence, 95% of farms have logistic curves between these two extremes. According to this graph, if you go to a typical farm and sample a female deer of average length (Length = 0; note that average length is the average of all deer, including males and females), it will have a probability of approximately 0.7 of having the parasite (at 0 on the x-axis, the bold line hits ~0.7 on the y-axis). However, depending on which farm you visit, for 95% of farms this probability can be anything between 0.1 and 0.9! So, there is considerable between-farm variation.
Now, create a similar plot for average probabilities of infection, ± 95% confidence intervals, but for male deer. Paste this plot into your word document. In Logistic Regression & GLMM
~ 1 sentence, describe the differences between female and male deer in terms of their average probability for hosting E. cervi.
Finally, you would probably like to know which of your main effects or interaction terms are significantly different from 0. The summary command provides Pvalues based on Z-statistics. The Z-statistics are based on the parameter estimates, which are calculated by taking each parameter estimate and dividing it by its standard error, just as in a simple Z-test (a close relative of the one-sample t-test). Unlike a Z-test or a onesample t-test, the standard errors of each parameter are obtained from the curvature of the likelihood surface around that parameter estimate. Using the Z-statistic is also known as a Wald Z-test. The Wald Z-test is generally not as reliable as a likelihood ratio test (LRT) for a GLMM, so it would be better to perform a LRT. To do this for the interaction term, for example, fit a full model with all terms and a nested model that is missing the interaction term. Then, compare the two using the anova command. This will perform a likelihood ratio test:
> DE1.lme4<-lmer(Ecervi.01 ~ CLength * fSex +(1|fFarm), family = binomial, data = DeerEcervi)
> DE2.lme4<-lmer(Ecervi.01 ~ CLength + fSex +(1|fFarm), family = binomial, data = DeerEcervi)
> anova(DE1.lme4,DE2.lme4)
According to the output, the interaction between sex and length is significant (Χ21 = 10.225, P = 0.001385). The results from the Wald Z-test and the LRT do not differ much in this case, but they can and do in other cases. The two tests are asymptotically equivalent as the sample size increases.
Note that plot has no method for objects of class mer.

Given Data:
Farm Sex Length Ecervi
AL 2 164 0
AL 1 216 0
AL 1 208 0
AL 1 206 14.37
AL 1 204 0
AL 1 200 0
AL 1 199 0.9
AL 1 197 0.8
AL 1 196 1.21
AL 1 196 0
AL 1 192 2.38
AL 1 191 0
AL 1 190 0
AL 1 180 0
AL 1 178 53.6
AU 1 195 36.0654
AU 1 190 3.9888
AU 1 188 24.93
AU 1 185 286.695
AU 1 185 0.6648
AU 1 184 104.1
AU 1 184 22
AU 1 184 5.4846
AU 1 180 49.86
AU 1 180 0
AU 1 178 143.14
AU 1 176 24.15
AU 1 175 50.9
AU 1 175 20
AU 1 174 77.6154
AU 1 174 9.7
AU 1 173 27.7554
AU 1 173 2.5
AU 1 172 5.4846
AU 1 170 27.62
AU 1 170 24.93
AU 1 170 0
AU 1 169 177.3354
AU 1 169 131.4
AU 1 169 3.6564
AU 1 167 5.4846
AU 1 161 237.8
AU 1 160 20.7
AU 1 158 3.6564
AU 1 155 0
AU 1 153 24.55
AU 1 153 0
BA 2 168.5 7.91
BA 2 168 11.634
BA 2 168 2.47
BA 2 165.2 0
BA 2 164.3 1.85
BA 2 164 4.21
BA 2 161 6
BA 2 161 2.8254
BA 2 159 15.58
BA 2 156 116.34
BA 2 156 0
BA 2 154 14.9
BA 2 153.5 19
BA 2 150 0
BA 2 148 38.226
BA 2 148 0
BA 2 145 28.5864
BA 2 145 0
BA 2 143 7.9776
BA 2 140 1.21
BA 2 139 66.48
BA 2 138 1.99
BA 2 138 0.3324
BA 2 137 477.36
BA 2 137 0
BA 2 129 58
BA 2 125 0
BA 2 122 0
BA 1 202 84.9
BA 1 187 252.8
BA 1 187 205.8
BA 1 187 51.9
BA 1 186 89.5
BA 1 185 199.6
BA 1 183 6.2
BA 1 182 285.3
BA 1 181 112.24
BA 1 180 147.8
BA 1 178 168.7
BA 1 178 114.4
BA 1 177 140
BA 1 176 266.9
BA 1 175 475.7
BA 1 173 126.1
BA 1 170 1357.8
BA 1 170 5.4846
BA 1 169 29.2
BA 1 159 40.1
BA 1 159 15.9
BA 1 132 0
BE 2 122 14.08
BE 1 180 41.51
BE 1 175 519.6
BE 1 170.5 1.15
BE 1 166.6 39.75
BE 1 165 17.99
BE 1 162 7.41
BE 1 160 56.9
BE 1 158 23.7
BE 1 158 23.54
BE 1 156 19.62
BE 1 154 117.14
BE 1 145 30.1
CB 2 174.8 109.86
CB 2 174 47.98
CB 2 171 82.56
CB 2 169 12.07
CB 2 168 145
CB 2 168 34.73
CB 2 167 88.3
CB 2 165 65.57
CB 2 164 70.009
CB 2 164 56.85
CB 2 164 20.85
CB 2 163 26.06
CB 2 162 427.3
CB 2 162 68.26
CB 2 162 3.33
CB 2 162 2.4
CB 2 162 0
CB 2 161 9.49
CB 2 160 67.59
CB 2 160 34.78
CB 2 160 0
CB 2 159 1.9
CB 2 157 1.9
CB 2 156.5 35.77
CB 2 155.2 288.05
CB 2 155 0
CB 2 154.5 133.87
CB 2 154 47.01
CB 2 153.5 47.84
CB 2 153.5 21.9
CB 2 153 335.35
CB 2 153 40.7
CB 2 153 3.25
CB 2 152.8 15.04
CB 2 149 18.51
CB 2 147 221.68
CB 2 146 126.61
CB 2 143 354.4
CB 2 143 88.96
CB 2 142 0
CB 2 139.1 402.14
CB 2 139 0
CB 2 139 0
CB 2 138.2 4.7
CB 2 136 0
CB 2 135.5 0
CB 2 133.5 0
CB 2 132 0
CB 2 131.5 0
CB 2 131 0
CB 2 128.5 326.13
CB 2 127.9 0
CB 2 126 0
CB 2 120.5 0
CB 2 116 0
CB 2 114.5 0
CB 2 111.9 0
CB 2 110.3 0
CB 1 197 62.11
CB 1 190 16.67
CB 1 189 27.09
CB 1 187 20.4
CB 1 183 25.24
CB 1 179 0
CB 1 177 12.52
CB 1 170 0
CB 1 169.8 16.22
CB 1 169.5 178.42
CB 1 165 0
CB 1 163.5 86.31
CB 1 160 17.6
CB 1 158 135.88
CB 1 157 35.35
CB 1 156.5 0
CB 1 156 0
CB 1 154 2.199
CB 1 151.5 178.02
CB 1 151 98.39
CB 1 150 0
CB 1 148 180.45
CB 1 148 1.29
CB 1 147 134.05
CB 1 137.5 71.65
CB 1 130.5 54.09
CB 1 121.5 0
CRC 1 116.5 0
HB 2 174 3.33
HB 2 153 0
HB 2 147 0
HB 1 172 0
HB 1 170 0
HB 1 165 0
HB 1 163 0
HB 1 163 0
HB 1 162 1.15
HB 1 159 0
HB 1 159 0
HB 1 153 0
HB 1 152 0
HB 1 149.8 0
HB 1 145 0
HB 1 143 0
HB 1 142 0
LN 2 177 73.05
LN 2 176 86.37
LN 2 165 97.97
LN 2 165 22.187
LN 1 199 125.89
LN 1 198 141.21
LN 1 191 79.81
LN 1 190 125.35
LN 1 190 20.4
LN 1 189 284.31
LN 1 189 89.64
LN 1 184 831.14
LN 1 184 127.33
LN 1 182 37.77
LN 1 181 210.1
LN 1 181 207.68
LN 1 181 163.34
LN 1 179 129.71
LN 1 178 112.1
LN 1 178 2.3
LN 1 177 41.6
LN 1 176 97.29
LN 1 175 71.19
LN 1 174 160.75
LN 1 174 37.03
LN 1 172 96.25
LN 1 170 161.48
LN 1 166 23.3
LN 1 163 0
LN 1 160 4.66
LN 1 145 33.31
LN 1 118 0
LN 1 103 0
MAN 2 163 0
MAN 2 159 0
MAN 2 151 8.81
MAN 2 146 69.75
MAN 2 146 0
MAN 2 143 5.9
MAN 2 129 0
MAN 2 115 0
MAN 1 199 35.8
MAN 1 199 0
MAN 1 190 0
MAN 1 188 66.37
MAN 1 187 0
MAN 1 187 0
MAN 1 182 0
MAN 1 181 2.5
MAN 1 180 0
MAN 1 178 0
MAN 1 176 0
MAN 1 174 0
MAN 1 174 0
MAN 1 173 17.33
MAN 1 172 24.61
MAN 1 171 2
MAN 1 170 0
MAN 1 168 10
MAN 1 128 1.6
MB 2 169 12.13
MB 2 168.5 17.47
MB 2 168 13.72
MB 2 166.5 47.12
MB 2 164.2 3.5
MB 2 159.7 8
MB 2 157 6.09
MB 2 156.5 48
MB 2 155.5 15.73
MB 2 154 7.19
MB 2 153 10.76
MB 2 152.2 30.2
MB 2 150 29.3
MB 2 148 35.21
MB 2 146 57.8
MB 2 142.2 0
MB 2 115 53.5
MB 1 190 3.15
MB 1 184 0
MB 1 182 0
MB 1 181.9 3.85
MB 1 181.5 7.71
MB 1 180.7 27.35
MB 1 180.3 52.1
MB 1 179.4 2.2
MB 1 179 31.35
MB 1 179 0
MB 1 176 5.3
MB 1 172.5 24.42
MB 1 172 22.43
MB 1 172 3.8
MB 1 171 1.17
MB 1 168 31.32
MB 1 153 61.9
MO 2 191 2.43
MO 2 181 4.48
MO 2 178 10.92
MO 2 178 1.13
MO 2 174 0
MO 2 173 0
MO 2 173 0
MO 2 173 0
MO 2 171 12.5
MO 2 171 4.024
MO 2 171 0
MO 2 171 0
MO 2 170.5 3.98
MO 2 170 0
MO 2 170 0
MO 2 170 0
MO 2 169 1.18
MO 2 169 0
MO 2 168 22.2
MO 2 168 15.8
MO 2 168 7.49
MO 2 168 0
MO 2 168 0
MO 2 167 67.83
MO 2 167 14.42
MO 2 167 1.98
MO 2 167 0
MO 2 167 0
MO 2 166.5 0
MO 2 166 10.34
MO 2 166 0
MO 2 166 0
MO 2 165 66.94
MO 2 165 38.8
MO 2 165 14.17
MO 2 165 1
MO 2 165 0
MO 2 165 0
MO 2 164 8.22
MO 2 164 0
MO 2 163 16.5
MO 2 163 14.91
MO 2 163 3.06
MO 2 163 0
MO 2 162 28.2
MO 2 162 0
MO 2 162 0
MO 2 162 0
MO 2 162 0
MO 2 162 0
MO 2 161 12.95
MO 2 161 0
MO 2 160.5 0.92
MO 2 160 0
MO 2 159.5 0
MO 2 159 72.54
MO 2 159 10.47
MO 2 159 9.47
MO 2 159 8
MO 2 159 3.59
MO 2 159 0
MO 2 158.5 4.4
MO 2 158.5 0
MO 2 158 23.16
MO 2 158 5.87
MO 2 158 0
MO 2 158 0
MO 2 158 0
MO 2 158 0
MO 2 158 0
MO 2 158 0
MO 2 157.2 30.78
MO 2 157 33.24
MO 2 157 24.5
MO 2 157 0
MO 2 156.5 7.9
MO 2 156 0
MO 2 155 0
MO 2 155 0
MO 2 153 16.24
MO 2 153 0
MO 2 153 0
MO 2 152 350.1
MO 2 152 20
MO 2 152 3.36
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 152 0
MO 2 151 0
MO 2 151 0
MO 2 150.5 0
MO 2 150 43.3
MO 2 150 0
MO 2 150 0
MO 2 149 0
MO 2 148 13.6
MO 2 148 0
MO 2 148 0
MO 2 147 15.04
MO 2 147 0
MO 2 147 0
MO 2 145 0
MO 2 143 0
MO 2 143 0
MO 2 141.5 0
MO 2 139 0
MO 2 138 0
MO 2 136 0
MO 2 135 0
MO 2 135 0
MO 2 132 0
MO 2 132 0
MO 2 131 8.63
MO 2 131 0
MO 2 127 0
MO 2 125 0
MO 2 124 0
MO 2 121 0
MO 2 116 0
MO 2 91.6 0
MO 2 85 0
MO 1 214 0
MO 1 209 0
MO 1 208 107.6
MO 1 205 17.7
MO 1 202 4.52
MO 1 202 0
MO 1 201 1.13
MO 1 200 44.59
MO 1 199 17.51
MO 1 198 7.28
MO 1 198 7.1
MO 1 197 91.44
MO 1 197 42.4
MO 1 197 37.23
MO 1 197 6.1
MO 1 197 0
MO 1 196 7.66
MO 1 196 1.09
MO 1 195 0
MO 1 194 1.7
MO 1 191 0
MO 1 188.5 28.19
MO 1 188 0
MO 1 187 26.37
MO 1 187 21.89
MO 1 187 0
MO 1 186 12.33
MO 1 186 1.97
MO 1 185 0
MO 1 184 0
MO 1 183 0
MO 1 182 17.3
MO 1 182 5.82
MO 1 182 0
MO 1 179 1.21
MO 1 178 5.92
MO 1 177 0
MO 1 176 2.15
MO 1 174 16.12
MO 1 174 1.19
MO 1 174 0
MO 1 174 0
MO 1 172 143
MO 1 171 0
MO 1 171 0
MO 1 170 16.5
MO 1 170 0
MO 1 170 0
MO 1 168 14
MO 1 167 0
MO 1 164 0
MO 1 164 0
MO 1 163 0
MO 1 162 15.92
MO 1 160 0
MO 1 158 130.4
MO 1 158 24.52
MO 1 158 0
MO 1 155 0
MO 1 155 0
MO 1 152 0
MO 1 152 0
MO 1 149 0
MO 1 148 7.35
MO 1 148 0
MO 1 147 0
MO 1 145 0
MO 1 144 0
MO 1 142.3 0
MO 1 142 1.14
MO 1 142 0
MO 1 139 0
MO 1 138 0
MO 1 137 0
MO 1 136 0
MO 1 136 0
MO 1 136 0
MO 1 132 2.17
MO 1 132 0
MO 1 130 0
MO 1 126 0
MO 1 126 0
MO 1 75 0
NC 2 166 3.324
NC 2 164 1.1634
NC 2 159 0
NC 2 158 2.22708
NC 2 153 0
NC 2 153 0
NC 2 150 0
NC 2 143 19.944
NC 2 143 0
NC 1 190.5 3.56
NC 1 187.5 0
NC 1 182 0
NC 1 178.6 8
NC 1 177 0
NC 1 177 0
NC 1 175 0
NC 1 174 0
NC 1 173 1.24
NC 1 172 32.53
NC 1 172 0
NC 1 171 1.23
NC 1 170 10.17
NC 1 169.4 5.89
NC 1 168.2 0
NC 1 168 0
NC 1 163 7.25
NC 1 160 3.37
NV 2 169 19.44
NV 2 166 9.72
NV 2 165 3.24
NV 2 164 68.04
NV 2 162.5 0
NV 2 159 6.48
NV 2 158 0
NV 2 154 0
NV 2 153 6.48
NV 2 152 0
NV 2 151 0
NV 2 150.5 0
NV 2 149 35.64
NV 2 148.7 32.4
NV 2 148 3.24
NV 2 147 0
NV 2 147 0
NV 2 138 6.48
NV 2 137.8 2.43
NV 2 132 0
PN 2 173.5 3.53
PN 2 168.1 7.48
PN 2 165.5 7.04
PN 2 165 1.1
PN 2 163 12.56
PN 2 161 37.7
PN 2 160 15
PN 2 155 3.41
PN 2 152.8 75.04
PN 2 152 13.55
PN 2 149.8 19.88
PN 2 149.6 7.75
PN 2 149 11.59
PN 2 148 14.94
PN 2 146.7 0
PN 2 130 0
PN 2 126 0
PN 1 184 32.9
PN 1 184 23
PN 1 183 44.7
PN 1 179 39.5
PN 1 178 5.7
PN 1 178 1
PN 1 177 21.8
PN 1 176 51.8
PN 1 176 8.5
PN 1 175 40.5
PN 1 175 7.2
PN 1 173 190.43
PN 1 172 86.3
PN 1 172 32.8
PN 1 171 145
PN 1 169 25.8
PN 1 166 28.2
PN 1 166 13.55
PN 1 166 6.7
PN 1 162 9.8
QM 2 168 39.93
QM 2 165 22.23
QM 2 165 0
QM 2 162 105.45
QM 2 162 28.12
QM 2 161 12.31
QM 2 160 0
QM 2 158.6 1.7
QM 2 157.5 94.2
QM 2 157 27.75
QM 2 157 22.04
QM 2 157 17.49
QM 2 155 15.54
QM 2 154 73.73
QM 2 154 32.54
QM 2 153 68.68
QM 2 152 108.42
QM 2 152 67.47
QM 2 151 8.83
QM 2 149.7 17.78
QM 2 149.5 89.74
QM 2 144 0
QM 2 143 678.13
QM 2 137 29.64
QM 2 136 0
QM 2 135 0
QM 2 131 0
QM 2 129 169.88
QM 2 117 170.2
QM 2 115 0
QM 2 114 13.44
QM 2 108 2.4
QM 2 106 0
QM 1 179 123.7
QM 1 177 43.7
QM 1 175.5 0
QM 1 175 60.39
QM 1 173.5 33.33
QM 1 168.5 179
QM 1 162 6.023
QM 1 161.5 209.11
QM 1 160 466.5
QM 1 160 124.44
QM 1 160 67.8
QM 1 157 5.8
QM 1 157 0
QM 1 155.5 51.33
QM 1 152.5 53.61
QM 1 148 192.34
QM 1 148 24.38
QM 1 145 0
QM 1 144.5 72.51
QM 1 144.2 645.23
QM 1 143 1055.7
QM 1 142.1 26.98
QM 1 138 92.37
QM 1 130 263
QM 1 120 6.35
QM 1 120 5
QM 1 112 0
RF 2 163 6.7
RF 2 163 4.79
RF 2 162.4 13.51
RF 2 157 5.93
RF 2 150 91.7
RF 2 141 32.9
RF 2 118 0
RF 2 114 0
RF 1 181 29.916
RF 1 180 13.296
RF 1 177 1442
RF 1 177 141.7686
RF 1 174 113.016
RF 1 172 160.7154
RF 1 170 8.8086
RF 1 168.5 109.2
RF 1 160 98.07
RF 1 160 2.33
RF 1 155 255.948
RF 1 155 12.1326
RN 2 164.3 0
RN 2 163 11.85
RN 2 151 0
RN 2 150 0
RN 2 137 0
RN 2 128 0
RN 2 122 0
RN 1 191 0
RN 1 189 11.9
RN 1 178 0
RN 1 176 26.9
RN 1 174 0
RN 1 174 0
RN 1 173 0.8
RN 1 172 0
RN 1 169 1.2
RN 1 169 0
RN 1 166 0
RN 1 160 32.9
RN 1 159 40.4
RN 1 143 0
RN 1 134 0
RN 1 131 0
RO 2 170 0
RO 2 157.5 2.5
RO 2 156 2.5
RO 2 151 8.75
RO 2 144.9 15
RO 2 144 39
RO 2 137.9 6.25
RO 2 131 25
RO 1 186 12.1
RO 1 182 71.1
RO 1 179 160.6
RO 1 178 556.3
RO 1 177 434.02
RO 1 177 45
RO 1 175 11.6
RO 1 174 221.3
RO 1 174 114.8
RO 1 172.5 8.75
RO 1 172 13.7
RO 1 171 29.4
RO 1 169 63.7
RO 1 168 110.4
RO 1 163 2186.6
RO 1 163 242.5
RO 1 162 108.1
RO 1 161 35
RO 1 160 10
RO 1 159 8.75
RO 1 158 77.7
RO 1 126.4 0
SAU 1 185.5 0
SAU 1 182 0
SAU 1 166 0
SE 2 175 260.2
SE 2 172 47.94
SE 2 167 96.53
SE 2 163.5 121.41
SE 2 162.5 84.11
SE 2 161 40.669
SE 2 159 0
SE 2 157.5 19.66
SE 2 151.5 152.67
SE 2 146.5 48.94
SE 2 105.5 0
SE 1 194 110.08
SE 1 191 0
SE 1 190 40
SE 1 189 48.84
SE 1 188 57.48
SE 1 181 0.837
SE 1 180 2.2
SE 1 179 11.86
SE 1 178 39.55
SE 1 177 19.07
SE 1 176 0
SE 1 174 81.31
SE 1 174 6.77
SE 1 121.5 0
SE 1 117 0
TI 1 200 7
TI 1 192 3.6
TI 1 188 4.5
TI 1 187 65.1
TI 1 185 22.7
TI 1 184 14.5
TI 1 184 14
TI 1 184 10.1
TI 1 183 0
TI 1 182 49.2
TI 1 182 17.5
TI 1 179 12.3
TI 1 179 8
TI 1 179 6.8
TI 1 178 22.9
TI 1 175 31.3
TI 1 175 17.6
TI 1 172 6.5
TI 1 170 0.8
TN 2 165 1.76
TN 2 163 4.62
TN 2 161 5.96
TN 2 160 86.2326
TN 2 160 12.96
TN 2 157 8.1
TN 2 151 16.2
TN 2 149 53.9946
TN 2 149 0
TN 2 148 109.1718
TN 2 148 9.72
TN 2 146.5 38.88
TN 2 146 16.2
TN 2 138 29.16
TN 2 135 0
TN 2 124 0
TN 1 179 84.24
TN 1 176 3.39
TN 1 173 0
TN 1 158 1.91
TN 1 155 87.48
TN 1 154.5 0
TN 1 145 0
TN 1 144.9 84.24
TN 1 125 0
VISO 1 174.5 250.9
VISO 1 167 1.4
VISO 1 167 0
VISO 1 166.5 495
VISO 1 166.1 18.6
VISO 1 165 96.6
VISO 1 164.7 24.1
VISO 1 161 58.3
VISO 1 160 38.6
VISO 1 158 0
VISO 1 157 4.9
VISO 1 149 7.99
VISO 1 114.3 21.7
VY 2 165 36.46
VY 2 151 11.28
VY 2 146 9.57
VY 1 166 364
VY 1 148 0
VY 1 143 20.12
VY 1 140 6.24