Continued misrepresentation of the 1994 RCT of Precision Tinted Lenses

'The greatest obstacle to discovery is not ignorance, it is the illusion of knowledge'

               Daniel Boorstin (also attributed to Steven Hawking)

This article appeared in Optometry in Practice (Vol 17 Issue 2 103-112) which is the College of Optometrists Quarterly Journal for Continuing Professional Development. For this reason, you might think it reflects mainstream knowledge and practice. You can download the article from Professor Arnold Wilkins personal web page at the University of Essex - reference number 228.

And apparently, the article above is not a review.....

The authors state that it is not a review even though it does review the literature - albeit in a partial manner. Although this may not be a review in the formal sense of the word,  the misrepresentation of research studies (see below)  and selective citation that occurs in this article can not be excused.  For example, the reader is referred to other reviews which the authors have written themselves but no mention is made of the more numerous reviews, written by independent scientists with no financial interest in coloured lenses and overlays (see Review of Reviews), that are uniformly critical of evidence for the treatment of visual stress. This a good example of citation distortion funnelling readers through reviews that support their argument while ignoring the remainder.

The 1994 RCT of Precision Tinted Lenses is discussed in detail in the blog post of  February 2015. In some respects, it was a pioneering study but it was so hampered by losses to follow up that no meaningful conclusions can be drawn from the data. 68 reading impaired children were enrolled. All participants had used overlays for at least 3 weeks and were subsequently assessed with the intuitive colorimeter. An optimum tint that minimised perceptual distortions was determined and prescribed in the form of glasses. A placebo - closely related tint - was also prescribed. Because subjects did not see the actual glasses they were going to receive and one month was left between colorimeter assessment and the start of the study good masking was achieved - 26 subjects did not know which lens matched the experimental tint 10 correctly guessed the experimental tint and 11 guessed the placebo tint. So far so good.
The study was of a crossover design so that participants wore the each set of glasses for one month and were randomised to receive placebo or experimental lens first. At the end of each month, participants were assessed using the Neale Analysis of reading test and throughout the study subjects kept symptom diaries. The big problem with this study was losses to follow up. For the Neale Analysis of Reading part of the study, data were available for 45 of 68 participants and no difference between placebo and experimental lenses was found for reading accuracy, speed or comprehension. With regard to subjective comfort completed diaries were only available for 36 of 68 participants. The correct response would have been to say that there was insufficient data to draw any meaningful conclusions and the correct response of any peer reviewed journal would be to reject a paper relying on complete case analysis of such depleted data. Instead, the authors were allowed to publish conclusions based on this data and have been misrepresenting the study for over 20 years.
 One of the authors does actually know better. In a correspondence in the British Medical Journal, it was acknowledged that there was a problem with losses to follow-up and it was stated that if the study is indeed compromised by attrition so are all the inferences that can be made from it both from positive and negative.  A study with loss to follow up of nearly 50% is indeed compromised.
However, in the OiP review there appears to be no problem with making positive inferences: On page 104 under the heading - Precision is necessary - strong evidence is now available - the 1994 study that was so hampered by poor follow that no positive or negative inferences could be made, is cited.

Elsewhere in the review where the authors might usefully have cited this paper there is no mention. For example on page 108 the authors state that single masked trials have demonstrated benefit from overlays (though they do not state which) but the double masked trials have not and go on to say that it is difficult, if not impossible, to mask the choice of overlays. Not true; the 1994 study showed that good masking can be maintained.

Continuing medical education(CME)
Pharmaceutical companies know that CME is a happy hunting ground for promoting products. If they can get their 'Key Opinion Leaders' to promote their product, that is more effective than any advertising and that is why pharmaceutical companies fund so much medical education. So the medical profession has 'form' in this regard and it is a significant blot on its record. For this reason I do not approach this with any sense of moral superiority. I hope that optometrists are better and finance their own continuing education. However, what looks like promotional activity dressed as science leaves me feeling uneasy.

Conclusion
The quotation at the start of this blog 'The greatest obstacle to discovery is not ignorance, it is the illusion of knowledge' sums up the problem with this 'review'. The authors have presented and continue to present the evidence for treatment with coloured lenses and overlays with an unwarranted certainty.