327 papers (13%) pass Stage 1 Review

I completed Stage 1 Review today. I budgeted two months for this and it only took one. Here are the counts:

Decision               N    Pct        Note
Grand Total          2,470 
Duplicates               2   0   Pct of Grand Total
Total w/o duplicates 2,468 
Not reviewed             0   0   Pct of Total w/o dups
Not in English           6   0   Pct of Reviewed
Not an RCT             267  11 
No valid participants   32   1 
NVI - not daily        234   9 
NVI - not D3           798  32 
NVI - other            526  21 
No valid comparison    209   8 
No valid outcome        69   3 
Stage 1 pass           327  13

It would be difficult for two reviewers to be totally consistent in this kind of review since many studies fail for multiple reasons. The critical kind of consistency is pass vs. fail.

However, a few notes: in retrospect, “not daily” should mean this study would otherwise pass, but the dosage is not daily. It’s usually easy to find large doses in an abstract, so, particularly at first, I probably coded some studies “not daily” when there other reasons, for example, “not D3,” that also would have caused the study to fail. Also, in retrospect, perhaps I should have used a looser requirement here, as studies that use ultraviolet light to increase vitamin D levels do not use daily treatments; at least three treatments a week would catch most of these.

There is a mix of stuff in “Not an RCT;” some of these are observational studies based on 25(OH)D status. That group is more interesting to me than the others in this group and I now wish I had separated them. A surprising large number of the studies in this group describe study protocols.

There would have been no studies in “No valid participants,” but I started using that category to describe studies with timing problems – either the treatment lasted less than four weeks or the outcome was measured after treatment ended. This group also got the studies in which mothers received supplementation and outcomes were measured in infants.

“NVI – not D3” mixes together studies where the intervention was vitamin D2, vitamin D metabolites (25(OH)D, 1-25(OH)2D, 24-25(OH)2D) and vitamin D analogues. In retrospect, I should have separated these.

“NVI -other” mixes together studies where the intervention didn’t include any type of vitamin D. In a lot of these the intervention was calcium, but there are probably even more in which the intervention was bone-strengthening drugs of various types.

“No valid comparison” generally means the intervention included more than just vitamin D, typically calcium. The problem with these if that if they show an effect, it’s not clear if the effect is due to the vitamin D, the “more”, or if it requires both.

“No valid outcome” typically means the study was designed to measure 25(OH)D response to supplementation and no other outcome measures were mentioned in the abstract.

There were more articles without abstracts than I expected, but I was surprised to find that I could use the PubMed link to easily locate the article in PubMed, which typically had a link to the full text. Many of these were letters to the editor reporting the results of a study. If PubMed didn’t help, I looked up the title of the article in Google Scholar. In one case using this method, I found a missing abstract, in French, in a French PubMed-like database and was able to use Google Translate to determine the study used vitamin D2.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.