I completed Stage 1 Review today. I budgeted two months for this and it only took one. Here are the counts:
Decision N Pct Note Grand Total 2,470 Duplicates 2 0 Pct of Grand Total Total w/o duplicates 2,468 Not reviewed 0 0 Pct of Total w/o dups Not in English 6 0 Pct of Reviewed Not an RCT 267 11 No valid participants 32 1 NVI - not daily 234 9 NVI - not D3 798 32 NVI - other 526 21 No valid comparison 209 8 No valid outcome 69 3 Stage 1 pass 327 13
It would be difficult for two reviewers to be totally consistent in this kind of review since many studies fail for multiple reasons. The critical kind of consistency is pass vs. fail.
However, a few notes: in retrospect, “not daily” should mean this study would otherwise pass, but the dosage is not daily. It’s usually easy to find large doses in an abstract, so, particularly at first, I probably coded some studies “not daily” when there other reasons, for example, “not D3,” that also would have caused the study to fail. Also, in retrospect, perhaps I should have used a looser requirement here, as studies that use ultraviolet light to increase vitamin D levels do not use daily treatments; at least three treatments a week would catch most of these.
There is a mix of stuff in “Not an RCT;” some of these are observational studies based on 25(OH)D status. That group is more interesting to me than the others in this group and I now wish I had separated them. A surprising large number of the studies in this group describe study protocols.
There would have been no studies in “No valid participants,” but I started using that category to describe studies with timing problems – either the treatment lasted less than four weeks or the outcome was measured after treatment ended. This group also got the studies in which mothers received supplementation and outcomes were measured in infants.
“NVI – not D3” mixes together studies where the intervention was vitamin D2, vitamin D metabolites (25(OH)D, 1-25(OH)2D, 24-25(OH)2D) and vitamin D analogues. In retrospect, I should have separated these.
“NVI -other” mixes together studies where the intervention didn’t include any type of vitamin D. In a lot of these the intervention was calcium, but there are probably even more in which the intervention was bone-strengthening drugs of various types.
“No valid comparison” generally means the intervention included more than just vitamin D, typically calcium. The problem with these if that if they show an effect, it’s not clear if the effect is due to the vitamin D, the “more”, or if it requires both.
“No valid outcome” typically means the study was designed to measure 25(OH)D response to supplementation and no other outcome measures were mentioned in the abstract.
There were more articles without abstracts than I expected, but I was surprised to find that I could use the PubMed link to easily locate the article in PubMed, which typically had a link to the full text. Many of these were letters to the editor reporting the results of a study. If PubMed didn’t help, I looked up the title of the article in Google Scholar. In one case using this method, I found a missing abstract, in French, in a French PubMed-like database and was able to use Google Translate to determine the study used vitamin D2.