Statistical certainty that the trend we observe is actually linked to the variable we’re studying.
Russian roulette (traditional 1 round in 6 chambers) in a hospice ward (where everyone has been given a prognosis of less than 6 months to live) would be a very high certainty of shaving months off the life of 1/6 of the studied population. In the grand scheme of things, that’s not a very high risk. But at the same time, we can look at it and say “yes, shooting oneself with a revolver is very bad for health.” Putting a more or less deadly round in the chamber is probably not going to be a hugely significant change in outcomes, even if we can objectively say that one is better or worse for the person’s health than the other.
Almost all dietary/nutrition studies involve much smaller swings in lifespan or health conditions, probabilistically over a smaller portion of the population, with less statistical certainty in the observations. But the science is still worth doing, and analyzing, because that all adds up.
There’s three metrics to think about:
Russian roulette (traditional 1 round in 6 chambers) in a hospice ward (where everyone has been given a prognosis of less than 6 months to live) would be a very high certainty of shaving months off the life of 1/6 of the studied population. In the grand scheme of things, that’s not a very high risk. But at the same time, we can look at it and say “yes, shooting oneself with a revolver is very bad for health.” Putting a more or less deadly round in the chamber is probably not going to be a hugely significant change in outcomes, even if we can objectively say that one is better or worse for the person’s health than the other.
Almost all dietary/nutrition studies involve much smaller swings in lifespan or health conditions, probabilistically over a smaller portion of the population, with less statistical certainty in the observations. But the science is still worth doing, and analyzing, because that all adds up.