View Single Post
Old 2014-05-14, 19:44   Link #7501
oompa loompa
Senior Member
Join Date: Jun 2007
Location: 28 37', North ; 77 13', East
Age: 28
Originally Posted by Ledgem View Post
I don't disagree entirely, but why should we stop at 100 repetitions? That's nothing compared to 1,000 repetitions... or is it? Standard deviation isn't anything overly complicated, and there's no P-value. It's just a way of presenting some repetitions. For more complicated statistical analyses people generally present their number of data points (N). What constitutes a good sample size? That depends on what we're trying to observe, and we can judge the strength (or power, to speak statistics lingo) of the study based on N.

But getting back to the repetitions of a single data point, something more than three points would be nice but I don't feel that it would be any more or less misleading. Seeing that something was done even three times convinces me that it wasn't just a one-time fluke, but I can still take issue with the technique or some other variable. Someone could repeat something even 1,000 times and it wouldn't answer that concern.

This isn't to say that I'm against the idea of doing more than three repetitions, or even stating how many repetitions were performed.
Well yes, there is a reliable sample size, depending on the amount of variables, and the distribution it comes from, because you fix the probability of precision, (usually 95%, I don't remember if that's just convention or if it maximizes power somehow), so your gains from increasing sample size beyond a point decrease dramatically.

Also, since you calculate your SD from your variance, it is again important to have a large sample size because variance IS something you can test statistically, not with a p-value, but still. In this example, the number of variables isn't given, but if it's more than 3 I can definitely say that 3 is not enough. Besides, SD is a useless statistic without a sample mean (which I'm assuming is also being calculated), I'm a little rusty (and generally stats is not my strong suit) but since the sample mean is derived from a normal distribution by the CLT, there is going to be an associated t-statistic and p-value.

It's not misleading if you're not trying to mislead anyone. It is misleading if the claim is that an SD with 3 data points (and less than 3 variables) is a good predictor for what the deviations of the results of future experiments will be. For example, if you get a result greater than x number of SD's for your 4th experiment, how will you know if something went wrong or right? Forget SD, if you don't have a reliable mean, how will you know whether your result was close to what the 'usual' result is or not? If you don't have a reliable variance, how will you know how much deviation is acceptable without being called an abnormality?

At the same time, I agree with you that 3 is better than nothing. Its always going to help, but one would have to say that the results are merely indicative, and not a reliable predictor for how future experiments will pan out.

Last edited by oompa loompa; 2014-05-15 at 04:22. Reason: wow.. i need to sleep before i post
oompa loompa is offline   Reply With Quote