**Navigation**

effectsize:two-sample-t-test

To use the calculator, information is needed according to whether you are using the raw data or summary statistics.

The analysis can be conducted based on the summary statistics. In this case, a user needs to supply the sample size, sample mean, and sample variance for each of the two independent groups.

In addition, a user can provide the following optional information.

First, a user can specify the confidence limit to use and the default one is 0.95 for the 95% confidence interval.

Second, a user can choose to get the approximate CI (“Theoretical”) based on Algina and Keselman (2003) or the bootstrap method. The default method is based on Algina and Keselman (2003).

Third, when the bootstrap method is selected, one can specify the number of bootstraps to use. The default is 1,000.

Raw data can also be used to calculate the effect size. In this case, a user can upload a comma-separated value (.csv) file. The file should follow the strict format with the first column being the outcome and the second column being the grouping variable. The grouping variable can only take two values. One example is given below:

y,g 31.9444012014312,1 20.5406532971108,1 13.4639000249002,1 ... 20.3628306367208,2 29.7063904254911,2 26.5223485727072,2 ...

The first row of the data can be variable names. In this case, the option “With variable names” should be selected.

As for the case with the summary statistics, one can also specify the confidence limit, choose different types of CI, and change the number of bootstraps.

A sample output is given below:

The estimated effect size is:-0.3224 The associated 95% CI is: [-0.8305 , 0.1884] t-test results t-value = -1.249 Degrees of freedom = 58 p-value = 0.2168

The first part is the estimated effect size and the confidence interval. The second part provides information on the two-sided t-test.

The most widely used effect size measure for independent two-sample t-test is Cohen's d. It can be calculated as

$$ d = \frac{\bar{y}_1 - \bar{y}_2}{\sqrt{\frac{(n_{1}-1)s_{1}^{2}+(n_{2}-1)s_{2}^{2}}{n_{1}+n_{2}-2}}} $$

where $n_1$ and $n_2$ are sample sizes, $\bar{y}_1$ and $\bar{y}_1$ are sample means, and $s_1^2$ and $s_2^2$ are sample variances under two different conditions, respectively.

Cohen's d is a biased estimator of the population effect size. An unbiased estimator, often referred to Hedge's g, was developed by Hedge (1981). It corrects the Cohen's d in the following way:

$$ g = d\times \frac{\Gamma(m/2)}{\sqrt{(m/2)} \Gamma[(m-1)/2]} $$

where $m = n_1 + n_2 - 2$.

Two types of confidence intervals are available - one is based on Algina and Keselman (2003) and the other is based on the bootstrap method.

We implemented the method by Algina, J., & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. *Educational and Psychological Measurement*, *63*, 537-553.

Algina and Keselman (2003) constructed a confidence interval for the population effect size based on a non-central t-distribution. In the method, one first gets the lower and upper bounds of the non-centrality parameter as $\lambda_L$ and $\lambda_U$ with

$$\lambda_L = t^{-1}_{ncp}(t, df=n-1, 1-\alpha/2)$$

and

$$\lambda_U = t^{-1}_{ncp}(t, df=n-1, \alpha/2)$$

where

$$ t = \frac{\bar{y}_1 - \bar{y}_2}{\sqrt{\frac{(n_{1}-1)s_{1}^{2}+(n_{2}-1)s_{2}^{2}}{n_{1}+n_{2}-2}} \times \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}} . $$

Now the confidence interval is given by

$$ \left[\lambda_L \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}, \lambda_U \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}\right] $$

When the summary statistics are provided, a parametric bootstrap method is used. In this case, the bootstrap samples are generated from normal distributions $N(\mu_j, \sigma_j^2), j=1,2$ in which $\mu_j$ and $\sigma_j^2$ are replaced by the sample statistics $\bar{y}_j$ and $s_j^2$.

When the raw data are provided, a non-parametric bootstrap method is used. In this case, the bootstrap samples are generated by sampling the raw data with replacement.

effectsize/two-sample-t-test.txt · Last modified: 2023/12/26 13:49 by johnny zhang