User Tools

Site Tools


effectsize:two-sample-t-test

Effect size and confidence interval for independent two-sample t-test


Calculator

Group 1
Sample size
Sample mean
Sample variance
Group 2
Sample size
Sample mean
Sample variance
Upload data
Select your file
Data information
Options
Confidence limit
Number of bootstraps
Type of CI
Correction



How to use

To use the calculator, information is needed according to whether you are using the raw data or summary statistics.

When summary statistics are available

The analysis can be conducted based on the summary statistics. In this case, a user needs to supply the sample size, sample mean, and sample variance for each of the two independent groups.

In addition, a user can provide the following optional information.

First, a user can specify the confidence limit to use and the default one is 0.95 for the 95% confidence interval.

Second, a user can choose to get the approximate CI (“Theoretical”) based on Algina and Keselman (2003) or the bootstrap method. The default method is based on Algina and Keselman (2003).

Third, when the bootstrap method is selected, one can specify the number of bootstraps to use. The default is 1,000.

When raw data are available

Raw data can also be used to calculate the effect size. In this case, a user can upload a comma-separated value (.csv) file. The file should follow the strict format with the first column being the outcome and the second column being the grouping variable. The grouping variable can only take two values. One example is given below:

y,g
31.9444012014312,1
20.5406532971108,1
13.4639000249002,1
...
20.3628306367208,2
29.7063904254911,2
26.5223485727072,2
...

The first row of the data can be variable names. In this case, the option “With variable names” should be selected.

As for the case with the summary statistics, one can also specify the confidence limit, choose different types of CI, and change the number of bootstraps.

Output

A sample output is given below:

The estimated effect size is:-0.3224
The associated 95% CI is: [-0.8305 , 0.1884]

t-test results
t-value = -1.249
Degrees of freedom = 58
p-value = 0.2168

The first part is the estimated effect size and the confidence interval. The second part provides information on the two-sided t-test.


Methods

Point estimate

The most widely used effect size measure for independent two-sample t-test is Cohen's d. It can be calculated as

$$ d = \frac{\bar{y}_1 - \bar{y}_2}{\sqrt{\frac{(n_{1}-1)s_{1}^{2}+(n_{2}-1)s_{2}^{2}}{n_{1}+n_{2}-2}}} $$

where $n_1$ and $n_2$ are sample sizes, $\bar{y}_1$ and $\bar{y}_1$ are sample means, and $s_1^2$ and $s_2^2$ are sample variances under two different conditions, respectively.

Cohen's d is a biased estimator of the population effect size. An unbiased estimator, often referred to Hedge's g, was developed by Hedge (1981). It corrects the Cohen's d in the following way:

$$ g = d\times \frac{\Gamma(m/2)}{\sqrt{(m/2)} \Gamma[(m-1)/2]} $$

where $m = n_1 + n_2 - 2$.

Confidence Intervals

Two types of confidence intervals are available - one is based on Algina and Keselman (2003) and the other is based on the bootstrap method.

Algina and Keselman (2003)

We implemented the method by Algina, J., & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63, 537-553.

Algina and Keselman (2003) constructed a confidence interval for the population effect size based on a non-central t-distribution. In the method, one first gets the lower and upper bounds of the non-centrality parameter as $\lambda_L$ and $\lambda_U$ with

$$\lambda_L = t^{-1}_{ncp}(t, df=n-1, 1-\alpha/2)$$

and

$$\lambda_U = t^{-1}_{ncp}(t, df=n-1, \alpha/2)$$

where

$$ t = \frac{\bar{y}_1 - \bar{y}_2}{\sqrt{\frac{(n_{1}-1)s_{1}^{2}+(n_{2}-1)s_{2}^{2}}{n_{1}+n_{2}-2}} \times \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}} . $$

Now the confidence interval is given by

$$ \left[\lambda_L \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}, \lambda_U \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}\right] $$

Bootstrap method

When the summary statistics are provided, a parametric bootstrap method is used. In this case, the bootstrap samples are generated from normal distributions $N(\mu_j, \sigma_j^2), j=1,2$ in which $\mu_j$ and $\sigma_j^2$ are replaced by the sample statistics $\bar{y}_j$ and $s_j^2$.

When the raw data are provided, a non-parametric bootstrap method is used. In this case, the bootstrap samples are generated by sampling the raw data with replacement.

effectsize/two-sample-t-test.txt · Last modified: 2023/12/26 13:49 by johnny zhang