L
L
Lesha Fedoseev2014-09-12 21:11:20
Mathematics
Lesha Fedoseev, 2014-09-12 21:11:20

How to calculate the sample size when the frequency is less than 0.25?

Task: Estimate sample size before running A/B test -> N
Given:

  • significance (alpha) -> alpha
  • power (1 - beta) -> power
  • frequency (base conversion) -> P
  • minimum frequency delta (minimum difference in conversion) -> delta

How I calculate sample size:
N = (Z alpha + Z power ) 2 * ( 2 * P * (1 - P) ) / delta 2 ,
where Z-values ​​are computed from significance and power.
But I came across an article that says that this formula gives pretty accurate results for 0.25 < P < 0.75. In other cases, a correction is introduced:
phi = 2 * arcsin(P 1/2 )
N = 2 * (Z alpha + Z power ) 2 / (phi 1 – phi 2 ) 2
Link to source: medstatistic.ru/articles/sample_size. pdf
From here I have three questions:
  1. Do you really need to make this adjustment for conversions below 25%?
  2. If a correction needs to be introduced, is the correction formula for this problem correct:
    N = 2 * (Z alpha + Z power ) 2 / (2 * (arcsin(P 1/2 ) –arcsin ((P – delta) 1/2 ) )) 2
  3. And just in case: is the first formula by which I count the sample correct?

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question