Hacker News new | past | comments | ask | show | jobs | submit login

The statement I see on that page is "There is a 91% chance that Variation A has a higher conversion rate".

I am not sure how to interpret that. We would have to dive into the GitHub repo and figure out which test it performs I guess.




It looks like the difference of two beta distributions based on the visualization.

So, assuming a uniform prior and updating with 39/3000 and 28/3000 conversions the difference between the two distributions is greater than zero 91% of the time. It's only guaranteed to be above zero at about the 80% credible interval, and since we started with an uninformed prior that'd be about p=.2?

I'm open to correction here.


You get 91% if you put a uniform prior on the proportion coming from each alternative.


Looks like I misunderstood the result, I will change the post to reflect this.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: