Additional ads made no difference in conversion rate

Inspired by patio11's comment on Hacker News, here are the results of my latest A/B test that I ran on Coolest Friends. I separated new users into three groups, the control group, the "no display ads" group and the "fix IE margin bug" group. The last group had a change that is actually a bugfix, and was included as a sanity test. 

The "no display ads" group was never exposed to the ad unit highlighted in green in the screenshot. I wanted to know if having this ad unit would lower conversions, and was happy to discover it did not.

"Conversion" here is "visited again the next day", or more accurately "generated a pageview 24-48 hours since installed app". The conversion rate of the control group was 11.00%, conversion rate of "no display ads" group was 11.29%. This is not a significant difference. There were 2348 users in "no display ads" group and 2271 users in the control group. If I am not mistaken, this amount of users would have been enough to make a statistically significant finding of a change in conversion rate of about 2 percentage points.

The sanity test group had 2322 users in it and a conversion rate of 12.53%. Most users use IE and the margins in the UI were messed up on it, so correcting the bug should raise the conversion rate, which it did. One interesting finding was that the "spin the wheel" link highlighted in red in the screenshot received a lot more attention when the ads were replaced by whitespace. The link lets users gamble with the points they have earned in the app. With ads replaced by whitespace 10.09% used the wheel vs. 4.88% normally.