I just gave advertising a try.
As you might already know, I run a service called Candy Japan which mails subscribers Japanese sweets twice a month. It has been mentioned in several blogs and traffic from those blogs has converted into paying customers at roughly 1% conversion rate. I suspected that blog mentions would probably work better than ads, but wanted to find out for sure.
Project Wonderful
Wanting to try something new instead of the same old AdSense, I gave Project Wonderful a try. It is an ad network popular especially among web comic authors and is more transparent in pricing than most. Instead of paying per click or per conversion, you pay "per day". What this means is that you can buy an ad slot on a site and no other ads are shown there while you hold it.
There is an auction system in place to decide the price of each slot. The prices vary from zero to almost a hundred dollars per day. You can search for slots based on several criteria. I decided to go with a mix of sites that had "japan" in the tags and then just some really popular ones to see if my service would have any appeal to a wider audience.
To get started I transferred money to the ad network and created several ads since almost all sites required a differently sized box.
Tracking conversions by channel
Initially I wanted to use Google Analytics conversion tracking, but found that many people do not click the "return to merchant" link after buying from PayPal, causing many conversions to go unreported. I discovered that rolling your own system is not at all difficult.
To know which ad resulted in a conversion, I set a cookie when a user comes in after clicking an ad. If the user just came in from an ad or has such a cookie, I pass it to PayPal so I can associate the conversion with that ad when I get an instant payment notification.
I learned from a talk given by Daniel James that when buying ads, you should not just track them as one lump but instead split by source of traffic. Each such source is called a "channel". In hindsight this is very obvious. Without knowing which channels work and which do not, you would not know what to keep doing and what to stop.
The results
Stop everything.
The rightmost column is named "visits", but really means landing page views. The "convs" column is the conversions. Leftmost column indicates the source of the traffic. For example c_mspaint refers to MS Paint Adventures.
I realized these sites would probably not convert well, but over 1000 visits and 0 conversions is still disappointing. Granted MS Paint Adventures and Pokéfarm are probably terribly converting sites, but still over 3 million ad impressions with 1000 people deciding that the service is interesting enough to click on, I would expect a conversion.
Conclusion
These ads only cost me $50 to run and might have been profitable if I had just two people sign up and remain signed up for a full year. There are thousands of possible traffic channels. This experiment really only tells me that these particular sites do not convert well for me.