Blog Hopping Servers, Be Back Shortly

Posted February 27, 2009 by Patrick
Categories: Uncategorized

This blog is getting transferred from wordpress.com to my own server at Slicehost.  This may result in some minor distortions as I get things working, and as DNS settings populate.

98, 99, 100!

Posted February 26, 2009 by Patrick
Categories: Uncategorized

Nice little milestone: Bingo Card Creator just crossed the 100th copy sold in February, for the first time.  The previous record was 93 back in October.

It is also fast approaching a much more consequential milestone, but more on that later.

Results of Native Compilation vs. Java Split Test

Posted February 24, 2009 by Patrick
Categories: Uncategorized

For the last month I’ve been testing a version of Bingo Card Creator compiled with Excelsior JET versus the standard Java one.  Half of all Windows downloaders have gotten one, half got the other.  I then have tallied when they confirm an installation (i.e. access my website through the application — for example by checking for updates or looking at the purchase page) and purchase the software.  

This is accomplished via a really simple expedient — Bingo Card Creator has a properties file in it, which contains all the strings for the interface like you would expect.  It also contains strings for the version number and site the trial was downloaded from, which I repurposed for the purpose of this split test (since I only track bingocardcreator.com versus “all the download sites” at present anyhow).  

So if you click “Purchase now” from version 2.51 of the Java version of the program after getting hit with the print quota limit, you end up accessing 

http://www.bingocardcreator.com/purchasing.htm?source=trial&version=2.51&site=bingocardcreator&button=printquota

At which point Google Analytics springs into action, uses some very simple logic to parse the query string, and cookies you up with that information.  If you purchase within the next 30 days, the cookie is passed in when you hit the order acknowledgement page and scored as a conversion.  There is a custom segment in Google Analytics devoted to both the JET-compiled and native versions of BCC 2.51.

Anyhow: in the last 30 days I have had 118 sales of Bingo Card Creator.  (Wow, that number shocks me.)  Google Analytics has racked up 116 conversions, meaning they caught the access of the confirmation page on almost all transactions.  Of these 116 conversions, 75 of them were cookied up with a cookie indicating they were positively part of this split test.  (You could easily have participated without getting cookied — for example, if you decided to buy the software and then opened up a browser and Googled [Bingo Card Creator], I would have no way of knowing which executable you had downloaded.)

Of the 75 purchases, 44 were for the JET version and 31 were for the native Java compiled version.  So what’s that mean?  OK, back to Stats 102: if we assume hypothetically that it doesn’t matter which version you were using, we would expect that 50% of the total number of conversions would turn out each way.  However, just like flipping a coin twice doesn’t automatically mean you get one head and one tail, it won’t necessarily be exactly 37.5 sales of each version.  (One would hope not, right?  I’d hate to support the half-and-half.)  

The probability of each possible outcome on the 0 purchases for JET through 75 purchases through JET continuum is given by the binomial distribution.  There is this notion of cumulative probability — given the assumption of 50/50 fairness between the two versions, the probability that any particular result is like to be random is equal to the sum of the probability of that result and all results less extreme than it.  

Plugging and chugging with a binomial probability calculator, we arrive at the result that, if the versions were fair, in only 8% of all samples of 75 sales would we find 44 or more sales for JET.  Accordingly, we conclude that JET is helping sales, by some amount which the binomial probability test is not quite sensitive enough to determine.

Which is good enough for me.  One niggle: I discovered that for reasons beyond my ken, the JET version cannot save to the program directory on Vista, whereas the Java version can.  I probably should have listened to the advice a year and change ago when people said I should revisit the choice of home directories.  (Vista prevents programs from writing to subdirectories of Program Files to prevent bad behavior.  However, it emulates writing to Program Files to avoid breaking legacy software, and that emulation is sufficient for BCC Java but apparently not good enough for BCC JET).  Ah, we live and we learn.  

After I patch that little issue, the JET compiled version of BCC is going to become the default version for all Windows users, and I will keep the Java version around only for support purposes, and to upgrade folks who already have one installed.

Much thanks to Dmitry at Excelsior, who donated a copy of JET to make this experiment possible.

Massive Update To Stats Tracking

Posted February 23, 2009 by Patrick
Categories: Uncategorized

Well, after a few hours of work I have revamped the way BCC presents its public stats.  You can now see everything from one single, easy location:  http://www.bingocardcreator.com/stats

Part of what I’ve been working on recently is taxes for 2008.  This required actually paging through and entering my expenses (oh, did I mention it tracks expenses now?), which is going to result in a bit of an earnings restatement for 2008.  I think I have everything but one stray freelancing invoice accounted for now.

Sales for 2008 (less returns): $21,116.65

Expenses for 2008 (excludes taxes): $11,782.62

Profit: $9334

You can see the detailed breakdown here — the big green slice is profit (~45% — a bit lower than I expected).  If you just want to see the relative expense breakdown that is available, too.  Unsurpisingly, my number one cost is advertising — the overwhelming majority of it is Google AdWords but I also paid much smaller amounts Microsoft, Stumbleupon, and a few directories. 

Since I’m using this to generate taxes for my business I included some things for my non-BCC projects which I intend to productize eventually but haven’t gotten around to yet.  Incidentally, expenses before 2008 aren’t totally entered yet, take that data with a grain of salt.  I don’t have easy access to all my old 2006/2007 credit card statements, and don’t even get me started about Paypal.

A/B Testing Lightbox Screenshot vs. Screenshot + Download Link

Posted February 22, 2009 by Patrick
Categories: web design

Everybody paying attention to this blog recently knows I have an unhealthy fascination with Lightboxes, particularly my new favorite iBox.  I like them because they work really well at increasing conversions, most recently in my shopping cart.  (Incidentally, guys, I went back and reset the test.  It really is about 100% improved over the old version.)

So this week I removed the old Lightbox lightbox from my home page and replaced it with an iBox.  I haven’t looked in any rigorous fashion whether that improved things or not — I just did it to cut a few kilobytes off all the Javascript people have to download to use my site, and also because I think the iBox looks cooler out of the box.

Then I noticed that the iBox introduces a loading screen, which the old lightbox did as well.  I hate load screens.  When I see them I think “This is me, losing money”.  Because if two seconds to load the shopping cart cost me half of my potential sales then 2 seconds to look at a screenshot must likewise be suboptimal, and MANY people look at the screenshot (something like 20% of the people who view the page, if CrazyEgg is counting right).

Also, unlike Lightbox, iBox can cause you to get dumped direct to image if you click before the script loads.  Bad news, bears, that means a bounce for most of my users.  I got around this by putting onclick=”false” on the relevant links.

Many, many moons ago (in late 2007!) I experimented with adding blue instructory text to the lightboxed image, exhorting people to download the trial.  That was a crashing failure — it looked like a link but if you clicked it, wham, the image vanished.  But I sort of like the general idea, so it is time for a new split test — take a gander at the screenshot behavior here and tell me if you like it.

You could see the old version of the behavior on my homepage but, well, only if you flip heads the first time you load it.  It is basically the stock lightbox effect — brings up an image, click anywhere to dismiss the image.

I’m hoping for a lift in trial downloads, particularly from clicks coming from advertising.  My worry is that the people are going to get stuck in the lightbox and bounce, since it is much harder to dismiss now.  Oh well, that is what testing is for.  Bring on the visitors!

Ninety. Four. Percent. “()$#(“)

Posted February 17, 2009 by Patrick
Categories: Uncategorized

The results from the shopping cart redesign are in.

Short version: Conversions increased by 94%.  I’m pretty flabbergasted.

Someday, someday…

Posted February 16, 2009 by Patrick
Categories: Uncategorized

… I will learn to not, under any circumstances, commit any code to the trunk which is not ready for immediate deployment.

Because when I do, I inevitably do “just a quick deploy to fix up that unrelated mistake” and then, boom, the lightbox on the front page silently fails for three days.

SIGH.

On the plus side, I once again have experimental confirmation that functional lightboxes beat the stuffing out of direct links to images!


Follow

Get every new post delivered to your Inbox.