Laugh To Keep From Crying…

I just found this morning, through ironically the same customer that was having difficulties yesterday (this must be karma), that a key feature of my software has been disabled in the build on my website for the last 2 weeks.  I had been seeing traffic higher than ever, double the number of confirmed downloads I had in January, and was wondering “Why am I seeing 20-30% of my usual sales?  Well, it turns out the reason was that I was performing an involuntary A-B test: A with the feature,  and B without, based on what day you had first downloaded the software.  A wins by a longshot.

Here’s what happened: I ship Bingo Card Creator with about a dozen preprogrammed word lists.  Since I’m skeptical of my customers’ ability to do the Open File -> Navigate To Folder -> Select File routine, I make this very easy through a Wizards menu.  Click on the Wizards Menu, click your subject, click the menu item which describes what grade level/skill you are working on.  I had known this was a crucial feature when I included it in 1.02 because it alleviates the “empty screen problem” (I essentially can’t sell to someone who hasn’t seen a card printed out, and if you have to type in 25 words before you can print a card then you’re much less likely to invest the time).  I hadn’t known it was quite so crucial though.

Here’s what happened.  The Wizard menu is autogenerated, and unlike the vast majority of my code the logic is pretty smart.  It spiders a particular subdirectory in the installation, doing a breadth-first search of the file system tree, and making every directory a submenu and every properly formatted file an entry in the submenu it appears in.  This lets me add new items to the Wizard menu without tweaking anything in the Java code.  If a directory is empty, it doesn’t get shown.  If no files are found at all, the Wizard menu doesn’t show, because nothing ticks people off like non-functional programs, right?  (Grumble grumble.)

Anyhow, this feature has worked since 1.02 and I test to make sure it works every time I make a build, because its just so critical and because without the feature I’d actually have to type in word lists to type printing and that is slow.

I test in Eclipse and it works fine.

I test after exporting my JAR and it works fine.

I test after wrapping the JAR in the EXE and it works fine.

I test after building my installer and installing and it works fine.

My customer downloads the installer and sees no Wizards menu.

Can you spot what edge case my testing missed?  Oh, its fun — I accidentally introduced a single extra character into my Inno Setup script.  Since my setup script is not in my Subversion I never even noticed that I had made the change.  This resulted in the setup program copying the SampleLists (where the wizards reside) folder into the application directory, not into the application/SampleLists directory.  On my machine, since I was installing without uninstalling (clobbers identically named files but DOES NOT DELETE files not present in the new installer), I still had the old C:\Program Files\Bingo Card Creator\SampleLists directory with the proper structure, and things worked peachy.  Then my old, loyal customers who were doing updates installed and everything worked peachy.  Then new people downloaded the installer and, boom, no Wizards for two weeks.

After work today I’ll release a .01 “upgrade” to bugfix and all the other places that cache installers.

Lessons learned:

  • Setup script goes into the Subversion repository so that at any time it is either known-good or marked as changed since the last known-good.
  • Uninstall before installing to do final testing.  If I had another machine lying around I’d even do a virtual machine or something so that I was guaranteed of a fresh test environment.
  • Keep the lines of communication open with customers.  They can save you from yourself.
Explore posts in the same categories: bugs, customer service

5 Comments on “Laugh To Keep From Crying…”

  1. NetManiac Says:

    I had similar issue once, and since then VMware Player is my friend ;-))

    Clean operating system install and all dependencies and old version leftovers issues are brought to attention…

  2. I’ll second that, VMware Workstation has paid for itself over and over and I haven’t released yet! Being able to do a snapshot ‘revert’ every time I test the installer never ceases to impress me.

    Without it I just wouldn’t know for sure if my application actually runs under the various Windows versions.

  3. Andy Brice Says:

    I always test the final image on a clean machine or (more practically) a clean VM.

    Test what you ship. Ship what you test.

  4. You should really have your installer script in your source repository, too. The rule of thumb I use is that every single file needed to build and package a project goes into svn. You should be able to sit down at a random computer, update from svn, and build your package. (Well, ideally anyway, you’d still need to install the compiler etc.)

  5. Rich Fletcher Says:

    Enjoying your blog; it’s motivating. Thanks.

    Another voice here in favor of putting your installer script in the repository. It’s simply *amazing* how many places don’t do this, but it is a very common source of bugs. InstallShield can make this harder IMHO, but with Inno Setup, heck, you probably already did it. 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: