Backend Brain Dump
May 12, 2006
This is a brain dump, you don’t have to read it, I won’t be asking questions later. It’s just like my brain right now – all over the place.
Reinvented on Rails
I’m currently in the process of completely rewriting my Java-based backend system in Ruby on Rails, which is cool. Not just because Rails is cool, but because it’s also really good. You can get so much more done with much less code.
For ages now, I’ve been wanting to get rid of the Java Enterprise system because it’s complete overkill for my needs (but I did it to keep my hand in for when I was applying for jobs back in 2004). I hate PHP, which is what most people are forced to use because they can’t install stuff on their servers (I can install anything on mine) and I also wanted to learn Rails as it’s something new and different to stimulate my tired and apathetic brain cells.
Anyway, the whole replacement thing is quite a big job. Here’s what the Java system does and what the new system needs to replicate:
- Accept payment notifications from both PayPal’s IPN (see The Thing About PayPal below) and Kagi’s KRPS (see The Thing About Kagi below). These verify the payments made with the respective providers, create the registration codes and send an email with the codes to the customer.
- Send Feedback form – this just attempts to get a sensibly formatted email to me, I intend to make this more interactive. Originally I was thinking I could track this stuff in the database, but email works fine.
- Lost Codes form – this sends out an email with all registration codes for a customer (the codes are never shown on screen for obvious reasons).
- Email Updates – subscribe / unsubscribe for news by email.
- Buy Now pages, which act as a shopping cart before sending the customer to PayPal or Kagi to enter their credit card details.
- Activation – registration codes entered into my apps are validated on the server (see The Thing About Activation below).
I am also adding some more stuff:
- PayPal Payment Data Transfer – this shows registration codes after a purchase has been made (Kagi displays them on their site). I could have done this with Java but never got around to it. I currently use Auto Return, which brings them back to the site but doesn’t show any codes.
- The ability to have discounts.
- Prettier emails, because I can.
- Some AJAX stuff so the web page doesn’t need to be reloaded or go off to another page to show messages and stuff.
- And some other neat additions that are so easy I can’t resist. I’ll show these off later.
The transition requires some changes to my database because of the way Rails works and the new features I am adding. The number of changes are scary and if I want to be able to go back to the Java system in a week or a month’s time with the minimum of fuss, my safest option is to ensure it will work with the new database structure.
At the moment I have done pretty much everything but the Kagi stuff, the Buy Now page, the activations and the Java changes. I think I will be done with this before the weekend is out. Next week I will need to test it all out with PayPal and Kagi’s before backing up and migrating to the new system.
I’m currently developing and testing this on my iMac and have installed and configured everything on my server. That was fun because I couldn’t get Apache to work nicely with the Apache FastCGI module (to make Ruby run quickly) and was also worried about the resource overhead, particular at times when I have to run both systems side-by-side.
Instead I have installed Lighttpd and configured Apache to proxy to Lighttpd when necessary. The good news is when I tested this with a full copy of my live database that not only did it work well but was also really fast. I’m a terrible sysadmin and this stops me worrying so much. My hosting company, Tektonic, will provide support for anything I do but I don’t want to veer off too much down the crazy route until I know it’s all working nicely.
On the admin side of things I currently use a Cocoa front-end that will be retiring after this. It’s really slowed down now that there are thousands of records. While it can do just about everything with the database, its only real purpose is to look up lost codes and resend failed transactions. It was quicker to write than anything in Java (especially as I already had the database code from the same abandoned project as that calendar view). Instead, my admin system will be based on Rails’ scaffolding code.
The system as a whole is quite meaty. It’s focused around the products, each of those have different versions and each version can have different licenses with different pricing. It can handle single sales or whole shopping carts and generally ties everything together. It needs to do all this because of the way that PayPal and Kagi work but it saves me so much work and interruption that it’s worth doing well. So far, Rails has really made all this plain sailing.
The Thing About Kagi
I really want to get rid of Kagi. I only ended up using them because people complained they couldn’t use PayPal. Everything about Kagi’s backend system sucks, and even though they’re meant to be rolling out changes soon it’s just endless hassle compared to PayPal, which is simple. And PayPal’s transaction fees are lower, transferring money is free and they don’t hold onto it for up to 7 weeks, and you can get at it at any time, and they don’t charge for refunds (esp. annoying when customer purchased wrong app), and, and, and…
However, Kagi’s strong point is the vast variety of payment methods they will accept. It costs me £10 ($18) to cash foreign cheques and when the net amount of the software you sell is £15, it’s not really an option. Also, I need to ensure a number of orders go to Kagi because they take a monthly wire fee of $15 and obviously the more sales there are the bigger my margin.
The Thing About PayPal
I had wanted to switch to using PayPal’s Website Payments Pro, but I can’t. Despite being released in the US in June of last year, I found to my dismay that it is still not available in the UK. Website Payments Pro lets you do away with PayPal pages altogether. The PayPal feaures I am using (Instant Payment Notification and Payment Data Transfer) allow me to send the customer to PayPal, have them make their payment there and bring them back to my website but it’s not quite as slick.
The Thing About Activation
OK, so activation is evil, right? I have been wondering about whether to keep this, but activation has its benefits. I introduced it as part of four separate measures I took after a cracked version of KIT appeared on the net a month after its release. I didn’t think that would be a huge problem, but it made me angry.
Anyway, those benefits:
- When emails appear to get bounced, I can see if someone has actually received the code. Quite often they have and this saves me from trying to track them down.
- I’ve had a number of transactions reversed on PayPal (which is unfair on me because the person will already have their codes) and this allows me to prevent that code from being used again in the case of a scam.
- I can see when someone is doing something wrong (e.g. a legit user entering their email address instead of their name).
- It allows me to see people trying to guess codes so I can have a good laugh.
- It allows me to track usage of complimentary codes to reviewers, sales reps, etc.
- It probably acts as a deterrent.
One of the drawbacks is not being able to activate a copy without an internet connection. However, you can use my apps for 15 days without any limitations and the trial period is reset each time a new version (even a x.x.x) is released, so I have never heard of this being a problem. On the privacy concerns, there is no issue. All that gets sent to the server is the same reg code that was sent from the server in the first place. Nobody has complained.
Also I should point out that apart from the reversed transactions, I have never deactivated a code. There is also virtually no abuse, except for two KIT users who seem to be sharing some codes around, but I’ve let that slide.