Blogging Bandits A behind the scenes look at BanditDeals

Duplicate listing bug resolved

BanditDeals is still in beta, and there have been a number of issues that we have been shoring up over the past few days. One of them involved duplicated listings in the search results. The root cause of that was a faulty approach to table locking in the database. That has been fixed.

There was also another revision to the number of days to archive listings for search results. Originally it was a month, then 2 weeks. However, after several fixes were made to the geolocation process, more listings could be dealt with than before so the size started to creep up again. Now, don’t get me wrong, BanditDeals it being written in a scalable fashion, but until the traffic arrives, it would be foolish to pay for hardware to support 3 weeks of extra listings when you know most visitors are going to be focused on results within that first week. So, recently the decision was made to limit the age of listings to 10 days. That puts the database well within acceptable limits and as traffic grows (and hopefully revenue with it) the server can be expanded to handle more traffic and more older listings. Our immediate goal is to keep search results to within a second and costs to a minimum for serving at least a week’s worth of listings across all of Craigslist, EBay, and Amazon. We are achieving that easily at the moment and with most of the phase 1 feature work behind us, I think this will hold true.