Much like with beards, when it comes to databases, bigger isn’t always bettter.  But at openplaces, we’re stuck between a rock and hard place.  A big hard place.  With terabytes of data.  With millions of records. Some painful truths:

• Distributed computing is messy.  Nodes go out of sync, data gets lost.
• There is no mature distributed database option.  HBase, Hypertable, Cassandra, CouchDB, MongoDB, Voldemort… They all have sharp edges.  You are going to get cut and you are going to bleed.
• It takes about 12 hours to upload our database to EC2.  But our data is growing.  Soon we’ll have to actually ship hard drives to Seattle: http://aws.amazon.com/importexport/.

On that up-beat note, Ruby coders, you are finally invited to start playing with BigRecord, your window to the world of Bigtable-esque data.  Which, for now, means HBase.  Click here to get started.

If you found this site and you’re wondering what the deal is, all I can say is: we’re working on it.  Basically we have some to strip dependencies to hbase and our application code, implement a driver for Cassandra to prove we’ve abstracting BigRecord properly, write some more tests and then package it all up nicely into a Gem.  Stand by for launch in about a month.

ActiveRecord is pretty awesome.  And column-oriented data storage?  Also awesome.  Relational databases for ambitiously scalable websites serving massive amounts of data?  Not so much.

So we were riding the Rails and we wanted to go big.  But replication and sharding with MySQL made us want to go home.  Then we got seduced by a Google-inspired bleeding-edge project called HBase, a siren who called out ‘distributed!’, ’schema-less!’ and ‘map/reduce!’.  The result?  A rewrite of ActiveRecord for a new kind of non-relational database.

BigRecord is ActiveRecord for web-scale column-oriented systems like HBase (e.g. Cassandra, Hypertable) and with some help, can be adapted for document-oriented databases, too (e.g. CouchDB, Dynomite, ThruDB, Tokyo Cabinet).

We’re putting it out there.