October 15, 2013
Ripple has not been maintained because we’ve learned that it’s not the right way to think about Riak. Using the riak-client APIs directly leads to better applications. We’re moving Ripple to basho-labs to avoid confusion.
The Ripple State of Mind
The Ripple document-relational mapper tool for Riak allows you to treat Riak objects like Ruby objects, very similarly to how ActiveRecord lets you treat Postgres rows like Ruby objects. This neglects the fundamental differences between Postgres and Riak, and encourages developers to use Riak badly.
SQL is a nice fit for Rails-like object usage because adding indexes isn’t prohibitively expensive, querying with indexes is cheap, and there’s a query planner that can use or mix indexes when available and can resort to a table scan when they’re not. Ripple, while it does have secondary index (2i) support, doesn’t have a planner to do set math on multiple indexes, so you either get to implement that yourself or write composite indexes. Adding an index after you have a dataset in production is hard too; it either only applies to new data or requires an expensive migration step, in which you load and re-save old records.
Ripple doesn’t provide any way to use some Riak 1.4 and planned 2.0 features, such as streaming 2i, multi-get, 2i return terms, and CRDTs. Ripple also doesn’t make it easy to make your frontend vector clock aware, which limits its usefulness in scenarios that create siblings.
These are complex features, and trying to wrap them in Ripple won’t necessarily make them easier to use.
My experience with complex Rails-style applications is that models eventually grow a bunch of class and instance methods to handle cases that are awkward for the ORM layer. An ActiveRecord model might have a SQL-using method as an optimization for a specific use case, or instance methods that perform a write and return something sensible in the case of a Postgres constraint violation.
Rails applications that want to use SQL without using ActiveRecord’s ORM can do so: just useconnection.select_all and write some SQL. With Ripple, you can always drop down to riak-ruby-client and do work that way.
With that in mind, instead of the generic Riak 1.0 feature set in Ripple, we recommend wrapping Riak client methods in model objects. This exposes more complexity initially but, as your application grows and evolves, provides better opportunities to integrate new Riak features that help queryability, denormalize to reduce the number of Riak interactions that have to be done, automate certain data types, or provide consistency guarantees in appropriate situations.
We’re moving Ripple to the basho-labs organization on GitHub to accurately reflect its status as unmaintained and deprecated.
October 19, 2012
This past weekend, Bryce Kerley, David Andersen, and I had the honor of participating in Rails Rumble 2012 (of which, I am proud to say, Basho was a sponsor). This event is a 48-hour competition to design, build and deploy a web application in Ruby and Rails (or one of the other Rack-based web frameworks). Developing at this pace is grueling, to say the least (especially since it took place immediately after we all flew home from Ricon 2012), but also really exciting. You get the whole product cycle experience, from design to feature releases to testing and QA, all in one intense weekend.
Our application is Brainload. It’s a social flashcard/memorization aid tool, which lets users create, share and review flash cards on various topics (useful for memorizing programming language syntax, new APIs, school exam preparations, and building foreign language vocabulary).
One of the design constraints that we undertook this weekend was to build this web app using only Riak as a back end — no traditional relational database involved. Specifically, we chose the Ripple object modeling library, which provides a Riak persistence layer for Rails models, as well as familiar ActiveModel type querying capabilities. This provided several advantages and challenges, in terms of agile web development in the accelerated timeframe of the Rumble challenge. On the plus side, since Ripple essentially treats Riak as a document store, there was no database schema management and no migrations — very helpful for rapid exploratory development. Ripple model relationships (powered by LevelDB-backed secondary index queries) provided the usual one-to-one and one-to-many capabilities required for web apps like this (users, decks, cards, categories and so on). On the challenge side, we couldn’t use some of the minor Rails features that are closely tied to ActiveRecord. For example, fixtures were not automatically loaded during rake testing tasks (the solution in that particular case was to explicitly load the needed YAML files, or to use a standalone test data generation package like Factory Girl). Fortunately, we had Bryce on our side, who was old hand at coding Rails with Ripple, to advise us and walk us through the finer points of Ripple-specific syntax.
Despite our best efforts, a last-hour commit did manage to introduce a bug (I think we were a bit ambitious with the social features we wanted to add right up to the last minute). If you try to add a card — ouch! — you get an error message. While it is frustrating that we can’t fix this bug while the judging continues, this is part of what makes the competition so exciting. Of course, now that the contest is over, we have future plans for our web app — bugfixes, features, and all.
I cannot emphasize enough what a fantastic learning experience this was. Both in terms of working with really talented developers on my team (who also turned out to have pretty amazing sysadmin and design chops), and also for getting unique hands-on insight into the benefits and rough spots of working with our own database client and object modeling libraries.
If you get the chance to next year, I highly recommend participating in the Rails Rumble (and meanwhile, looking through and voting for the other entries in the competition). It was exhilarating and educational, and I am especially proud of Basho for being a sponsor of this competition.
July 30, 2012
Basho’s vision is to “be the best organization in the world.” This vision applies to every aspect of Basho as a company, including the quality of our products, our culture and of course our customer support. We strive to provide the best customer support possible.
We have chosen Zendesk as our help desk platform given its ease of usability, customization, integration with other tools, and API. In our journey to dive into our deluge of now almost 2 years of accumulated Zendesk data, I first ventured over to Zendesk’s API page to check out the existing clients. Currently, only a Ruby client is listed but a Python library also exists.
As a data scientist desiring to do some complex analytics on our customer support data, I was stunned that no one had yet developed an R wrapper for Zendesk! Having used R since 2005 (mainly to analyze genetic and genomic data) as well as plenty of its libraries and the mailing list (often), I realized this was finally my chance to give back to the open source community, which is also very much in alignment with Basho’s commitment to open-source software.
As of this blog posting, zendeskR v0.2 supports 6 different API calls:
These calls access the data types that I was analyzing most often, but I will add more features, as well as update and maintain the package regularly. If there is an API call you would like to see supported, please feel free to shoot me an e-mail at the address noted in the package description.
The analytic possibilities are nearly endless by having all of this data in R. Some example questions that an organization could begin to answer with this data are:
- What is the average time to close a ticket for each customer?
- How many comments were posted to a ticket before it was finally resolved?
- How much does it cost to support customer X based on support and engineering time?
- Using sentiment analysis and a text corpus, what are the most frequently used words for tickets that receive a Good Satisfaction rating versus a Poor Satisfaction rating?
- Based on past trends, how many tickets will open this month?
- What is a developer advocate’s ticket closing rate and satisfaction rating?
Here’s a simple charting example that displays the number of tickets and users created by month.
To install zendeskR from CRAN, open an R console and type:
The analytical opportunities as described above are almost endless and at Basho we have only begun to scratch the surface. Ultimately, we aim to provide the highest quality products possible coupled with the best customer support system in the world, which is partially achieved by data-driven customer support system optimization strategies. Aim high.
April 12, 2011
Two of the best things about working with open-source software are being part of a community and collaborating with other skilled people. When your project becomes more than just a toy, you need others to help keep it going and to infuse fresh ideas into it. You need additional maintainers. Ripple (the Ruby client driver and tools) has been no exception to this rule, so this past weekend I was thrilled to welcome two new committers to the project.
Myron Marston (@myronmarston, github/myronmarston) is porting a large MySQL-backed application over to Riak at SEOmoz. Myron’s project uses the Ripple ORM fairly heavily, so he’s already fixed a bunch of things related to associations and validations.
Kyle Kingsbury (@aphyr, github/aphyr) has been working on some awesome Riak infrastructure at his day job at Vodpod. In the course of his work, he removed the ActiveSupport dependency from riak-client and created another Riak ORM called Risky.
Thank you Myron and Kyle for your continued contributions!
March 24, 2011
Adam Hunter is the newest Basho Developer Advocate!
Adam first got involved with Riak when he used it in conjunction with Ripple, Riak’s Ruby library, to build several applications at his previous position. (In addition to being a deeply-skilled Ruby developer, Adam has also spent some happy years writing PHP for fun and profit.) In the process, he started contributing patches and features to Ripple, and we liked his code and enthusiasm for the project so much that we extended him committer rights. Since then he has become an active and visible member of the Riak community, so we were quite pleased when he accepted the offer to come aboard.
Home base for Adam is Charlotte, North Carolina, so be sure to look him up if you’re in the area and are interested in getting an earful about Riak and distributed systems. You can also find him on Twitter and on GitHub as adamhunter.
December 16, 2010
The community contributions to Riak have been increasing at an exciting rate. The pull requests are starting to roll in, and I wanted to take a moment and recognize several of the many contributions we’ve received over the past months and weeks (and days).
Anyone who uses Riak with Ruby knows about Ripple. This is Basho’s canonical Ruby driver and its development has been spearheaded by Basho hacker Sean Cribbs. Not long after Sean started developing this code, he saw a significant influx in Rubyists who were interested in using Riak with Ruby and wanted to lend a hand in the driver’s development. Sean was happy to oblige and, as a result, there are now 15 developers in addition to Sean who have contributed to Ripple in a significant way. Special recognition should also be given to Duff Omelia and Adam Hunter who have made significant contributions to the code and use it in production.
Francisco Treacy and the team at Widescript made it known many months ago that they were looking into Riak to power part of their application. They, along with several other community members, were experimenting with Riak and Node.js. There were a few Node clients for Riak, but they were primarily experimental and immature. Basho had plans to write one but development time was stretched and a node client was several months off.
So, they rolled their own. Francisco, along with Alexander Sicular, James Sadler, Jakub Stastny, and Rick Olson developed and released riak-js. Since its release, it has picked up a ton of users and is being used in applications all over the place. (We liked it so much we even decided to build an app on it… more on this later.).
Thanks, guys, for the node client and helping to kickstart the Riak+Node.js community.
Riak Support in Spring Data
VMware’s Spring Data project is an ambitious one, and it has huge implications for the proliferation of new database technologies in application stacks everywhere. VMware made it known that Riak was slated for integration, needing only someone to take the time to write the code to connect the two. Jon Brisbin took up the task and never looked back.
Jon’s twitter stream is essentially a running narrative of how his work on Riak developed and, as you can see, it took about a month to build support for Riak into the Grails framework, the culmination of which was the 1.0.0.M1 release of the Riak Support in Spring Data.
So, if you’re using Riak with Spring Data, you have Jon Brisbin to thank for the code that made it possible. Thanks, Jon.
I met Daniel Lindsley at StrangeLoop in October. Rusty Klophaus and I were helping him debug a somewhat punishing benchmarking test he was running against a three node Riak cluster on his laptop (during a Cassandra talk) using Basho’s Python client. About a month later Daniel wrote a fantastic blog post called Getting Started With Riak & Python. Though his impressions of Riak were positive on the whole, one of the main points of pain for Daniel was that the Python library had poor documentation. At the time, this was true. Though the library was quite mature as far as functionality goes, the docs had been neglected. I got in touch with Daniel, thanked him for the post, and let him know we were working on the docs. He mentioned he would take a stab at updating the docs if he had a free moment. Shortly thereafter Daniel sent over a huge pull request. He rewrote all of the Python documentation! And it’s beautiful. Check them out here.
Thanks to Daniel and the rest of the team at Pragmatic Badger, we have robust Python documentation. Thanks for the contribution.
Want to contribute to Riak? There is still much code to be written and the Riak community is a great place to work and play. Download the code, join us on IRC, or take a look at GitHub repos to get started.
February 11, 2010
The Basho Dev. Team has been very excited about working with the Ruby community for some time. The only problem was we were heads down on so many other projects that it was hard to make any progress. But, even with all that work on our plate, we were committed to showing some love to Rubyists and their frameworks.
Enter Sean Cribbs. As Sean details in his latest blog post, Basho and the stellar team at Sonian made it possible for him to hack on Ripple, a one-of-a-kind client library and object mapper for Riak. The full feature set for Ripple can be found on Sean’s blog, but highlights include a DataMapper-like API, an easy builder-style interface to Map/Reduce, and near-term integration with Rails 3.
And, in case you need any convincing that you should consider Riak as the primary datastore for your next Rails app, check out Sean’s earlier post, “Why Riak should power your next Rails app.”
So, if you’ve read enough and want to get your hands on Ripple, go check it out on GitHub.
If you don’t have Riak downloaded and built yet, get on it.
Lastly, you are going to be seeing a lot more Riak in your Ruby. So stay tuned because we have some big plans.