Tag Archives: Rails

Using Client APIs to Build Better Riak Applications

October 15, 2013

Ripple has not been maintained because we’ve learned that it’s not the right way to think about Riak. Using the riak-client APIs directly leads to better applications. We’re moving Ripple to basho-labs to avoid confusion.

The Ripple State of Mind

The Ripple document-relational mapper tool for Riak allows you to treat Riak objects like Ruby objects, very similarly to how ActiveRecord lets you treat Postgres rows like Ruby objects. This neglects the fundamental differences between Postgres and Riak, and encourages developers to use Riak badly.

SQL is a nice fit for Rails-like object usage because adding indexes isn’t prohibitively expensive, querying with indexes is cheap, and there’s a query planner that can use or mix indexes when available and can resort to a table scan when they’re not. Ripple, while it does have secondary index (2i) support, doesn’t have a planner to do set math on multiple indexes, so you either get to implement that yourself or write composite indexes. Adding an index after you have a dataset in production is hard too; it either only applies to new data or requires an expensive migration step, in which you load and re-save old records.

Missing Features

Ripple doesn’t provide any way to use some Riak 1.4 and planned 2.0 features, such as streaming 2i, multi-get, 2i return terms, and CRDTs. Ripple also doesn’t make it easy to make your frontend vector clock aware, which limits its usefulness in scenarios that create siblings.

These are complex features, and trying to wrap them in Ripple won’t necessarily make them easier to use.

My experience with complex Rails-style applications is that models eventually grow a bunch of class and instance methods to handle cases that are awkward for the ORM layer. An ActiveRecord model might have a SQL-using method as an optimization for a specific use case, or instance methods that perform a write and return something sensible in the case of a Postgres constraint violation.

Replacing Ripple

Rails applications that want to use SQL without using ActiveRecord’s ORM can do so: just useconnection.select_all and write some SQL. With Ripple, you can always drop down to riak-ruby-client and do work that way.

With that in mind, instead of the generic Riak 1.0 feature set in Ripple, we recommend wrapping Riak client methods in model objects. This exposes more complexity initially but, as your application grows and evolves, provides better opportunities to integrate new Riak features that help queryability, denormalize to reduce the number of Riak interactions that have to be done, automate certain data types, or provide consistency guarantees in appropriate situations.

Ripple Today

We’re moving Ripple to the basho-labs organization on GitHub to accurately reflect its status as unmaintained and deprecated.

Bryce Kerley

Rails Rumble with Riak and Ripple – Three Developers and a Lot of Rs

October 19, 2012

This past weekend, Bryce Kerley, David Andersen, and I had the honor of participating in Rails Rumble 2012 (of which, I am proud to say, Basho was a sponsor). This event is a 48-hour competition to design, build and deploy a web application in Ruby and Rails (or one of the other Rack-based web frameworks). Developing at this pace is grueling, to say the least (especially since it took place immediately after we all flew home from Ricon 2012), but also really exciting. You get the whole product cycle experience, from design to feature releases to testing and QA, all in one intense weekend.

Our application is Brainload. It’s a social flashcard/memorization aid tool, which lets users create, share and review flash cards on various topics (useful for memorizing programming language syntax, new APIs, school exam preparations, and building foreign language vocabulary).

One of the design constraints that we undertook this weekend was to build this web app using only Riak as a back end — no traditional relational database involved. Specifically, we chose the Ripple object modeling library, which provides a Riak persistence layer for Rails models, as well as familiar ActiveModel type querying capabilities. This provided several advantages and challenges, in terms of agile web development in the accelerated timeframe of the Rumble challenge. On the plus side, since Ripple essentially treats Riak as a document store, there was no database schema management and no migrations — very helpful for rapid exploratory development. Ripple model relationships (powered by LevelDB-backed secondary index queries) provided the usual one-to-one and one-to-many capabilities required for web apps like this (users, decks, cards, categories and so on). On the challenge side, we couldn’t use some of the minor Rails features that are closely tied to ActiveRecord. For example, fixtures were not automatically loaded during rake testing tasks (the solution in that particular case was to explicitly load the needed YAML files, or to use a standalone test data generation package like Factory Girl). Fortunately, we had Bryce on our side, who was old hand at coding Rails with Ripple, to advise us and walk us through the finer points of Ripple-specific syntax.

Despite our best efforts, a last-hour commit did manage to introduce a bug (I think we were a bit ambitious with the social features we wanted to add right up to the last minute). If you try to add a card — ouch! — you get an error message. While it is frustrating that we can’t fix this bug while the judging continues, this is part of what makes the competition so exciting. Of course, now that the contest is over, we have future plans for our web app — bugfixes, features, and all.

I cannot emphasize enough what a fantastic learning experience this was. Both in terms of working with really talented developers on my team (who also turned out to have pretty amazing sysadmin and design chops), and also for getting unique hands-on insight into the benefits and rough spots of working with our own database client and object modeling libraries.

If you get the chance to next year, I highly recommend participating in the Rails Rumble (and meanwhile, looking through and voting for the other entries in the competition). It was exhilarating and educational, and I am especially proud of Basho for being a sponsor of this competition.

Ripple 0.8 is Full of Good Stuff

August 31, 2010

This is a repost from the blog of Sean Cribbs, our Developer Advocate. 

It’s been a while since I’ve blogged about a release of Ripple, in fact, it’s been a long time since I’ve released Ripple. So this post is going to dig into Ripple 0.8 (released today, August 31) and catch you up on what has happened since 0.7.1 (and 0.5 if you don’t follow the Github project).

The major features, which I’ll describe in more detail below, are:

  • Supports Riak 0.12 features
  • Runs on Rails 3 (non-prerelease)
  • Adds Linked associations
  • Adds session stores for Rack and Rails 3 apps

Riak 0.12 Features

The biggest changes here were some bucket-related features. First of all, you can define default quorum parameters for requests on a per-bucket basis, exposed as bucket properties. Riak 0.12 also allows you to specify “symbolic” quorums, that is, “all” (N replies), “quorum” (N/2 + 1 replies), or “one” (1 reply). Riak::Bucket has support for these new properties and exposes them as attr_accessor-like methods. This is a big time saver if you need to tune your quorums for different use-cases or N-values.

Second, keys are not listed by default. There used to be a big flashing warning sign on Riak::Client#bucket that encouraged you to pass :keys => false. In Ripple 0.8 that’s the default, but it’s also explicit so that if you use the latest gem on Riak 0.11 or earlier, you should get the same behavior.

Runs on Rails 3

I’ve been pushing for Rails 3 ever since Ripple was conceived, but now that the actual release of Rails 3 is out, it’s an easier sell. Thanks to all the contributors who helped me keep Ripple up-to-date with the latest prereleases.

Linked associations

These are HOT, and were the missing features that held me back from saying “Yes, you should use Ripple in your app.” The underlying concepts take some time to understand (the upcoming link-walking page to the Fast Track will help), but you actually have a lot more freedom than foreign keys. Here’s some examples (with a little detail of how they work):

You’ll notice only one and many in the above examples. From the beginning, I’ve eschewed creating the belongs_to macro because I think it has the wrong semantics for how linked associations work (links are all on the origin side). It’s more like you “point to one of” or “point to many of”. Minor point, but often it’s the language you choose that frames how you think about things.

Session stores

Outside the Ruby-sphere, web session storage is one of Riak’s most popular use-cases. Both Mochi and Wikia are using it for this. Now, it’s really easy to do the same for your Rails or Sinatra app.

For Sinatra, Padrino and other pure Rack apps, use Riak::SessionStore:

For Rails 3, use Ripple::SessionStore.

Webinar Recap – Riak with Rails

August 8, 2010

Thank you to those who attended our Rails-oriented webinar yesterday. Like before, we’re recapping the questions below for everyone’s sake (in no particular order).

Q: When you have multiple application servers and Riak nodes, how do you handle “replication lag”?

Most web applications have some element of eventual consistency (or potential inconsistency) in them by their nature. Object and view caches sacrifice immediate consistency for gains in throughput and latency, and hopefully provide a better user experience. With Riak, you can achieve acceptable data freshness by “reading your writes”. That is, use the same read quorum as your write quorum and make sure that the R+W is greater than N. For example, using R=W=DW=2 when N=3 will give a strong assurance of consistency.

Q: I find myself doing def key; id; end. Is there any easier way to tell Ripple the key?

Currently there is not. However, I’ve found myself using this pattern frequently when I want a meaningful key that is also an attribute. There’s an issue on the tracker just for this feature. In the meantime, you could use two method aliases:

“`ruby
class User
include Ripple::Document
property :email, String, :presence => true

# This forces all attribute methods to be defined
define_attribute_methods
alias_method :key, :email
alias_method :key=, :email=
end
“`

As long as your property is a string, this should work just fine.

Q: Any tips on how to handle pagination over MapReduce queries?

The challenge with pagination in Riak is that reduce phases are not guaranteed to run only once, but instead are run in parallel as results from the previous phase come in asynchronously, and then followed by a final reduce. So in a sense, you have to treat all invocations of your reduce function as a “re-reduce”. We have plans to allow reduce phases to specify that they should be run only once, but for right now you can get around this limitation.

Reduce phases are always run on the coordinating node, so if you put a reduce phase before the one where you want to perform pagination, you are pretty much guaranteed that the whole result set is going to be available in a single application of the final reduce. A typical combination would be a “sorting” phase followed by a “pagination” phase. Riak.reduceSort and Riak.reduceSlice are two built-in functions that could help accomplish this task.

Sean and Grant

Free Webinar – Riak with Rails – August 5 at 2PM Eastern

July 29, 2010

Ruby on Rails is a powerful web framework that focuses on developer productivity. Riak is a friendly key value store that is simple, flexible and scalable. Put them together and you have lots of exciting possibilities!

We invite you to join us for a free webinar on Thursday, August 5 at 2:00PM Eastern Time (UTC-4) to talk about Riak with Rails. In this hands-on webinar, we’ll discuss:

  • Setting up a new Rails 3 project for Riak
  • Storing, retrieving, manipulating key-value data from Ruby
  • Issuing map-reduce queries
  • Creating rich document models with Ripple
  • Using Riak as a distributed cache and session store

The presentation will last 30 to 45 minutes, with time for questions at the end. Fill in the
form below if you want to get started building Rails applications on top of Riak!

Sorry, registration is closed.

The Basho Team