September 29, 2010
Basho hackers will be giving quite a few presentations between now and the end of the week. And they all happen to be in Baltimore! Here is a quick rundown (in no particular order) of where we will be, who will be there, and what we will be talking about:
Rusty Klophaus at CUFP
Rusty Klophaus will be at the Commercial Users of Functional Programming (CUFP) Event taking place this weekend in Baltimore, Maryland. His talk is called “Riak Core: Building Distributed Applications Without Shared State” and it should be downright amazing.
From the talk’s description: Both Riak KV (a key-value datastore and map/reduce platform) and Riak Search (a Solr-compatible full-text search and indexing engine) are built around a library called Riak Core that manages the mechanics of running a distributed application in a cluster without requiring a central coordinator or shared state. Using Riak Core, these applications can scale to hundreds of servers, handle enterprise-sized amounts of data, and remain operational in the face of server failure.
All the details on his talk can be found here.
And, in case you haven’t been following your Riak Core developments, check out Building Distributed Systems with Riak Core.
Dave Smith at the Ninth ACM SIGPLAN Erlang Workshop
Dave Smith (a.k.a “Dizzyd”) will be keynoting the ACM SIGPLAN Erlang Workshop taking place on Friday, Sept 30th, also in Baltimore. Dave’s talk is called “Rebar, Bitcask and how chemotherapy made me a better developer.”
Rebar and Bitcask are both pieces of software that Dizzy had a major hand in creating and they have played a huge role in Riak’s adoption (not to mention that Rebar has quickly become an indispensable tool for Erlang developers everywhere). Dave was also fighting follicular lymphoma while writing a lot of this code. Needless to say, this one is sure to be memorable and of immense value.
More details on his talk can be found here.
It should also be noted that newly-minted Basho Developer Scott Fritchie is the Workshop Chair for this event. He is an accomplished Erlang developer and Riak is not the only distributed key/value store about which Scott is passionate – he will also happily talk your ear off about Hibari.
Justin Sheehy at Surge
Just when you thought they couldn’t fit another conference in Baltimore on the same weekend… And this one is big: it’s the Surge Conference put on by the team at OmniTI. Basho will be there in the form of CTO Justin Sheehy.
Justin will be giving a talk about concurrency at scale, something about which every distributed systems developer should care deeply. Additionally, he will be taking part in a panel discussion – I haven’t seen an official name for it yet but rumor has it that it’s something along the lines of “SQL versus NoSQL.”
Check out the Surge site for more conference details.
As you can see, Baltimore is the place to be this weekend. Get there at all costs. And then go download Riak.
September 16, 2010
Justin Sheehy, Basho’s CTO, and Bryan Cantrill, Joyent’s VP of Engineering, are two acknowledged authorities on distributed systems and cloud computing. Riak SmartMachines offer you all the advantages of Riak running on the best cloud technology there is.
What do you get when you put Justin Sheehy, Bryan Cantrill and a microphone in a room for 45 minutes to talk about NoSQL and Riak SmartMachine performance? One of the best webinars we’ve ever had the chance to take part in!
Watch this one at least three times. Seriously. And then go download Riak.
You can download a PDF version of the slides used in the webinar here.
September 9, 2010
At long last we have all the details ironed out for the upcoming September Riak Meetup in San Francisco. The crew here in SF is quite excited about this month’s event, and here’s why:
Date: Thursday, Sept. 23rd
Location: Engine Yard Offices, located at 500 Third Street, Suite 510
- 7:15 – Riak Basics
After the first meetup, one of the attendees remarked, “Good, but looking for some basics and some hands on demo as well.” Admittedly, this is something we could have addressed a bit better. So at the beginning of this meetup (as well as all meetups moving forward) we are going to devote at least 15 minutes to discuss Riak basics. There are no stupid questions. Ask away.
- 7:30 – Riak vs Git: NOSQL Battle Royale
Presenter: Rick Olson, Github
This talk will compare and contrast Riak and Git on their merits as key/value stores, and look at how the two can work together.
- 8:00 – From Riak to RabbitMQ
Presenter: Andy Gross, Basho Technologies
This will cover using Riak to publish to RabbitMQ using post-commit hooks and gen_bunny.
- 8:30 – General Riak/Distributed Systems Conversation and Networking
Note: There is only seating for 50, so you’ll want to get there on time to secure a seat.
Basho will be providing food (pizza) and refreshments (beer, soda, etc.). And for those of you who can’t join us next Thursday, I will also be filming the talks with the goal of posting them online if everything goes to plan.
You can RSVP on the Riak Meetup Page. So go do it. Now!
Hope to see you there.
September 3, 2010
Links and link walking are two very powerful concepts in Riak. If used appropriately within an application, they can help to add another level of relationships to your data that are typically not possible when using a key/value store as your primary storage engine. It’s for this reason that we thought new users should know how to use them.
Today we are adding a Links and Link Walking tutorial to the Riak Fast Track. In about 20 minutes, you will learn what links are and how to use them. And, to cap it all off, there is a 12 minute screencast put together by Sean Cribbs to give you a more in depth look at what links are all about.We think this is pretty cool stuff and we have a feeling you’ll enjoy it, too.
As usual, we love, want and need feedback, so send an email to firstname.lastname@example.org with any questions or comments you might have. (I’ve also been known to send t-shirts in exchange for thoughts on how to make Riak and our documentation better…)
Enjoy, and thanks for using Riak!
August 31, 2010
This is a repost from the blog of Sean Cribbs, our Developer Advocate.
It’s been a while since I’ve blogged about a release of Ripple, in fact, it’s been a long time since I’ve released Ripple. So this post is going to dig into Ripple 0.8 (released today, August 31) and catch you up on what has happened since 0.7.1 (and 0.5 if you don’t follow the Github project).
The major features, which I’ll describe in more detail below, are:
- Supports Riak 0.12 features
- Runs on Rails 3 (non-prerelease)
- Adds Linked associations
- Adds session stores for Rack and Rails 3 apps
Riak 0.12 Features
The biggest changes here were some bucket-related features. First of all, you can define default quorum parameters for requests on a per-bucket basis, exposed as bucket properties. Riak 0.12 also allows you to specify “symbolic” quorums, that is, “all” (N replies), “quorum” (N/2 + 1 replies), or “one” (1 reply). Riak::Bucket has support for these new properties and exposes them as attr_accessor-like methods. This is a big time saver if you need to tune your quorums for different use-cases or N-values.
Second, keys are not listed by default. There used to be a big flashing warning sign on Riak::Client#bucket that encouraged you to pass :keys => false. In Ripple 0.8 that’s the default, but it’s also explicit so that if you use the latest gem on Riak 0.11 or earlier, you should get the same behavior.
Runs on Rails 3
I’ve been pushing for Rails 3 ever since Ripple was conceived, but now that the actual release of Rails 3 is out, it’s an easier sell. Thanks to all the contributors who helped me keep Ripple up-to-date with the latest prereleases.
These are HOT, and were the missing features that held me back from saying “Yes, you should use Ripple in your app.” The underlying concepts take some time to understand (the upcoming link-walking page to the Fast Track will help), but you actually have a lot more freedom than foreign keys. Here’s some examples (with a little detail of how they work):
You’ll notice only one and many in the above examples. From the beginning, I’ve eschewed creating the belongs_to macro because I think it has the wrong semantics for how linked associations work (links are all on the origin side). It’s more like you “point to one of” or “point to many of”. Minor point, but often it’s the language you choose that frames how you think about things.
Outside the Ruby-sphere, web session storage is one of Riak’s most popular use-cases. Both Mochi and Wikia are using it for this. Now, it’s really easy to do the same for your Rails or Sinatra app.
For Sinatra, Padrino and other pure Rack apps, use Riak::SessionStore:
For Rails 3, use Ripple::SessionStore.
August 20, 2010
Thank you to those who attended our webinar yesterday. Like before, we’re recapping the questions below for everyone’s sake (in no particular order).
Q: How would solve full text search with the current versions of Riak? One could also take Wriaki as an example as most wikis have some sort of fulltext search functionality.
I recommend using existing fulltext solutions. Solr has matched up well with most of the web applications I have written, and would certainly work for Wriaki as well.
Q: Where in the course of the interaction (shown on slide 18) are you defining the client ID? Don’t you need the client ID and vclock to match between updates?
On slide 42, we talk about “actors” which are essentially client IDs. Using the logged-in user as the client ID can help prevent vclock explosion and is a sensible way of structuring your updates.
August 13, 2010
Documentation is great, but playing with examples can also be a helpful way to tackle steep learning curves. To help you learn about ways of using Riak, we’d like to present “Wriaki”, an example implementation of a wiki that stores its data in Riak.
We invite you to join us for a free webinar on Thursday, August 19 at 2:00PM Eastern Time (UTC-4) about Riak in Action: Wriaki. During the presentation, Bryan Fink will cover:
- Modeling wiki data in the Riak key/value store
- Access patterns using both get/put and map/reduce
- Three strategies that Wriaki uses for dealing with eventual consistency
- how the user interface changes to accommodate Wriaki’s models
The code for Wriaki will be open-source at the time of the presentation. The presentation will last 30 to 45 minutes, with time for questions at the end.
Fill in the Sorry, registration has closed! form below to reserve your seat!
If you cannot attend, the video and slides will be made available afterward in the recap post on the blog.
August 8, 2010
This is a huge day for Basho Technologies, Riak, and our growing community of users.
We are thrilled to announce Basho’s partnership with Joyent to bring our community hosted Riak on Joyent’s Smart platform. With both open source and enterprise versions available, anyone can quickly spin up a Riak cluster and start building applications.
When we first began talking to Jason and David and the rest of the Joyent team early this year, we realized we shared a common vision for the future of infrastructure. The past several months have been spent finalizing the details, and in just a few weeks you’ll be able to go to my.joyent.com and, with a few clicks, purchase and deploy as many nodes of Riak you want, need, and can handle.
Making pre-configured Riak SmartMachines available in the Joyent cloud will enable developers to combine all the benefits of Riak with the proven, advanced hosting platform that businesses like LinkedIn, Gilt, and Backstage rely on every day.
Mark your calendar, because hosted Riak is here!
August 8, 2010
Thank you to those who attended our Rails-oriented webinar yesterday. Like before, we’re recapping the questions below for everyone’s sake (in no particular order).
Q: When you have multiple application servers and Riak nodes, how do you handle “replication lag”?
Most web applications have some element of eventual consistency (or potential inconsistency) in them by their nature. Object and view caches sacrifice immediate consistency for gains in throughput and latency, and hopefully provide a better user experience. With Riak, you can achieve acceptable data freshness by “reading your writes”. That is, use the same read quorum as your write quorum and make sure that the R+W is greater than N. For example, using R=W=DW=2 when N=3 will give a strong assurance of consistency.
Q: I find myself doing
def key; id; end. Is there any easier way to tell Ripple the key?
Currently there is not. However, I’ve found myself using this pattern frequently when I want a meaningful key that is also an attribute. There’s an issue on the tracker just for this feature. In the meantime, you could use two method aliases:
property :email, String, :presence => true
# This forces all attribute methods to be defined
alias_method :key, :email
alias_method :key=, :email=
As long as your property is a string, this should work just fine.
Q: Any tips on how to handle pagination over MapReduce queries?
The challenge with pagination in Riak is that reduce phases are not guaranteed to run only once, but instead are run in parallel as results from the previous phase come in asynchronously, and then followed by a final reduce. So in a sense, you have to treat all invocations of your reduce function as a “re-reduce”. We have plans to allow reduce phases to specify that they should be run only once, but for right now you can get around this limitation.
Reduce phases are always run on the coordinating node, so if you put a reduce phase before the one where you want to perform pagination, you are pretty much guaranteed that the whole result set is going to be available in a single application of the final reduce. A typical combination would be a “sorting” phase followed by a “pagination” phase.
Riak.reduceSlice are two built-in functions that could help accomplish this task.
CAMBRIDGE, MA – August 3, 2010 – Basho Technologies today announced Wikia, Inc. has selected Riak, Basho’s next-generation distributed data store, as the foundation for a new set of global services. Wikia is the 70th largest site on the Internet according to Quantcast and brings millions of people together daily to create and discover engaging content. Wikia selected Riak over traditional databases and other emerging data storage technologies to distribute its data around the world and bring it closer to its global audience.
“Riak has allowed us to do something that was impossible before,” said Artur Bergman, Wikia’s Vice President of Engineering and Operations. “With Riak we can break through the ceiling on performance imposed by traditional database technologies and continue to improve the experience of our users. We invest in technology that benefits Wikia’s growing user base, therefore Riak made perfect sense. Riak is fast, easy to run, and extremely resilient to the failure scenarios anyone with real operational experience knows are all too common.”
Founded in 2008 by former Akamai Technologies (NASDAQ: AKAM) executives and senior engineers, Basho designed Riak to provide the same high availability and rapid scaling properties provided by leading content delivery networks. Applications built with Riak can sustain catastrophic server, data center, and network failures without outages, while avoiding the complexity and expense that characterize applications built using traditional databases.
“Basho is excited to have a respected and forward-looking client like Wikia so readily embrace Riak,” said Earl Galleher, Basho’s Chairman and CEO. “More and more, we see companies reject the limitations of traditional databases like Oracle and MySQL in favor of Riak’s flexibility and ease of use. Riak doesn’t just solve problems for organizations running applications on old database architectures; it frees them to build entirely new classes of applications.”
Wikia intends to deploy a replicated user session service running simultaneously in three data centers in the U.S. and Europe, replacing its current solution which is restricted to a single data center. Mr. Bergman has already contributed a file system adapter to the Riak open source community which will be used in the Wikia production environment.
“We did not set out to build a disruptive technology. We simply wanted to solve a problem faced by anyone running old database technologies,” said Mr. Galleher. “We have only scratched the surface of what Riak can do.”
Wikia, founded by Wikipedia founder Jimmy Wales and Angela Beesley, is the place where millions of passionate people come to discover, create, and share an abundance of information on thousands of topics. Wikia sites are written by community members that are deeply excited and knowledgeable about subjects ranging from video games, television shows, and movies to food, fashion, and environmental sustainability. With over four million pages of content and 150,000 enthusiast communities, Wikia attracts more than 30 million unique global visitors per month and has been listed in the Quantcast top 100 sites on the Internet since early 2009.
About Basho Technologies
Basho Technologies, Inc., founded in January 2008 by a core group of software architects, engineers, and executive leadership from Akamai Technologies, Inc. (Nasdaq:AKAM – News), is headquartered in Cambridge, Massachusetts. Basho produces Riak, a distributed data store that combines extreme fault tolerance, rapid scalability, and ease of use. Designed from the ground up to work with applications that run on the Internet and mobile networks, Riak is particularly well-suited for users of cloud infrastructure such as Amazon’s AWS and Joyent’s Smart platform and is available in both an open source and a paid commercial version. Current customers of Riak include Comcast Corporation, MIG-CAN, and Mochi Media.
CEO, Basho Technologies, Inc.