April 26, 2012
At Basho we love Yammer. Besides making a product that we rely on internally, they are long-time Riak fans and advocates, and have built a large Riak cluster to power notifications for their entire user base. But not every use case is a fit for Riak. Running multiple databases in production is not uncommon, and skilled engineering teams like Yammer’s will always select the best tool for the job.
To that end, Ryan Kennedy, Yammer’s Director of Core Services, presented at BashoChats 003 about some of the impressive work that he and his colleagues are doing with Berkeley DB. He goes in depth on how they came to select BDB, what they added on top of Berkeley to ensure it could scale and satisfy their availability requirements, and what their data set and request profile look like in production. There’s a lot of worthwhile and valuable information in here. (Ryan’s slides are here if you’re interested in the PDF.
Enjoy, and if you’re interested in speaking at a future BashoChats meetup, email me – email@example.com. Also, if you want to work with companies like Yammer, Twitter, Square, Simple, LinkedIn, and Basho building distributed systems, you should be at the next meetup. Keep an eye on the Meetup page for details.
March 26, 2012
This is a big week for Basho.
The first three days of Erlang Factory are primarily workshops, and Daniel Reverri will be teaching a 3 day class on Building Distributed Clusters with Riak. All attendees will walk away with a clear understanding of exactly why Riak is the best distributed database you will ever run in production.>
The actual conference spans Thursday – Friday, and the talk lineup for this year’s event is exceptional. The Basho team will be well-represented. Put these talks on your calendar if you’re attending:
- Test-First Construction of Distributed Systems – Joseph Blomstedt
- Building Healthy Distributed Systems – Mark Phillips
- Building Cloud Storage Services with Riak – Andy Gross
Several members of the Riak Community are also on the schedule:
- Erlang for .NET Developers – OJ Reeves
- Rewriting GitHub Pages with Riak Core, Riak KV, and Webmachine – Jesse Newland
Basho Bash West
We’re really excited about all the success surrounding Riak in 2011 and we’re continuously building on that momentum as we move deeper into 2012. The number of Riak users and community members are growing exponentially so we decided to throw a party to celebrate. We’re calling it Basho Bash West 2012, and it’s co-sponsored by our friends at Joyent, Yammer and Voxer.
Come join us on Thursday, March 29th, at 6:30PM. We are renting out Roe, and you won’t be allowed to pay for anything. You’ll also be leaving with some limited edition Riak swag that will make you the envy of all your friends. Various members of the Basho team will be in attendance, along with hundreds of developers, executives, and technology enthusiasts from the Bay Area. Miss this at your peril.
You must RSVP to attend.
I’m happy to announce the addition of Thomas Santero to the Basho family. Thomas (who also goes by “Tom”) is joining as Technical Evangelist and will be part of the Community Team.
He’s based in Staten Island and will be focusing his time on bringing Riak to developers everywhere along the Eastern seaboard. If you’re anywhere near Boston, New York, Philadelphia, Washington, DC, etc., and aren’t yet using Riak, Tom will soon change this. He’ll also be supporting global community growth in various venues as Riak adoption continues to explode in all corners of the world.
Tom can be found on Twitter as tsantero and goes by the same nick on IRC. Say “hello” when you get the chance and ask him a hard question about Riak.
*We are hiring another Technical Evangelist based in London or Amsterdam. If you want to help us take over Europe, get in touch with me – firstname.lastname@example.org.
March 5, 2012
Happy Monday. What better way to start the week than with a smart, insightful tech talk from a thoughtful, talented engineer?
Ted Nyman is the lead engineer at Simple, a startup that is changing the way people bank. Ted was kind enough to come out to BashoChats last week in San Francisco to share his views on building scalable, JVM-based services that just work in a talk called *Instant-ish Real Service Architecture*.
This talks runs just under 40 minutes and covers everything from valuable lessons learned shipping production applications to high-level, illustrative code samples that demonstrate how and why you should use the Dropwizard framework and its reliable, battle-tested underlying libraries when you need to ship services that make money. Ted’s also an entertainer and he brings almost as much humor and wit as he does worthwhile lessons and knowledge.
March 2, 2012
The second BashoChats Meetup was held last week at BashoWest. The office was packed with area developers and our two speakers, Ted Nyman and Peter Bailis, each delivered exceptional talks. Our awesome videographer Matt Fisher finished Peter’s talk first and it’s so good that we didn’t see any reason to keep it from you while he put the final touches on Ted’s.
Peter is Graduate Student in the much-heralded Berkeley CS department. Suffice it to say that we were honored to have him at BashoChats. He and some colleagues have been working on something called Probabilistically Bounded Staleness for Practical Partial Quorums (PBS). In short, PBS aims to define just how eventual “eventual consistency” is, and their research produced some fascinating findings that should affect how people view and deploy distributed databases like Riak, Cassandra and Voldemort.
This talk, the subject matter, and the presenter are all fantastic. Watch it twice and tell three friends about it. (The PDF version of the slides are here for any interested parties.)
We’ll have Ted Nyman’s talk up next week. In the meantime, join BashoChats so you can be a part of the next event.
February 22, 2012
Riak Control is Basho’s new OSS, REST-driven, user-interface for Riak. The code has been available for a few months now, but it’s officially supported in Riak 1.1, so we wanted share some details on what it’s about and why you should be excited about it.
Lowering the Barrier for Entry
Once a Riak cluster is up and running we want it to be as hands-free to administer as possible. Things should “just work,” like plumbing. But we’ll be the first to admit that a new user’s initial welcoming with Riak isn’t always as pleasant as it should be.
Some steps are unavoidable: downloading, installing, and/or building from source, etc. But once the initial work is done, the experience should be as inviting as possible. Riak is a very powerful database with numerous options and commands. Riak Control allows you to easily manage/inspect your cluster while ignoring many of these until needed.
Empowering Riak Administrators
When we first sat down to decide what the more important features for any Riak interface should be, one theme stood out above all the others: cluster management. We wanted to give developers and administrators the ability to quickly build a cluster, inspect nodes, and diagnose the health of their cluster. And we wanted it to happen fast.
Riak is about large datasets and clusters replicating that dataset for maximum availability and persistence. We’re working hard to help companies that write many, many GBs per day to clusters containing 50+ nodes. Riak Control is a tool that brings issues and risks front-and-center. And it gives customers the ability to take action in real-time.
The Two-Minute Tour
Riak Control is currently broken up into nested levels of detail. Each page in Riak Control is designed to give you just as much information as you need, nothing more. As you navigate the UI, you’ll gradually be taken deeper into the rabbit hole.
The Snapshot is what you’ll see when you first fire up Riak Control. It should give you a warm-fuzzy feeling when everything is A-okay: an unmistakable, beautiful green check mark.
For times when things aren’t perfect, you will be presented with a list of concern areas. Each will have links to other pages of Riak Control where you can take a closer look at the problem.
The cluster page is where you can get a quick look at all of the nodes in your cluster and manage membership.
With a glance you can see which nodes are partitioned from the rest of the cluster or offline, which are leaving or joining the cluster, view partition ownership, monitor memory, and more. And with a click you can add nodes to the cluster, take nodes offline for maintenance, and leave the cluster.
One level deeper than the cluster view is the ring page. This is where you can see the health of each partition. Most of the time, your ring will be too large to really manage from the ring view. But with the filters you can immediately find which partitions are owned by which nodes, partitions whose primary nodes are unreachable, current handoffs, and more.
Riak Control is not standing still. Riak 1.1 includes Riak Control in its early stages so we can begin to gather feedback. We want to know what it does right and what it does wrong. Your feedback and ideas are encouraged. Additionally, we have a list of features and functionality slated for future releases. None of these are set in stone, but here is a list of what we have planned…
While Riak Control is – at its heart – a simple REST API, we’re working to modularize it in a way that allows you to write your own modules/plugins. We want to see Riak Control become a collection of pieces that all snap and work together, empowering you to manage your cluster in the way that best fits your needs.
Currently Riak Control uses a pull model to gather information about the cluster. While this isn’t a performance issue, we very much want to make it a push-system. As things happen to the cluster, the cluster should notify Riak Control of the changes, which in-turn will notify the user.
Clicking on a node name from anywhere should take you to a page giving details specifically about that node, similar to the data you would get from a
riak-admin status command.
Bucket & Object Inspection
While low-level object manipulation isn’t designed to be a primary feature of Riak Control, it is a very handy tool to have, and extremely valuable when initially setting up Riak for the first time. More importantly, Riak buckets will be available to create and inspect.
Riak Control will feature a powerful interface for creating MapReduce queries. You will be able to debug, save, load, and execute previously saved queries with ease.
Customer Support Tools
In addition to the general tools provided for manipulation of the cluster and data, we also are planning for improve monitoring tools.
- View the log files of individual nodes
- See graphs of load, memory latency, disk usage, etc.
- Coalesce and bundle data for support tickets
- File support tickets
Any Comments, Questions, or Feature Requests?
Anything you’d like to share or ask? Join the Riak-Users Mailing List and tell us what you think. The other option is to fork the code and make your opinions known with a pull request or by filing an issue. You can also find some formal documentation on the Riak Wiki.
Thanks for being a part of Riak.
February 07, 2012
Basho’s usage of Travis is mainly via Riak’s client libraries and we’ve been thrilled with it to date. Over the past few months, Sean Cribbs, Basho’s lead on Riak’s Ruby Client, has been using Travis. Reid Draper, another member of the Basho Team, has also started using it for builds of Sumo, a Riak Clojure Client that he and a few others are spearheading. There is also a handful of other Riak-related projects that are using Travis regularly.
Based on Sean and Reid’s experience with Travis and their opinion of the project’s usefulness and viability, it was an easy decision for us to donate money to support its ongoing development. Basho is also dedicated to the usage and proliferation of quality open source projects, and Travis is a great example of this. So we are proud to be a Silver Sponsor of Travis CI and are looking forward to watching the tool and its community grow.
January 31, 2012
The videos from last month’s San Francisco Riak Meetup are online and ready for consumption. The first features Julio Capote giving a short overview of the work he and Posterous are doing with Riak as a post cache. The second presentation was from Mark Phillips and it was all about Riak Control, the new Riak Admin Tool that will be fully supported in the forthcoming Riak 1.1 release.
Enjoy, and thanks for being a part of Riak.
Riak In Production At Posterous
This talk runs about 11 minutes. In it, Julio details the importance of the post cache at Posterous, what their initial solution to the problem was, and how they went about selecting Riak over MongoDB, MySQL, and Redis.
Preview of Riak Control
This talk runs just under 30 minutes. Mark starts with a history of the Riak Admin UI, details Basho’s motivations for writing and open-sourcing Riak Control, and then gives a live demo of the tool and talks about future enhancements.
Former Basho Developer Advocate Mathias Meyer authors a comprehensive, hands-on guide to Riak.
CAMBRIDGE, MA – January 17, 2012 – Basho Technologies, the leader in highly-available, distributed data store technologies, today announced that former Basho developer advocate Mathias Meyer has completed Riak Handbook, a comprehensive, hands-on guide to Riak, Basho’s industry-leading, open source, distributed database.
Riak Handbook begins by exploring the driving forces behind Riak, including Amazon Dynamo, eventual consistency and CAP Theorem. Through a collection of examples and code, Mathias Riak Handbook walks through Riaks many features in detail including the following capabilities:
- How to store-and-retrieve data in Riak
- Build and search full-text indexes with Riak Search
- Index and query data using secondary indexes
- Model data for eventual consistency
- Scale to multi-node clusters in less than five minutes
- Operate Riak in production
- Handle failures in your application
Mathias Meyer is an experienced software developer, consultant and coach from Berlin, Germany. He has worked with database technology leaders such as Sybase and Oracle. He entered into the world of NoSQL in 2008 and worked at Basho Technologies from 2010 to 2011.
“We are excited that Mathias took on the endeavor to build a comprehensive book all about Riak,” said John Hornbeck, Vice President of Client Services, Basho Technologies. “Our customers and community will benefit from having a single source that covers everything from setting up Riak, to scaling out quickly, to operating and maintaining Riak. We have already seen strong customer interest in Riak Handbook, including many seeking site licenses to outfit their entire teams.”
Riak Handbook is available for purchase at riakhandbook.com. Single editions are available at $29/download. Site licenses are available for organizations implementing Riak for only $249.
About Basho Technologies
Basho Technologies is the leader in highly-available, distributed data store technologies used to power scalable, data-intensive Web, mobile and e-commerce applications. Our flagship product, Riak, frees customer applications from the performance, scalability, and availability constraints of traditional databases while reducing overall storage and support costs by up to 80%. Basho customers, including fast-growing Web businesses and large Fortune 500 enterprises, use Riak to implement global session stores, to aggregate large amounts of data for logging, search, and analytics, and to manage, store and stream unstructured data.
Riak is available open source for download at basho.com/resources/downloads. Riak EnterpriseDS is available with advanced replication, services and 24/7 support. For more information visit basho.com or follow us on Twitter at www.twitter.com/basho.
Basho Technologies is based in Cambridge, MA, and maintains regional offices in San Francisco, CA and Reston, VA.
December 28, 2011
Distilling a year’s worth of work, innovation, and growth into one blog post is a fool’s errand. But we wanted to give it a shot regardless. This post is long, but it’s well worth the read. Make it to the end and you’ll see why. If you get there and regret it, let me know. I’ll send you some stickers.
2011 – A Look Back
2011 started off big for Basho and Riak. The fruits of our engineering labor were revealed in the Riak 0.14 Release that was made official on January 5th. This was a momentous event for us, and in the release were various feature additions and enhancements, along with copious bug fixes and usability improvements.
Next, in February, came a $7.5 Million Round of funding from some new and existing investors; they believed (and still believe) in our vision and product, and this money was put to good use building out the Basho team and pushing Riak farther.
With fresh funding in our coffers, we kept our heads down and continued to hack and hustle through February and March, picking up production users and closing new deals. April brought new interest in Riak Core, the framework that forms the backbone of Riak’s distributed capabilities. Companies like Yahoo! and AOL began to build applications on it for various use cases, and we did our best to make the project more usable outside of Riak. (There is still much to do to make Core truly accessible to developers, and, time permitting, we hope to address this in 2012.)
May arrived and we ruffled a few feathers with a blog post about what we thought was a theme that needed addressing in the NoSQL space. Also in May, Basho Board Member Eric Brewer was recruited to help Google plan and execute their cloud vision, one of the many accomplishments various members of the Basho Team would notch this year.
Corporate developments took center stage in June. We opened a new office in San Francisco, a move precipitated by massive user and customer growth on the West Coast. BashoWest, as we call it, has since become a co-working space of sorts in addition to our West Coast HQ, and we’re continuing to expand our efforts to spread knowledge about distributed systems and sound computing practices to developers. Later on that month we announced additional funding and the addition of Don Rippert as Basho CEO.
To start off July, we made it known that support for Google’s LevelDB would be part of the next release, a move that would let users take better advantage of Riak’s pluggable storage capabilities. Lager, a new logging framework for Erlang/OTP was also released and announced by Andrew Thompson. Writing and open-sourcing Lager was one of many steps we took in 2011 to address Riak’s (and Erlang’s) developer-friendliness. Client libraries were also on display in July. The Riak Java Client was given a makeover in response to user and customer demand, and Russell Brown and various community members continue to enhance the code. Ripple, Riak’s Ruby client, shared the spotlight. Sean Cribbs and his team of committers took over BashoWest for a week to hold the Ripple Hackathon, an event that contributed to what was a monumental year for Riak’s adoption in the Ruby Community.
We had our heads down in August, steadily grinding, only to re-emerge in September with a string of announcements about the code and features we were polishing off for the upcoming 1.0 release. The long awaited Secondary Indexing component of Riak was announced and chronicled by Rusty Klophaus; the work Bryan Fink was doing on Riak Pipe, our new MapReduce framework, was revealed in detail; Joseph Blomstedt, who we luckily snatched up after he released riak_zab, demonstrated the extensive work that he and the team had been doing to refine Riak’s Clustering Capabilities.
Then, to end the month, mere hours before September concluded, we released Riak 1.0. The culmination of years of hard work and innovation from Basho and our community, this release was the biggest in the history of Riak and it’ll be some time before any of us forget this day. Glance at the release notes to grasp the scope of this release if you’re not already running the code.
Onto October and November. The 111 Minna Gallery in downtown San Francisco played host to the Riak 1.0 Release party, an event attended by the entire Basho team along with almost 200 users, customers, and Basho supporters. Several weeks later we publicized the work we had been doing on the Riak-Hadoop Connector. In November it was also revealed that Basho’s Director of Engineering Dave “Dizzy” Smith had been named Erlang User of the Year for his work on Rebar. Also noteworthy from November: Scott Lystig Fritchie, another member of the Basho Engineering Team, shared some details on the work he and others were doing (and continue to do) to get DTrace added to Erlang.
Which brings us to December. Two big things happened this month, both on the 15th. First the Basho Developer Advocate Team released Riaknostic, a chunk of code, complete with beautiful documentation, intended to help eliminate operational issues before your Riak cluster goes live. Also on the 15th, Community Member Mathias Meyer released the Riak Handbook, a short yet near-comprehensive guide to using Riak, and the first extensive publication dedicated solely to Riak. (I’m told sales are booming.)
Community, Contributions, and Production Deployments
And even with all this, we are nothing without our community of users, customers, and contributors. As our COO Tony Falco has been known to say, we have a “community that sustains us with hard work and positivity.” Going into 2012, this could not be any less true.
The number of contributors to the projects that compose and are connected to Riak grew in a massive way, and the THANKS file now contains 170 names, up from about 40 at the beginning of the year. To date, hundreds of organizations and companies have contributed to the codebase, including Comcast, Yammer, GitHub, Trifork, Rails Machine, DISQUS, Formspring, Simple, Clipboard, Boundary, The Fedora Project, SEOmoz, SpawnGrid, Spreedly, ShowYou, Apollo Group… The list goes on. On the individual level, a special thanks is also owed to Tuncer Ayaz, for his dedication to Riak and Rebar.
Client library work that helped drive grass-roots adoption was done by people like Francisco Treacy (riak-js), Greg Stein, Soren Hansen, and Gilles Devaux, and Brett Hoerner (Riak’s Python Client), and Jeremiah Peschka and OJ Reeves, who took it upon themselves bring Riak to the .NET world. The Riak PHP Client was and continues to be refined by developers like KevBurnsJr, Jonathan Langevin, Mark Steele, and Eric Stevens.
We are immensely lucky, thankful, and grateful for these and future contributions, and we consider it a privilege to have you spend time working on and with Riak. Thank You!
Production deployment numbers also exploded, to the point where we are now comfortable saying there are more than 1000 Riak clusters either in production or that will be there very soon. Some of the noteworthy use cases:
- Voxer relied on Riak when they needed to scale their backend to handle billions of daily requests on their way to becoming the number one Social Networking application on the iOS.
- The Danish Government turned to Riak when they needed a datastore that could be trusted with the prescription records of their entire citizenry.
- Bump, the #7 free iPhone Application of all time, switched to Riak when they realized their existing infrastructure wasn’t sustainable.
- Yammer, which counts 80% of the Fortune 100 as customers, selected Riak to provide notifications to its millions of users
- DotCloud chose Riak to scale critical components of their internal infrastructure.
- ShowYou built out two Riak clusters to power their social video application and have developed a custom storage backend with integrated search and analytic capabilities.
These represent just a small portion of the hallmark deployments. We would need many more blog posts to provide details on all of them. Please add your use case details to the comments if you’re feeling compelled.
We also saw the appearance of a healthy dialog (the “good” and the “bad”) around what it takes to run Riak in production, driven by companies like Inaka Networks, The NetCircle, Production Scale/Solution Set, and Linklfluence sharing their stories. Riak isn’t perfect yet, and you’re driving us to make it better.
Another stat worth sharing: at least nine of the FORTUNE 100 have either deployed Riak or are committed to deploying it for services that generate revenue.
And so, with this, we close out a momentous 2011 knowing full well that what we have planned for 2012 will make the accomplishments and growth we saw over the past 12 months pale in comparison. Are we “market leaders”? Hard to say. This is not a title we can bestow upon ourselves. But this past year’s successes, coupled with the code, partnerships, new hires, products, customer announcements, and initiatives we have in the pipeline for 2012 have us feeling very good about the coming year.
Thanks for being a part of Riak. Happy Holidays.
On behalf of the entire Basho Team and our Community,