January 30, 2013
Many teams run Riak in public cloud environments, either as a part of their infrastructure or as the foundation of it. Increasingly, we see enterprises and startups use a hybrid implementation that leverages both private infrastructure and public cloud services. This hybrid model is often used to address burst capacity issues, tenancy/location concerns, and simple proof-of-concept implementations prior to hardware acquisition.
Over the past few years, we have seen substantive adoption of Riak on Amazon Web Services. To that end, we are pleased that Basho has been approved as an Amazon Web Services Technology Partner. We look forward to highlighting interesting use cases, publishing detailed case studies of usage, and continuing to improve the usability and deployment speed of Riak on the AWS platform.
This post provides a high-level overview of your deployment options for using Riak on Amazon.
How Many Nodes?
Before we discuss the mechanics of implementation, it is important to consider the size of your deployment. One of the most frequent questions Basho is asked is, “How many nodes should I start with?”
If you have played with the Riak Fast Track you are familiar with deploying three nodes on a single machine. However, for production deployments, we recommend that your cluster be setup with a minimum of five nodes. For more details on how this minimum ensures the performance and availability of your implementation, please read the post entitled: Why Your Riak Cluster Should Have At Least Five Nodes.
So, you have a minimum of five nodes and you’ve decided that leveraging a cloud provider is appropriate for your current business needs. Now, how do you get started?
Amazon Machine Image
At its simplest, an Amazon Machine Image (AMI) is a pre-built machine image and configuration of Riak for Amazon EC2 users.
Obtaining and configuring the image is a relatively straightforward process. However, since Riak needs the nodes in the cluster to communicate with each other, there is some manual setup involved.
First, provision the Riak AMI onto the server of your choice via the AWS marketplace.
Once the virtual machine is created, manually configure the EC2 security group to allow the Riak nodes to speak to each other. The details of this step can be found on our docs portal under Installing on AWS Marketplace. However, this is generally as simple as opening a few inbound ports and defining a “Custom TCP rule.”
At this point, the machines can be clustered together. When the individual virtual machines are provisioned and the security group is configured, simply SSH into each machine and use internal riak-admin tools to join the nodes to the cluster.
But what if you want to automate some of the configuration of your cluster? Or, what if you want the ability to setup a VPC-based stack that includes:
- a front-end load balancer,
- a cluster of application servers,
- a Riak powered demo application,
- a back-end load balancer,
- and a cluster of Riak servers.
In that case, the Basho team has made available scripts that leverage AWS CloudFormation to build out your cluster in a scripted fashion.
Since this is a much different process than the previous method, it is well worth watching the introductory video (embedded below). In addition, the scripts in the cloudformation-riak repo can be thought of as “known good” templates. We accept Pull Requests and happy forking!
As always, there is a manual option.
If you need to control the system configuration or are most comfortable with software that you have built and deployed yourself, there is always the option to install from package or source.
For a full list of supported operating systems, check out the Installing and Upgrading page of the doc portal. In addition, we have recently launched a new download page that includes the source for the OSS version of Riak.
And easier to deploy than ever before. If you have feedback on present deployment alternatives, or recommendations on ways to make Riak support for cloud infrastructure easier, please drop us a note in the mailing list.
January 28, 2013
Copious is a social commerce marketplace that makes it easy for people to buy and sell the things they love. Copious uses social data from Facebook and Twitter to customize the shopping and selling experience for each individual user around their interests, taste, and style. This connects buyers and sellers to each other in a unique way that also presents exciting technical challenges.
In the summer of 2012, Copious decided to switch to Riak from MongoDB for their social login functionality. They currently store all registered accounts in Riak as well as the tokens that make it possible for users to authenticate with Copious via their Facebook or Twitter accounts. Copious now stores hundreds of thousands of keys in Riak.
Copious found that Riak’s key/value data model was a natural fit for their authentication scheme and that it was easy to set up and run on Amazon Web Services. They run an HTTP service in front of Riak that presents data in the appropriate form and handles events between the front-end application and the backend data storage.
“Operating in a cloud environment, our infrastructure must be resilient to failure. Sometimes machines or even entire availability zones just disappear, and Riak’s fault-tolerant design means we remain available despite these failures. Riak is one component we never worry about, and requires much less operational work than other datastores,” said Robert Zuber, co-founder of Copious.
Copious is one of the many companies now using a polyglot approach to their platform – using Riak alongside MySQL, Redis, SOLR, and other technologies depending on the problem they need to solve. They chose to use Riak for their social login functionality because of its operational simplicity, which allows them to easily scale up without sharding and provides the high availability required for a smooth user experience. Based on their success with Riak, Zuber has said that they are looking to move more data to Riak in the future.
You can sign up now on Copious to earn $10 off your first $20+ purchase, plus the added bonus of knowing that Riak is in the background helping the site do its magic. If you’re in San Francisco, make sure to attend our Riak meetup on Wednesday, February 13, where Copious will be presenting how they use Riak alongside other datastores.
To learn more about how Riak can benefit your eCommerce or retail platform, join our webcast, “Riak on Retail” on February 8th!