Tag Archives: RICON

Basho in 2013

December 30, 2013

2013 was a huge year for Basho Technologies and before we dive into 2014, we thought we’d take a moment to reflect on how far we’ve come.

Case Studies

2013 was the year of the Riak User. We love hearing about all the amazing ways companies across various industries are using Riak. This year, we were able to share dozens of exciting case studies. These include:

For even more Riak Users, check out the Users Page.

Releases

We released Riak 1.3, Riak 1.4, and the Technical Preview of Riak 2.0 this year. These releases added such features as Active Anti-Entropy, revamped Riak Control, queryability improvements, Riak Data Types, and much more. Be on the lookout for the general release of Riak 2.0 early next year.

This year we also open sourced Riak CS with the 1.3 release and released Riak CS 1.4. These releases added multi-part upload, Riak CS Control, and integration with OpenStack.

RICON

This year, we expanded RICON, Basho’s distributed systems conference, to both RICON East and RICON West. These were both sold out conferences that featured speakers from bitly, Comcast, Google, Netflix, Salesforce, State Farm Insurance, The Weather Company, Turner Broadcasting, Twitter, and many more.

Partners

We drastically increased the number of Basho partners in 2013. For a full list of partners, check out the Partnerships Page. Some key ones to note include Tokyo Electron Device, SoftLayer, and Seagate.

Community

Our amazing community team hosted over 200 meetups around the world this year. On top of that, they also attended dozens of industry events to spread the word about Basho. Keep an eye on the Events Page to see where we’ll be in 2014.

2013 was a busy year but, with some exciting announcements coming, we look forward to an even busier 2014. Happy New Year!

Basho

RICON West 2013 Survey

November 5, 2013

RICON West, Basho’s distributed systems conference, took over San Francisco last week. RICON West was the largest RICON to date and was a huge success. We brought in speakers from both industry and academia to discuss the theory, practice, and importance of running distributed systems in production – as well as some predictions on what’s in store for the future.

For those that couldn’t attend RICON West, there was also a live stream available so you could tune in and watch all of the great talks. While we are editing the recorded talks, the video stream will remain available for you to watch any of the talks you missed or would like to rewatch. You can rewatch the video stream here.

If you attended RICON West or tuned in to the live stream, we would love your feedback! What did you love? How can we improve? Please take five minutes to fill out this quick survey.

We look forward to hearing from you!

Basho

Basho Releases eKinetic Driver and Integrated Riak Backend With Seagate Partnership

October 22, 2013

Today, Seagate has announced the availability of their Kinetic Open Storage platform, which simplifies data management, improves performance and scalability, all while lowering expenses. This fundamentally new architecture reduces costs by allowing applications to communicate directly with the storage system, eliminating the acquisition, deployment, and support costs of hyperscale storage infrastructures.

Basho has partnered with Seagate to help them develop this platform to provide interoperability and testing with Riak. Now, with the release of this platform, we want to make it easier for developers to test the Kinetic Open Storage platform with Riak. We have just released an alpha version of our eKinetic driver, which enables an Erlang-based high-performance socket connection to the drive. We have also released software to improve Riak backend compatibility by mapping a Riak backend to the drive library. Both are available for download https://github.com/basho-labs/riak_kinetic.

Not only does deploying Riak on this platform drastically simplify the management of data through a straightforward socket-based network interface, this simplification also increases I/O efficiency by removing bottlenecks and optimizing cluster management, data replication, migration, and active multi-datacenter performance. Additionally, it is expected that users will realize up to a 50% decrease in the Total Cost of Operations through simplified operations alone. Users can also maximize storage density through reduced power and cooling costs and build out cloud datacenters for even more savings.

Seagate Principal Technologist, James Hughes, will be speaking about the Kinetic Open Storage platform and Riak at RICON, Basho’s distributed systems conference. His talk, “Device Based Innovation to Enable Scale-Out Storage” will take place on October 29th at 12pm in Track Two. Seagate is also a sponsor of RICON.

For more information on Seagate’s Kinetic Open Storage platform, check out their full release. Additional information on the Basho/Seagate partnership can also be found here.

Basho

Basho and Seagate Partner to Deliver Scale-Out Cloud Storage Breakthrough

Basho makes available alpha eKinetic (Erlang) driver and an integrated backend for Riak

CAMBRIDGE, MA – October 22, 2013 – Basho, an expert in distributed systems and cloud storage software, announced today that it has partnered with Seagate Technology (NASDAQ:STX) to help significantly advance the economics and performance potential of cloud architectures. For the past six months, Basho has worked with Seagate on its development of its Seagate Kinetic Open Storage platform, providing interoperability and testing with Riak, Basho’s distributed NoSQL database.

The Seagate Kinetic Open Storage platform eliminates the storage server tier of traditional data center architectures by enabling applications to speak directly to the storage system, thereby reducing expenses associated with the acquisition, deployment, and support of hyperscale storage infrastructures. The platform leverages Seagate’s expertise in hardware and software storage systems integrating an open source API and Ethernet connectivity with Seagate hard drive technology.

Basho developed Riak to offer businesses a highly-available, fault-tolerant, distributed database ensuring ultra-low-latency performance that is simple-to-operate – at any scale. Today, Riak is used by thousands of companies including over 30 percent of the Fortune 50. Basho’s partnership with Seagate aimed to continue to improve on these ambitions and customers deploying Riak on the Seagate Kinetic Open Storage platform will see the following benefits:

  • An increase in I/O efficiency by removing bottlenecks and optimizing cluster management, data replication, migration, and active multi-data center performance
  • An improvement in customer Total Cost of Operations (TCO) by up to 50 percent through simplified operations
  • An additional cost savings by maximizing storage density through reduced power and cooling costs, and receiving potentially dramatic savings in cloud data center build outs

To assist with developers seeking to test the Seagate Kinetic Open Storage platform, Basho is making available an eKinetic driver enabling an Erlang-based high-performance socket connection to the drive. Basho is also providing software that maps a Riak backend to the drive library. Both the eKinetic driver and Riak backend compatibility are available as alpha version software.

“Seagate is bringing device based innovation to the scale-out cloud market in a new and open way,” said Ali Fenn, Senior Director of Advanced Storage at Seagate Technology. “This is a fundamentally new architecture – integrating an open source key/ value API and Ethernet connectivity into devices – that represents a vital leap forward in decreasing cloud architecture TCO while improving performance. We are very excited to work with Basho, a leader in distributed object storage software, to bring complete solutions to customers.”

“Basho’s distributed database relies on key/value stores directly attached to servers,” commented Jon Meredith, Senior Vice President of Engineering at Basho. “Seagate’s kinetic drive simplifies the management of key/value store, filesystem, logical volume manager, RAID controllers and actual devices by replacing them with a simple socket-based network interface. Freeing drives from server chassis enables independent scaling of capacity and throughput of a cloud architecture. We look forward to continuing to work with Seagate to offer customers significant performance and cost benefits when combining Riak on kinetic drive technology.”

Seagate at RICON West
James Hughes, Principal Technologist from Seagate, will be speaking at Basho’s distributed systems conference RICON West held in San Francisco October 29-30. His session is entitled Device Based Innovation to Enable Scale Out Storage. Seagate is a sponsor of RICON.

About Seagate
Seagate is a world leader in storage solutions. Learn more at www.seagate.com. Follow Seagate on Twitter, Facebook, YouTube, Instagram and subscribe to our blog.

About Basho Technologies
Basho is a distributed systems company dedicated to making software that is highly available, fault-tolerant and easy-to-operate at scale. Basho’s distributed database, Riak, and Basho’s cloud storage software, Riak CS, are used by fast growing Web businesses and by over 30 percent of the Fortune 50 to power their critical Web, mobile and social applications and their public and private cloud platforms.

Riak and Riak CS are available open source. Riak Enterprise and Riak CS Enterprise offer enhanced multi-datacenter replication and 24×7 Basho support. For more information, visit basho.com. Basho is headquartered in Cambridge, Massachusetts and has offices in London, San Francisco, Tokyo and Washington DC.

Riak CS vs. Riak

October 2, 2013

What Is Riak CS?

Riak CS is Basho’s open source large object (aka cloud storage) software, built on the rock-solid Riak database. It is API-compatible with Amazon’s S3 and OpenStack’s Swift object storage solutions.

In May of this year, we posted the top 5 questions we heard from customers and our community about Riak CS; today we’ll take a deeper dive into the technical details, specifically the differences between Riak CS and Riak itself.

Riak CS as Compared to Riak

Both Riak CS and Riak are, at their core, places to store objects. Both are open source and both are designed to be used in a cluster of servers for availability and scalability.

The fundamental distinction between the two is simple: Riak CS can be used for storing very large objects, into the terabyte size range, while Riak is optimized for fast storage and retrieval of small objects (typically no more than a few megabytes).

There are subtle differences; however, that can be obscured by the similarities between the two.

Why Would I Use Riak CS?

Riak CS is used for a variety of reasons. Some examples:

  • Private object storage services, for example for companies that want to store sensitive data behind their own firewalls.
  • Large binary object storage as part of a voice or video service.
  • An integrated component in an OpenStack cloud solution, storing and serving VM images on demand.

Tier 3, Yahoo! Japan, Datapipe, and Turner Broadcasting are just a few of the big names using Riak CS today.

What Does Riak CS Do That Riak Doesn’t?

Chunking

Riak CS carves large objects into small chunks of data to be distributed throughout a Riak cluster and, when used with Riak CS Enterprise, synchronized with remote data centers.

S3/OpenStack APIs

Without Riak CS, developers have the choice of using Riak’s native HTTP or Protocol Buffers APIs when developing solutions.

Riak CS adds compatibility with Amazon’s S3 and OpenStack’s Swift APIs. These offer very different semantics than Riak, and the advanced search capabilities in Riak such as Secondary Indexes and full text search are not available using S3 or Swift clients.

We strongly advise against it, but it is possible to work with Riak’s standard APIs “under the hood” when deploying a Riak CS solution.

Multi-Tenancy

The latest release of Riak offers no way to differentiate between clients. Riak CS, on the other hand, supports both authentication and authorization.

Work is actively underway to add a security model to Riak in the upcoming 2.0 release.

Buckets or Buckets?

Users of Riak CS store their objects in virtual containers (called buckets in Amazon S3 parlance, containers in OpenStack).

Riak also relies heavily on buckets for data storage and configuration but, despite the names, these buckets are not the same.

As an example of how this can cause confusion: the replication factor in Riak (the number of times a piece of data is stored in a cluster) is configurable per-bucket. Because Riak’s buckets do not underly the user buckets in Riak CS, this feature cannot be used to create tiered services.

Strong Consistency

Riak is designed to maximize availability; the price paid for that is delayed consistency when the network is split and clients are writing to both sides of the cluster.

Creating user accounts in Riak CS; however, led to the need for a mechanism to maintain strong consistency. If two people attempt to create user accounts with the same username on either side of a network partition, both cannot be allowed to succeed, or else a conflict will occur that is very difficult to automatically recover from.

Furthermore, user buckets in S3 (and OpenStack APIs as implemented in Riak CS) reside in a global rather than a user-specific namespace, so bucket creation must also be handled carefully.

Riak CS introduced a service named Stanchion that is designed to handle these specific requests to avoid conflicts. Stanchion is a single process running on a single Riak server (thus introducing a single point of failure for user account and bucket creation requests).

While it is possible to deploy Stanchion using common system tools to make a daemon process run in a highly available manner, Basho recommends doing so carefully and testing it thoroughly. Since the only impact of failure is to prevent user and bucket creation, it may be preferable to monitor and alert on failure. If two copies of Stanchion are running due to a network partition, its strong consistency guarantees will be lost.

With strong consistency options targeted for Riak 2.0, expect to see some changes.

Other Differences

Replication

Basho offers multi-datacenter replication with its Enterprise software licenses, and Riak CS Enterprise takes full advantage of that feature. Data can be written to one or more clusters in multiple data centers and be synchronized automatically between them.

There are two types of synchronization: real-time, which occurs as objects are written, and full sync, which happens on a periodic basis to compare the full contents of each cluster for any changes to be merged.

One key difference is that Riak CS maintains manifest files to track the chunks it creates, and it is these manifests that are distributed between clusters during real-time sync. The individual chunks are not synchronized until a full sync replication occurs, or until someone requests the file from a remote cluster. The manifest is made active for someone to retrieve the chunks after the original upload to the source cluster is complete.

Backends

A common mistake while installing Riak CS is to configure it using information specific to Riak rather than Riak CS. As an example, per the Riak CS installation instructions the relevant backend data store must be configured to riak_cs_kv_multi_backend, which is forked from Riak’s riak_kv_multi_backend. Using the latter will cause problems.

Riak (CS) Control

Riak Control is a web management console for Riak clusters; Riak CS Control is a web management console for Riak CS user accounts. Both are optional and both are useful in a Riak CS cluster.

Exposure to Internet

Exposing any database directly to the Internet is risky. Riak, currently lacking any concept of authentication, absolutely must not be accessible to untrusted networks.

Riak CS; however, is designed with Internet access in mind. It is still advisable to place a load balancer or proxy in front of a Riak CS cluster, for example to ease cluster maintenance/upgrades and to provide a central location to log and block potentially hostile access.

Riak CS servers will still have open Riak ports that must be protected from the Internet as you would any Riak servers.

Where to Next for Riak CS?

2013 has been a big year for Riak CS: it was released as open source in the spring, with OpenStack support added this summer. Still, there is much to do.

As mentioned above, improving or replacing Stanchion is a high priority.

We will continue to expand the API coverage for Riak CS. The next major targets are the copy object operations that Amazon S3 and OpenStack Swift offer.

Compression and more granular replication controls are also under consideration for future releases.

By building Riak CS atop the most robust open source distributed database in the world, we’ve created a very operationally friendly, powerful storage solution that can evolve to meet present and future needs. Feel free to give it a try if you aren’t already using it.

If you’re interested in hearing from the engineers who’ve made this software possible (and seeing just how far a highly available data storage solution can take you), join us October 29-30th for RICON West. RICON West is where Basho brings together industry and academia to discuss the rapidly expanding world of distributed systems, including Riak and Riak CS.

John Daily

Basho Events in September

September 4, 2013

September will be a busy month for all of us at Basho. Not only is our own RICON developer conference coming up at the end of October, but we will be traveling all over the world to attend various conferences and host meetups. Here is a look at where you can find us this month.

Open Source Conference 2013 Hokkaido: Basho Engineer, Kaz Suzuki, will provide an “Introduction to Riak and Riak CS” talk during the Open Source Conference in Hokkaido, Japan on September 14th. He will also be demoing Riak CS at our booth.

Linux Cloud Open: Basho is a proud sponsor of the 2013 Linux Cloud Open conference. If you are in New Orleans from September 16-18th, stop by our booth to learn more about our open source cloud storage software, Riak CS.

Strangeloop 2013: Basho is a sponsor of Strangeloop 2013, taking place September 18-20th in St. Louis. Garrett Eardley, Software Engineer at Riot Games, will also be speaking about how they use Riak at Riot Games.

Hosting & Cloud Transformation Summit 2013: Basho is sponsoring and speaking at the Hosting & Cloud Transformation Summit, hosted by 451 Research. Basho EVP and CMO, Bobby Patrick, will be speaking on a panel entitled “Profiting from Cloud Storage in an Era of Software-Defined Everything” on September 25th. Check out this panel or visit our booth from September 23-25th in Las Vegas.

For a full list of where we’ll be for the rest of the year, check out the Events Page.

Basho

RICON Early Bird Sale Ends This Week

August 27, 2013

If you still haven’t gotten your ticket to RICON West, make sure to grab one before the early bird sale ends on August 29th. RICON West is Basho’s distributed systems conference and will take place in San Francisco on October 29-30th.

RICON West will feature speakers that are using and researching distributed systems to solve a wide range of problems. Some highlights include:

We will be also hosting a Riak training on October 28th, right before the conference. During this training, you’ll learn about the core principles behind Riak and how it manages to scale both performance and capacity while evenly distributing data throughout the cluster. At the end of the day, you’ll be able to create and deploy your own cluster, as well as be familiar with query patterns, data modeling, and running Riak in production.

Tickets to both RICON West and the Riak Training are available here. Be sure to grab one before they sell out and see who else is attending on the RICON West Lanyrd Directory.

Basho

Riak Community Survey

July 15, 2013

Today, we are sending out our quarterly Riak Community Survey. This survey is to help us better understand how you’re using Riak. By understanding how Riak is being used, we can make more educated decisions about how to improve Riak in the future. We will also anonymize this data and share it with the community to provide a more holistic view of how Riak is being used.

To participate in this survey, simply click here to get started. All survey participants will receive Basho swag and a discount code for RICON West tickets. One lucky participant will be selected to receive a free RICON West ticket.

Thanks for participating in our survey and be sure to grab a RICON West ticket. Early bird prices end August 29th.

Basho

Turner Broadcasting System at RICON East

June 20, 2013

At RICON East, Basho’s developer conference, we had dozens of speakers discussing distributed systems in production. These speakers included academics and developers from all different industries.

Brian Akins, Senior Principal Architect at Turner Broadcasting, spoke on “Large Scale Data Service as a Service,” and discussed the challenges Turner faced serving data to millions of clients over HTTP for several large sites (including CNN.com and NBA.com).

Brian’s talk goes into the general architecture at Turner, the growing pains they went through, and why they ultimately decided on Riak. He also goes into details about how Riak is being used to power large events at Turner, such as the presidential election, CNN Breaking News videos, and March Madness. You can watch his talk below.

[youtube http://www.youtube.com/watch?v=jZ9giBXaaKY&w=640&h=390]

His slides are also available below:

For more videos from RICON East, check out the RICON East Archive. Tickets for RICON West (Oct. 29-30th in San Francisco) are also now available.

Basho

Riak Available On SoftLayer Platform

April 30, 2013

Today we are pleased to announce the availability of Riak and Riak Enterprise on SoftLayer’s global cloud platform. Users can now easily configure and deploy Riak environments on the SoftLayer platform with a flexible, pay-as-you-go service model. The solution makes it easy for organizations to quickly deploy scalable production-grade Riak systems on-demand. The partnership accelerates the speed of developing and launching applications with Riak, provides ease of operations with scale, and enables global multi-datacenter replication.

Features of the joint offering include:

  • Web-based tool to configure and deploy Riak on demand and at the click of a button
  • Pay-as-you-go model providing the flexibility of monthly or annual billing and no long-term contracts
  • Rapid deployment on dedicated, bare-metal servers for optimum performance

With the Riak Enterprise offering on SoftLayer, users can replicate data stored in Riak across SoftLayer’s global infrastructure. This multi-datacenter replication capability provides data locality, disaster recovery, global load balancing, and active backups. SoftLayer’s global private network allows for high-speed, secure replication between clusters.

The integrated solution provides the availability, fault tolerance, operational simplicity, and scalability of Riak combined with the flexibility, performance, and agility of SoftLayer’s on-demand infrastructure.

Bump is one of the most popular mobile apps on the market today, and is already using Riak on the SoftLayer platform. “Operational ease is key to our business success,” says Mark Smith, Operations Lead at Bump. “The combination of SoftLayer, who we already trust with our business and data, and Basho, who makes the database that we trust at scale, saves us time and effort and allows us to focus on our business, not our data infrastructure.”

For more information on how Bump uses Riak, check out the case study. For more information on Riak Enterprise, visit the product page or documentation.

SoftLayer is also sponsoring the RICON East 2013 after party. On night one of the conference, we’re renting out Hudson Terrace for a one-of-a-kind party. SoftLayer and Basho are furnishing drinks, food, and entertainment. All RICON attendees are automatically registered for the party…but, as of today, the party is open to anyone who wants to register.