Yesterday it was announced that Apple has acquired FoundationDB. As you may imagine, I have been asked to comment on what this means for the NoSQL database industry and for those who are investing heavily in retooling their traditional database infrastructures with new technologies to meet the availability, scalability, and fault-tolerance characteristics required by the massive influx of data.
NoSQL databases are an increasingly critical part of enterprises’ ability to derive real business value from the massive amounts of data that users, devices and online systems generate. They are also an important part of the developers’ toolkits when building applications for the Internet of Things, a major contributor to this ever-growing body of data. Apple is acutely aware of the importance of being able to reliably scale to meet the real-time data needs of today’s global applications. The news of Apple’s intent to acquire FoundationDB greatly amplifies these points to a growing number of IT and engineering leaders.
Part of the comments around the announcement are the discussion of Open Source software both as an underpinning for enterprise infrastructure and as a viable business model. I contributed to a detailed discussion about the latter in a recent article on Silicon Angle entitled NoSQL market frames larger debate: Can open source be profitable?, noting that there is enormous opportunity for Open Source NoSQL companies if they can serve the specific needs of enterprise customers. We feel that we are doing so, and that our approximately 1:10 ratio of paying customers to Open Source users is an indicator of our solution’s value and the strength of our business. Our clear path to being cash-flow-positive includes a measured, strategic investment in R&D which is essential to ensuring Basho’s corporate viability for all customers who have, already, made multi-million dollar investments in their business critical workloads.
Unlike others, the core underpinnings of Riak as a distributed, multi-model data persistence platform are, and will remain, Open Source. Basho builds premium, enterprise-grade features atop this distributed infrastructure, and these features help us attract a higher percentage of paying customers than others in the industry.
Acquisition and consolidation — whether done to enhance technical capabilities, secure talent, or expand a company’s customer base — are essential to the high technology arena. The NoSQL space will be the focus of more of this activity than most in the coming year, given the amount of attention it has already received, with PwC naming NoSQL as one of the “surprising digital bets for 2015” and given the success of the HortonWorks IPO. Combine that buzz with the fact that a prominent database ranking tool lists more than 200 different database management systems, and we are certain to see more industry consolidation.
The decision to re-architect an existing enterprise data workload infrastructure is not one to be taken lightly. Basho’s commitment to Open Source, our commitment to long-term business viability, and our impressive list of customers making substantial investments, point to a bright future not only for our company but for those who choose Riak as a core underpinning of their persistence infrastructure. Apple’s acquisition of FoundationDB strongly validates the value of the solutions we offer and underscores the criticality of these technologies to companies that need to scale business-critical applications.
December 23, 2014
This is a continuation in our blog series covering Riak 2.0. Below are links to the previously covered blogs and we continue with a discussion on Values, Keys, Buckets and Bucket Types.
- Riak 2.0 – New Capabilities, New Use Cases, Available for Download
- Write it like Riak; Query it Like Solr
- Riak Security 2.0: Not Just a Firewall Anymore
- Distributed Data Types – Riak 2.0
- Strong Consistency in Riak 2.0
If you have tested Riak 2.0, or used it in production, you will also be aware of a new feature called Riak Bucket Types
The Using Bucket Types documentation covers the implementation, usage, and configuration of Bucket Types in great detail. Throughout the documentation there are code samples (e.g. Using Data Types) including code for creating the bucket types associated with each individual Riak Data Types.
Bucket types are a major improvement over the older system of bucket configuration. The ability to define a bucket configuration, and then change the configuration if necessary, for entire group of buckets, is a powerful new way to consider data modeling. In addition, bucket types are more reliable as buckets that have a given type (or configuration) only have their properties change when the type is changed. Previously, it was possible to change the properties of a bucket only through client requests.
In prior versions of Riak, bucket properties were altered by clients interacting with Riak…in contrast, bucket types are an operational concept. The
riak-admin bucket-type interface enables Riak users to manage bucket configurations at an operational level, without recourse to the Riak clients.
In versions of Riak prior to 2.0, all queries were made to a bucket/key pair as in the following example:
Now in Riak 2.0 with the addition of bucket types, there is an additional namespace on top of buckets and keys. The same bucket name can be associated with completely different data if it is used in accordance with a different bucket type.
If a request is made to a bucket/key pair without a specified bucket type,
default will be used in place of a bucket type. The following request are identical.
Bucket types allow groups of buckets to share configuration details. This allows Riak users, and administrators, to manage bucket properties more efficiently than in the older configuration systems that were based on bucket properties.
For a broader discussion of application data modeling, and using Riak as a data store, the Building Applications with Riak documentation covers these concepts, and more, in great detail.
Riak 2.0 was the culmination of substantial effort by the Basho team. A particular focus was adding functionality (like Riak Search) that had been requested by customer. In addition, this release further the Basho commitment to providing a scalable and available database that has the operational simplicity for which Riak is known.
It just works.
April 28, 2014
On Monday, May 5th, Basho is co-hosting an open source happy hour and networking meetup. Partnering with GoGrid, we will bring together some of the greatest minds in open source to discuss current trends and where open source will be in the future. Drinks and appetizers will be provided. This meetup will take place from 6pm-8pm on May 5th at the 111 Minna Gallery in San Francisco. Space is limited, so be sure and RSVP quickly.
This meetup is also in conjunction with the Open Business Conference. Open Source has become ubiquitous in the enterprise and in the business layer, as more and more organizations are reaping its considerable benefits, including speed, efficiency and cost savings. Open Business Conference 2014 brings together the people who are building and deploying the latest in enabling technologies and solutions and teaches businesses how to put their data to use. Basho will be available to answer any questions about Riak and open source at the GoGrid booth. Open Business Conference takes place May 5-6 at the Palace Hotel in San Francisco.
To learn more about Basho’s partnership with GoGrid (and other partnerships), visit our Partners Page.
December 30, 2013
2013 was a huge year for Basho Technologies and before we dive into 2014, we thought we’d take a moment to reflect on how far we’ve come.
2013 was the year of the Riak User. We love hearing about all the amazing ways companies across various industries are using Riak. This year, we were able to share dozens of exciting case studies. These include:
- Synacor’s TV Everywhere platform
- Enstratius (acquired by Dell)
- Best Buy
- Alert Logic
- Viggle (through OmniTI)
- Turner Broadcasting
- Hosted Graphite
- Gilt Groupe
- Praekelt Foundation
- National Health Service
- City Maps
- The Weather Company
For even more Riak Users, check out the Users Page.
We released Riak 1.3, Riak 1.4, and the Technical Preview of Riak 2.0 this year. These releases added such features as Active Anti-Entropy, revamped Riak Control, queryability improvements, Riak Data Types, and much more. Be on the lookout for the general release of Riak 2.0 early next year.
This year, we expanded RICON, Basho’s distributed systems conference, to both RICON East and RICON West. These were both sold out conferences that featured speakers from bitly, Comcast, Google, Netflix, Salesforce, The Weather Company, Turner Broadcasting, Twitter, and many more.
We drastically increased the number of Basho partners in 2013. For a full list of partners, check out the Partnerships Page. Some key ones to note include Tokyo Electron Device, SoftLayer, and Seagate.
Our amazing community team hosted over 200 meetups around the world this year. On top of that, they also attended dozens of industry events to spread the word about Basho. Keep an eye on the Events Page to see where we’ll be in 2014.
2013 was a busy year but, with some exciting announcements coming, we look forward to an even busier 2014. Happy New Year!
July 31, 2013
In his talk, he discusses the data structures and message patterns that are used to craft a distributed system à la Riak. To create a system like Riak is more than asking which programming language to use, or which libraries to include. He explains how a better approach is if you take a collection of well-known patterns and put them together in the right way for the right reasons, you can distribute all sorts of tasks across multiple nodes.
The slides from his talk can be found here.
Interested in more Basho talks? You can see where we’ll be presenting next on our Events Page.
July 23, 2013
This week is O’Reilly OSCON, a conference dedicated to all things open source. Basho is a sponsor and Basho engineer, Eric Redmond, will be delivering a presentation entitled “Distributed Patterns In Action“.
Basho first open sourced Riak in 2009. It’s a decision that helped us grow our business, and become a leader in newer, agile enterprise environments. Our participation in the open source community benefits our culture, our development process, and our business.
In honor of OSCON, we thought it important to explore the commercial aspects of our open source decision.
The Business of Open Source
Open source is in the DNA of our company, with both Riak and Riak CS available under the Apache 2 license. (It is worth noting that these products are but a few of our open source contributions, which also include Webmachine and Lager.) To turn this great code into a business, we chose to stay true to our roots as a software company, instead of just selling services. The enterprise versions of Riak and Riak CS offer the entirety of our open source software, with the addition of multi-datacenter replication and monitoring capabilities.
The decision to sell licenses to the enterprise, rather than to rely just on services, makes Basho unique. It allows us to engage with our enterprise customers in the transformation of their application architecture. They can be confident in the software’s availability and in Basho’s commitments to support them – as customers. Enterprises need an alternative to traditional database vendors, but one that can still fit — in license structure, operational management, and process integration — into a traditional organization.
Our licensing model for Riak Enterprise and Riak CS Enterprise lets us balance agility with tradition. Our community helps us develop groundbreaking software, while the enterprise license helps corporate IT and Operations sleep at night.
Open source drives adoption (a concept discussed at length in Stephen O’Grady’s book The New Kingmakers). That means Riak is used across many different industries, powering thousands of applications. That commercial validation — our success in production deployments — is accelerated due to the open source availability.
We remain keenly aware, and tremendously appreciative, that our community (from the individuals to the large organizations) guides Riak and Riak CS updates, and has been crucial to the refinement and forward momentum of this software.
Basho’s success is open source’s success. Our strengths reside both in our team and in our community, as their combined efforts improve our technology and its utilization. We are excited to see what other open source showcases are in view at OSCON 2013.
Portland, OR – July 22, 2013 – Basho is a proud sponsor of O’Reilly OSCON, a conference focused on the entire open source ecosystem. In addition to sponsoring, Basho software engineer, Eric Redmond, will also be speaking at OSCON.
Redmond’s talk, “Distributed Patterns in Action,” will go through a set of distributed data structures and discuss how each works and when should be used. Redmond is a longtime advocate of alternative storage mechanisms (especially the distributed kind) and has co-authored the books, “Seven Databases in Seven Weeks” and “A Little Riak Book.” His talk begins at 11:30am on Thursday, July 25th.
In its fifteenth year, OSCON is known as the primary gathering place for the open source community – bringing together developers, innovators, businesspeople, and investors. It works to bridge partnerships between businesses and the open source community, while also unveiling some of the most groundbreaking open source products and projects. OSCON takes place July 22-26 in Portland, OR.
May 6, 2013
The free Riak AMI available on the AWS Marketplace has been updated to the latest version, Riak 1.3.1.
In Riak 1.3, we introduced:
- Active Anti-Entropy
- Updates to Riak Control
- Expanded IPv6 support
- Improved MapReduce
- Simplified Log Management
Riak 1.3.1 includes all these features with some additional changes enumerated in the release notes.
For those of you currently using Riak on AWS, or interested in testing Riak on AWS, the AMI makes installation and configuration much easier. We see open source and Riak Enterprise users leverage AWS both as their primary infrastructure and to support hybrid implementations.
Installation instructions for the AMI are available on in our docs.
May 1, 2013
This post looks at five commonly asked questions about Riak CS – simple, available, open source storage built on top of Riak. For more information, please review our full documentation, or sign up for an intro to Riak CS webcast on Friday, May 10.
What is the relationship between Riak and Riak CS?
Riak CS is built on top of Riak, exposing higher-level storage functions including large object support, an S3-compatible API, multi-tenancy, and per-user storage and access statistics. Riak itself provides the replication, availability, fault-tolerance, and underlying storage functions for the Riak CS implementation. Riak and Riak CS should both be installed on every node in your cluster. While Riak and Riak CS could be run on separate virtual or physical nodes, running them on the same machine minimizes intra-cluster bandwidth usage and is the recommended approach. As with Riak, we advise a minimum 5-node cluster.
When objects are uploaded to Riak CS, the object is broken up into smaller chunks which are then streamed, stored, and replicated in the underlying cluster. A manifest is maintained for each object, that points to which blocks comprise the object, and is used to retrieve all blocks and present them to the client on read. In addition to running Riak and Riak CS on each node, Stanchion, a request serializer, must be installed on at least one node in the cluster. This ensures that global entities, such as users and buckets, are unique in the system.
What use cases does Riak CS support that Riak doesn’t?
Riak CS has several features that are not provided in the standalone Riak database. One of the most obvious differences is in the size of objects supported. Riak CS exposes large object support, and includes multi-part upload so you can upload objects as a series of parts. This allows you to upload single objects to the system into the terabyte range. In Riak, the data model is simply key/value; in Riak CS, the key/value model provides the underlying structure for higher-level storage semantics – users, buckets and objects. The Riak CS interface is an S3-compatible HTTP API, allowing you to use existing S3 libraries and tools. In contrast, Riak exposes an HTTP and protobufs API and offers many language-specific clients. Unlike Riak, Riak CS is multi-tenant, with the concept of “users” and per-user reporting on storage and access. This makes it a fit for both private cloud scenarios, with multiple internal users, or as a foundation for a public cloud storage offering.
How does multi-tenancy, authentication and reporting work?
Riak CS exposes an interface for user creation, disablement and credential management. Riak CS can be set so that only administrators can create new users. Administrators also have special privileges including being able to retrieve a list of all users in the system and query the user account information of any user. Once issued credentials, users are able to authenticate, create buckets, upload and download files, retrieve account information, obtain new credentials, or disable their account through the API. Riak CS supports the standard S3 authentication scheme, with support for header and query string authorization.
Riak CS exposes storage, usage and network statistics that support use cases like accounting, subscription, billing or multi-group utilization for public or private clouds. Riak CS will report information on how much storage a user is consuming and the network operations related to access. This data is exposed via an HTTP interface and can be queried on the default timespan “now” or as a range from start time through end time. Access statistics are reported as bytes in and bytes out for both object and bucket operations. Reporting of this information can be scheduled for a set interval or manually triggered.
What’s the difference between Riak CS and Riak CS Enterprise?
Riak CS Enterprise provides multi-datacenter replication on top of Riak CS. For multi-datacenter replication in Riak CS, global information for users, bucket information and manifests are streamed in real-time from a primary implementation to a secondary site so global state is maintained across locations. Objects can then be replicated in either full sync or real-time sync mode. The secondary site will replicate the object as in normal operations. Additional datacenters can be added in order to create availability zones or provide additional data redundancy and locality. Riak CS Enterprise can also be configured for bi-directional replication. Riak CS Enterprise also comes with 24/7, enterprise-level support. More information and pricing can be found here, and full technical information is available on our docs portal. Ready to get started? Sign up for a developer trial of Riak CS Enterprise.
What are your plans for integration of Riak CS with open source compute solutions?
Riak CS provides highly available, distributed storage, making it a natural fit for usage alongside compute solutions. We have partnered with Citrix to collaborate on the integration of Apache CloudStack and Riak CS to create a complete cloud software offering that combines compute and storage in an integrated platform. For more information on our partnership with CloudStack, check out this blog post with the latest update. API and authentication support for OpenStack is also in progress.
April 3, 2013
As you might have heard, we recently open sourced Riak CS, cloud storage built on Riak. You can find all of the code on our GitHub account and download Riak CS here. To help you get started with Riak CS, here are some common use cases.
- Large Object Storage For Applications and Services: Riak CS is built for storing large objects of all types. It is content agnostic so you can store images, text, video, documents, database backups, software binaries, or other data types. Riak CS can store objects into the terabyte size range using the new multipart upload feature. When an object is uploaded, Riak CS breaks it into smaller blocks that are streamed, stored, replicated in the underlying Riak cluster.
- On-Demand Internal Storage Capacity: Riak CS provides highly available storage for internal business units. Built on Riak, Riak CS has a masterless, redundant design that ensures availability and fault-tolerance. Use cases might include document storage or backing for internal applications.
- Storage Layer for Public Clouds/Cloud Services: Riak CS’ flexibility and scalability provide the ideal foundation for building public clouds or cloud services. Capacity can be added by installing Riak CS on a new physical node and joining it with the cluster. Riak automatically redistributes data and ownership so all nodes have equal responsibility, which prevents storage hot spots and decreases the operational burden of adding new nodes. Additionally, Riak CS is multi-tenant, a requirement of most public cloud services today.
- Amazon S3 Compatibility: Riak CS is S3-compatible, making it easy for your developers to be productive quickly. Riak CS can be used with existing S3 clients and libraries. The HTTP REST API supports service, bucket, and object-level operations to easily store and retrieve data. Riak CS makes sense for companies that are trying to provide internal, S3-like services or using a hybrid approach with some public and some private cloud storage.
- Disaster Recovery and Active Backups: Riak CS Enterprise extends Riak CS with multi-datacenter replication. By replicating data across datacenters using either real-time or full-sync, you can maintain redundant storage in case of disaster scenarios. Multi-datacenter replication can also be used to maintain active backups or create availability zones.