Tag Archives: public cloud

Basho Engineer Speaking at Cloud Connect Chicago

October 16, 2013

Basho is a proud sponsor and exhibitor of Cloud Connect Chicago. The Cloud Connect conferences provide CIOs, IT professionals, and developers with the necessary tools to navigate cloud technologies and deploy cloud solutions in their organizations. Cloud Connect Chicago takes place October 21-23rd.

Basho engineer and Apache CloudStack PPMC member, John Burwell, will also be speaking on “Building Complete Private Clouds with Apache CloudStack and Riak CS” as part of the CloudStack track. During this session, he will explain how the combination of the CloudStack Infrastructure as a Service platform and the Riak CS object store will allow you to establish operational agility to drive rapid innovation and embrace commodity infrastructure without sacrificing scalability or reliability. He will also explore more general cloud system architecture principles and best practices.

While Burwell’s talk will be focused on Riak CS and CloudStack, Riak CS also offers OpenStack integration. Stop by the Basho booth to learn more about Basho’s object storage solution, Riak CS, and how it can be used in conjunction with both CloudStack and OpenStack.

For more information on how enterprisers can use private clouds for their business needs, check out Burwell’s blog post, “Learnings from Private Cloud Storage.”


Learnings from Private Cloud Storage

September 26, 2013

Big Data. eCommerce. Mobile. Suddenly, information technology has shifted from cost center to business opportunity. This opportunity favors fast movers with the ability to rapidly execute on emerging trends. Therefore, the length of traditional IT procurement cycles and provisioning processing has become a significant barrier to capitalizing on these opportunities. To increase their operational agility, some organizations are employing public infrastructure as a service (IaaS) or cloud providers (such as Amazon Web Services and Joyent) to rapidly provision compute and storage resources. However, technical incompatibilities, regulatory restrictions, cost at scale, and/or existing capital investments prevent many organizations from utilizing public cloud providers to achieve this operational agility. Private clouds allow these organizations to realize the value of public clouds with the flexibility to comply with their unique combination business and technical requirements.

Fundamentally, a cloud (public or private) creates a composable infrastructure with the following capabilities:

  • Resource Pooling: Presents compute, storage, and network resources through a unified set of vendor neutral abstractions and manages them based on service-level requirements.
  • Rapid Elasticity: Optimizes resource allocation based on performance relative to service-level requirements.
  • Self Service: Delegates management responsibilities for a subset of the infrastructure resources to end-users.
  • Metering/Charge Back: Records resource utilization on a per customer basis to support usage billing.

Private clouds implement these characteristics by orchestrating infrastructure provisioning and management through the following services:

  • Compute: Physical or virtual machines with a specified number of processing cores and RAM.
  • Block Storage: Random access, read/write persistent storage capable of supporting disk partitioning and file systems.
  • Object Storage: Write-once, read-many (WORM) oriented storage for large files (multiple gigabytes to terabytes in size) accessed through a key-value oriented interface.
  • Network: Network topology definition and connectivity management between compute, block storage, and object storage services, as well as public networks such as the Internet.

Typically, these services are exposed via an HTTP API, as well as a web-based dashboard allowing end-users to simultaneously script complex workflows and visualize their infrastructure.

Superficially, private clouds appear to be traditional virtualization infrastructures with a web interface and HTTP API. While both models share a number of common components, cloud infrastructures achieve reliability by horizontally scaling commodity hardware instead of vertically scaling specialized hardware. The following table contrasts the storage strategies employed by the traditional virtualization and cloud models:

Data Type Traditional Virtualization Cloud
Application Data VM direct attached storage (e.g. NAS, SAN, etc) Elastic database service (e.g. Riak)
Static Content VM direct attached storage Object Storage (e.g. Riak CS)
Templates VM direct attached storage Object Storage (e.g. Riak CS)
Backups VM direct attached storage Object Storage (e.g. Riak CS)

Static content, templates, and backups typically represent the majority of a system’s storage consumption. Employing object storage to manage this data brings the following benefits to private cloud infrastructures:

  • Reduced Hardware Costs: By replicating multiple copies of data across a cluster of services, object storage systems such as Riak CS guarantee data durability through software rather than hardware. This approach allows users to employ cheaper commodity hardware using ubiquitous SATA/SAS storage subsystems without sacrificing reliability.
  • Horizontal Scalability: Since storage coordination and data replication occurs in software, storage is expanded by simply adding new servers to the cluster.
  • Operational Simplicity: Accessed via HTTP/HTTPS, object storage systems provide secure access to data using a simple, ubiquitous protocol. Unlike iSCSI and Fiber Channel solutions, this approach typically has little to no impact on network infrastructure designs.

The Apache CloudStack IaaS platform has supported Swift-based object storage since version 4.0.0 and S3-based object storage since version 4.1.0. With the 4.2.0, CloudStack supports S3 and Swift as native secondary storage devices – allowing the system to provision and backup VMs directly from an object store. When coupled with Riak CS Enterprise, Apache CloudStack-based clouds are able to replicate template and snapshot data across multi-data centers to meet off-site backup and disaster recovery requirements.

The OpenStack Object Storage API specifies the semantics of OpenStack’s object storage service. The Swift implementation of this API is provided as the default implementation of this API. With the 1.4.0 release, Riak CS implements both the OpenStack Object Storage API allowing it to serve as a drop-in Swift replacement.

As organizations work to understand the opportunities created by information technology, private clouds have emerged as a key component of their strategies to increase operational agility. While private clouds can be constructed using traditional virtualization approaches, such designs will simply mask core infrastructure brittleness and high infrastructure costs. By embracing design principles such as object storage that underpin cloud infrastructure platforms, organizations can realize the promise of increased operational agility and cost savings.

John Burwell

Riak on AWS – Technical Guidance

June 27, 2013

Today, we are excited to share a recent whitepaper released by the Amazon team entitled, “NoSQL Database in the Cloud: Riak on AWS.” This paper provides technical guidance on running Riak on the Amazon platform, including an overview of:

  • Basic Installation
  • Riak Architecture and Scale
  • Operational Considerations (including sizing and configuration)
  • AWS specific security configuration
  • A discussion of Replication (as enabled by Riak Enterprise)

Given the number of Riak users (both open source and enterprise) who leverage public cloud environments, either as a part of their infrastructure or as the foundation of it, Basho will continue to invest in partnerships that provide deployment choice and deployment ease. Whether it’s for a hybrid cloud model – used to address burst capacity, tenancy/data locality, and proof of concept needs – or for an investment solely in public cloud, Riak will provide the operational simplicity and scalability required for your critical data.

For more information about deploying Riak on AWS, check out our posts about the Riak AMI and our other deployment options, including automated scripts and manual installation. You can also find more information about what to consider when installing Riak on AWS in our documentation.


Use Cases for Riak CS

April 3, 2013

As you might have heard, we recently open sourced Riak CS, cloud storage built on Riak. You can find all of the code on our GitHub account and download Riak CS here. To help you get started with Riak CS, here are some common use cases.

  • Large Object Storage For Applications and Services: Riak CS is built for storing large objects of all types. It is content agnostic so you can store images, text, video, documents, database backups, software binaries, or other data types. Riak CS can store objects into the terabyte size range using the new multipart upload feature. When an object is uploaded, Riak CS breaks it into smaller blocks that are streamed, stored, replicated in the underlying Riak cluster.
  • On-Demand Internal Storage Capacity: Riak CS provides highly available storage for internal business units. Built on Riak, Riak CS has a masterless, redundant design that ensures availability and fault-tolerance. Use cases might include document storage or backing for internal applications.
  • Storage Layer for Public Clouds/Cloud Services: Riak CS’ flexibility and scalability provide the ideal foundation for building public clouds or cloud services. Capacity can be added by installing Riak CS on a new physical node and joining it with the cluster. Riak automatically redistributes data and ownership so all nodes have equal responsibility, which prevents storage hot spots and decreases the operational burden of adding new nodes. Additionally, Riak CS is multi-tenant, a requirement of most public cloud services today.
  • Amazon S3 Compatibility: Riak CS is S3-compatible, making it easy for your developers to be productive quickly. Riak CS can be used with existing S3 clients and libraries. The HTTP REST API supports service, bucket, and object-level operations to easily store and retrieve data. Riak CS makes sense for companies that are trying to provide internal, S3-like services or using a hybrid approach with some public and some private cloud storage.
  • Disaster Recovery and Active Backups: Riak CS Enterprise extends Riak CS with multi-datacenter replication. By replicating data across datacenters using either real-time or full-sync, you can maintain redundant storage in case of disaster scenarios. Multi-datacenter replication can also be used to maintain active backups or create availability zones.

For more information about Riak CS, visit our site and download the technical overview.


Multi-Datacenter Replication: Availability Zones and Public Cloud

February 28, 2013

In the last post, we looked at how Riak Enterprise’s multi-datacenter replication can be configured for backups and data locality. In this post, we examine two other common implementations: availability zones and public cloud use cases. For more information on Riak Enterprise architecture and configuration, download the complete whitepaper.

Availability Zones

Availability zones provide efficient multi-datacenter replication and data redundancy within a geographic region (such as a coast or a country). In this configuration, data is replicated within an availability zone’s series of datacenters. In the event that one of datacenters experiences an outage or serious failure, data can still be served from other datacenters within the same region.

One approach to setting this up is to have a “primary” site in a region where all reads and writes for specific users, applications, or data sets are directed. This primary cluster can then be replicated to one or more proximal secondary clusters. In other approaches, data can be replicated in real-time from one cluster to both another datacenter and other cold backups maintained for emergency conditions. The right approach is highly dependent on the requirements of users, availability, expense of bandwidth, and other constraints.

Public Cloud Use Cases

Riak is designed to be easy to use and operate on public clouds, and is partnered with many of the leading cloud providers, including Amazon Web Services, Microsoft Azure, and Joyent. Hosted Riak is also available from Engine Yard and Riak packages can always be manually installed on any physical or virtual provider, even if a machine image isn’t explicitly supported.

There are several use cases for Riak Enterprise’s multi-datacenter replication in the public cloud. Many enterprises want to maintain a cold or hot backup of their cluster in a public cloud for business continuity in the event of a datacenter outage in their private infrastructure. For other customers, the public cloud can provide a more cost-effective way of meeting peak loads, rather than building out private infrastructure to accommodate them year-round. For example, many retailers and media providers need to offer increased capacity over the holiday season. Riak Enterprise is used to scale out capacity on public clouds over these periods, either with full-sync or real-time sync depending on the business needs.

Finally, some enterprises run certain applications or services entirely on public clouds. Riak Enterprise allows for redundancy and data locality across public cloud availability zones for this use case, ensuring optimal performance and resiliency.

For a more in-depth look at common architectures and use cases for Riak Enterprise, download our technical overview. You can also sign up for our webcast on Thursday, March 7th.