By: Peter Coppola
We had the opportunity to stop by DATAVERSITY’S NoSQL Now! conference in San Jose last week. I was very impressed with the level of energy and the wide-ranging selection of sessions offered. According to Tony Shaw, the CEO of DATAVERSITY, the organizer of NoSQL Now, registrations were up 15 percent from 2013.
The exhibition hall was packed and lively as attendees jostled between booths. DATAVERSITY did an outstanding job keeping the show floor tightly packed with exhibitors. The industry was well represented by Cloudera (saw “Data is the new bacon” t-shirts), MarkLogic, MongoDB, Oracle and EnterpriseDB – all present as major sponsors. Between conversations, I was able to nab a nifty versatile screw-driver disguised as a pen from DataStax.
NoSQL Now sessions do rely heavily on sponsors, but with such a wide selection of tracks there’s bound to be a topic of interest at any given time slot. I had a choice of the following concurrent sessions at 4:15 p.m. on Wednesday:
- Internet of Things with MongoDB – MongoDB
- Out with MapReduce, In with Spark – DataStax
- Case Studies in Search and Semantics – MarkLogic
- Just the Right Weather for our Company: How We Chose Our Data Stores – The Weather Company
- NoSQL on ACID – EnterpriseDB
I attended The Weather Company’s session – not only was it the only non-vendor presenter, but the company is also a customer and big fan of Riak. The Weather Company manages five data centers that in production handle 25,000 requests per second and distribute 60 GB of data to each data center every 10 minutes. Surya Kangeyan Sivakumar took us through the journey of how The Weather Company selected its data store solutions and how it overcame the mindset of having to use its existing relational database solution just because the company had invested so much in it. Riak was selected, along with other NoSQL solutions, due to the speed and ease at which it could be stood up.
In 2015 Basho looks forward to being a more active participant in NoSQL Now.
April 23, 2014
Traditional database architectures were the default option for many pre-Internet use cases and architectures, such as MySQL, remain common today. However, these traditional solutions have limits that quickly become apparent as companies (and data) grow. Modern companies have changing priorities: downtime (planned or unplanned) is never acceptable; customers require a fast and unified experience; and data of all types is growing at unimaginable rates. Solutions such as Riak are designed to handle these shifting priorities.
Top Reasons to Move to Riak
- Zero Downtime: Distributed NoSQL solutions like Riak are designed for always-on availability. This means data is always read/write accessible and the system never goes down. Downtime, planned or unplanned, can make or break a customer experience.
- Ease-of-Scale: Traffic can be unpredictable. Businesses need to scale up quickly to handle peak loads during holidays or major releases, but then need to scale back down to save money. Riak makes it easy to add and remove any number of nodes as needed and automatically redistributes data across the cluster. Scaling up or down never needs to be a burden again.
- Flexible Data Model: From user generated data to machine-to-machine (M2M) activity, unstructured data is now commonplace. Riak can store any type of data easily with its simple key/value architecture.
- Global Data Locality: Every company is a global company and needs to provide consistent, low-latency experiences to everyone, regardless of physical location. Riak’s multi-datacenter replication makes it easy to set up datacenters wherever users are, for both geo-data locality and maintaining active backups.
Users That Switched to Riak
Many top companies have already moved from relational architectures to Riak. Here’s a look at a few that have made the switch.
Bump (acquired by Google)
Bump, acquired by Google in 2013, allows users to share contact information and photos by bumping two phones together. Bump uses Riak to store almost all of its user data: contacts, communications sent and received, handset information, social network OAuth tokens, etc. Bump moved from MySQL to Riak due to its operational qualities: “No longer will we have to do any master/slave song and dance, nor will we fret about performance, capacity, or scalability; if we need more, we’ll just add nodes to the cluster.” Learn more about their move in their case study.
Alert Logic helps companies defend against security threats and address compliance mandates, such as PCI and HIPAA. Alert Logic switched from MySQl to Riak to collect and process machine data and to perform real-time analytics, detect anomalies, ensure compliance, and proactively respond to threats. Alert Logic processes nearly 5TB/day in Riak and has achieved performance results of up to 35k operations/second. Learn more about how Alert Logic improved performance through Riak in our blog post.
The Weather Company
The Weather Company provides millions of people every day with the world’s best weather forecasts, content and data, connecting with them through television, online, mobile and tablet screens. Riak is central to The Weather Company’s weather data services platform that delivers real-time weather services to aerospace, insurance, energy, retail, media, government, and hospitality industries. Check out our blog to see why The Weather Company selected Riak over MySQL to support their massive big data needs.
Dell uses Riak as the core distributed database technology underlying its customer cloud management solutions. Riak is used to collect and manage data associated with customer application provisioning and scaling, application configuration management, usage governance, and cloud utilization monitoring. In 2012, Enstratius (acquired by Dell) switched to Riak from MySQL in order to provide cross-datacenter redundancy, high write availability, and fault tolerance. Check out the full Enstratius case study.
Data Modeling in Riak
Riak has a “schemaless” design. Objects are comprised of key/value pairs, which are stored in flat namespaces called buckets. Below is a chart with some simple approaches to building common application types with a key/value model.
|Session||User/Session ID||Session Data|
|Advertising||Campaign ID||Ad Content|
|Sensor||Date, Date/Time||Sensor Updates|
|User Data||Login, eMail, UUID||User Attributes|
|Content||Title, Integer||Text, JSON/XML/HTML Document, Images, etc.|
December 30, 2013
2013 was a huge year for Basho Technologies and before we dive into 2014, we thought we’d take a moment to reflect on how far we’ve come.
2013 was the year of the Riak User. We love hearing about all the amazing ways companies across various industries are using Riak. This year, we were able to share dozens of exciting case studies. These include:
- Synacor’s TV Everywhere platform
- Enstratius (acquired by Dell)
- Best Buy
- Alert Logic
- Viggle (through OmniTI)
- Turner Broadcasting
- Hosted Graphite
- Gilt Groupe
- Praekelt Foundation
- National Health Service
- City Maps
- The Weather Company
For even more Riak Users, check out the Users Page.
We released Riak 1.3, Riak 1.4, and the Technical Preview of Riak 2.0 this year. These releases added such features as Active Anti-Entropy, revamped Riak Control, queryability improvements, Riak Data Types, and much more. Be on the lookout for the general release of Riak 2.0 early next year.
This year, we expanded RICON, Basho’s distributed systems conference, to both RICON East and RICON West. These were both sold out conferences that featured speakers from bitly, Comcast, Google, Netflix, Salesforce, The Weather Company, Turner Broadcasting, Twitter, and many more.
We drastically increased the number of Basho partners in 2013. For a full list of partners, check out the Partnerships Page. Some key ones to note include Tokyo Electron Device, SoftLayer, and Seagate.
Our amazing community team hosted over 200 meetups around the world this year. On top of that, they also attended dozens of industry events to spread the word about Basho. Keep an eye on the Events Page to see where we’ll be in 2014.
2013 was a busy year but, with some exciting announcements coming, we look forward to an even busier 2014. Happy New Year!
December 3, 2013
The videos from RICON West 2013 are now available. RICON West brought together hundreds of developers to discuss distributed systems. It also featured 26 speakers over two days who spoke about new features in Riak, Riak in production, distributed systems in production, and research related to distributed systems.
In particular, there were some interesting talks related to how major companies are using Riak. James Hughes, Principal Technologist at Seagate Technology spoke on “The Seagate Kinetic Open Storage Platform: Innovation to Enable Scale Out Storage.” The week prior to RICON West, Seagate announced their Kinetic Open Storage Platform that utilized Riak as the backend. On Day One of RICON West, James Hughes did the first public demo of this platform during his session. The platform leverages Seagate’s expertise in hardware and software storage systems, integrating an open source API and Ethernet connectivity with Seagate hard drive technology. Check out his full talk below for more information on the Kinetic Open Storage Platform, how it utilizes Riak, and the live demo.
RICON West also featured two speakers from The Weather Company, Raja Selvaraj (Manager of Data Systems Engineering) and Arvinda Gillella (SUN Architect). Their talk, “Building Next Generation Weather Data Distribution and On-demand Forecast Systems Using Riak,” describes The Weather Company’s need to effectively store and distributed terabytes of data, while still maintaining manageability, scalability and usability. Their talk focuses on the challenges they face trying to solve this problem and why they ultimately chose Riak as its solution. It also discusses their new IT platform, which uses Riak to build the next generation on-demand forecasting system. The full talk is available below:
To watch all of the sessions from RICON West 2013, visit the Basho Technologies Youtube Channel.
December 2, 2013
A few weeks ago, at AWS re:Invent 2013, The Weather Company announced their new IT platform, which focuses on next generation forecasting using big data. To build this platform, they required an architecture that was both flexible and reliable, and selected Riak to achieve that. Riak underpins the new IT platform and is used to store a variety of data from satellites, radars, forecast models, users, and weather stations worldwide.
At re:Invent, the Vice President of Enterprise Data at the Weather Company, Sathish Gaddipati, spoke about The Weather Company’s overall IT transformation in a talk titled, “How the Weather Company Monetizes Weather, the Original Big Data Problem.” If you missed Sathish’s session at re:Invent, the entire talk is now available on the AWS Youtube Channel and can be watched below.
This talk provides more details on The Weather Company’s architecture, technology choices, performance results, and business benefits realized from this IT transformation. It also discusses how the application of these technologies can help keep people safe and help businesses plan and become more profitable, thanks to the latest intersection of consumer behavior and weather forecasting and reporting.
For additional details on why The Weather Company selected Riak over Cassandra, MongoDB, and Hadoop, check out Doug Henschen’s InformationWeek article “Big Data Reshapes Weather Channel Predictions.”
November 25, 2013
RICON is Basho’s distributed systems conference series for developers. Last month, RICON West took place at the St. Regis in San Francisco and brought together hundreds of engineers, developers, scientists, and architects to discuss the theory, practice, and importance of running distributed systems in production. It was a packed two-days with 26 talks from Basho, Google, Netflix, Seagate, The Weather Company, Twitter, and many others.
- Lindsey Kuper
- Miles O’Connell
- Eric Redmond
- James Hughes
- Justin Shoffstall and Charlie Voiselle
- Jeff Hodges
- Peter Bailis
- Ryland Degnan
- Joseph Blomstedt
- Derek Murray
All talks from Day Two will be added to the playlist over the next few weeks. Be sure to subscribe to Youtube.com/BashoTechnologies for notifications about new RICON West videos and other Basho content.
The Weather Company VP of Enterprise Data to Unveil New IT Platform at Amazon AWS re:Invent 2013
LAS VEGAS, CA – November 13, 2013 – Amazon AWS re:Invent 2013 – During the past year, The Weather Company has undergone a remarkable transformation to its technology infrastructure as a data-driven media and technology company. As a part of the transformation, The Weather Company, which oversees popular brands such as The Weather Channel, weather.com, Weather Underground, Intellicast and WSI, has selected next generation technologies to underpin its new IT platform, including Basho’s Riak, Apache Hadoop, and Dasein.
“Weather is the original big data. However, weather changes, and so must IT,” commented Bryson Koehler, executive vice president and CIO, The Weather Company. “A massive data explosion is at the center of our growth strategy. The Weather Company requires an architecture that is both flexible and reliable, allowing us to deliver higher accuracy through superior data. Basho has been a valuable partner in our transformation and Riak has proven to be a critical component as the NoSQL distributed database powering our new platform.”
Since joining The Weather Company in July 2012 as EVP and CIO, Bryson Koehler has renewed the company’s vision for technology and executed a global IT transformation to propel their business growth strategy.
The Weather Company deployed Riak across multiple Amazon Web Services (AWS) availability zones using Riak Enterprise’s Multi-Datacenter Replication for ultra high-availability. Riak is used to store a variety of data from satellites, radars, forecast models, users, and weather stations worldwide. The next-generation data services platform enables The Weather Company to expand its ability to serve superior professional weather services in major weather-influenced markets, including media, hospitality, insurance, energy, and retail.
“The Weather Company typifies the customer that Riak was designed for,” said Greg Collins, CEO of Basho Technologies. “Aligning data services with customer consumption requires a flexible, available, and operationally simple database architecture. Riak Enterprise solves these needs while also working with the The Weather Company to ensure it is able to scale to meet the demand of an increasing set of data points and be well positioned for future initiatives.”
During Amazon re:Invent 2013, Sathish Gaddipati, vice president of enterprise data at The Weather Company will unveil their newest Weather Forecast and Data Services Platform and TWC’s overall enterprise IT transformation. Sathish’s presentation, titled How the Weather Company Monetizes Weather, the Original Big Data Problem, will be Thursday, November 14 at 5:30pm at The Venetian Las Vegas.
November 7, 2013
AWS re:Invent 2013 is the second annual Amazon Web Services’ global customer and partner conference. It is the largest gathering of developers and technical leaders from the AWS community and provides four days of sessions, training bootcamps, hands-on labs, and a Hackathon.
Sathish Gaddipati, The Weather Company VP of Data Enterprise, will be presenting at re:Invent on Thursday, November 14th at 5:30pm. His talk, “How The Weather Company Monetizes Weather, the Original Big Data Challenge,” will look at the transformation of the most widely distributed cable TV network in the United States (building on one of the world’s most visited digital properties) to create a world class Big Data platform. As part of this transformation, The Weather Company has selected Basho’s Riak to underpin their new IT platform. Gaddipati will also discuss the architecture, technology choices (including how and why they selected Riak), performance results, and business benefits realized as part of their use of AWS services to host an exciting set of weather.com solutions and generate new revenue streams.
Basho is also a gold sponsor of re:Invent and will have a booth set up to answer any questions about Riak or Riak CS. Be sure and stop by to chat and grab some great Basho swag.
November 6, 2013
RICON West may be over, but we still have a few major conferences and events to wrap up before the year is up. Here are some of the highlights of where we’ll be in November.
OpenStack Summit: The OpenStack Summit will bring together key technical minds to discuss the future of cloud computing. This Summit will feature case studies, workshops, and technical sessions for both cloud operators and developers. Basho engineer, Eric Redmond, will be on-site to answer any questions about Riak CS and OpenStack. The OpenStack Summit will take place in Hong Kong from November 5-8th.
Build a Cloud Days: Build a Cloud Days helps attendees deploy a cloud computing environment using CloudStack and other cloud infrastructure tools. Basho consulting engineer, John Burwell, will present on private cloud design principles and best practices. It will take place on November 8th in Washington DC.
QCon: Basho is a proud sponsor of QCon San Francisco, which will take place November 11-13th. Basho technical evangelist, Tom Santero, will discuss the benefits and challenges of working on distributed teams in his talk, “Making Virtual and Remote Teams Shine.” Basho will also have a booth set up to answer any questions about Riak or Riak CS.
AWS re:Invent: Basho is a gold sponsor of AWS re:Invent, the largest gathering of developers and technical leaders from the AWS community. Bryson Koehler, CIO of The Weather Company, will be speaking about how The Weather Company uses Riak in his session, “How the Weather Company Monetizes Weather, the Original Big Data Challenge.” Basho will also be available at the booth to discuss Riak or Riak CS. AWS re:Invent will take place in Las Vegas from November 12-15th.
For a full list of where we’ll be for the rest of the year, check out the Events Page.
October 31, 2013
If you attended RICON West, we’d love to hear your feedback! Please fill out the survey here.
Thanks to everyone who helped make the sold-out RICON West a huge success! RICON has come a long way in just one year and we are excited to see how it grows and evolves in the future.
RICON West featured over 25 speakers from academia and industry, including speakers from Basho, Google, Microsoft Research, Netflix, Salesforce, Seagate, The Weather Company, and Twitter. Over two days, they discussed the theory, practice, and importance of running distributed systems in production as well as some predictions on what’s in store for the future. Over the next few weeks, we will be posting slides and videos from all of the talks on both ricon.io and the blog.
In case you missed it, we also received some great press. Here’s a quick recap:
- “Salesforce’s data-center design: ‘Go for web scale, and build it out of s**t!’“
- “What do we want? Strong consistency! When do we… oh, it’s in Riak v2“
What’s next for RICON? With three conferences already under our belt, we are excited to get to work on RICON Europe (our first international conference!) and continue RICON East. Keep an eye on Ricon.io and our blog for more details.
RICON West was also paired with a one-day Riak training. We plan on making these a more regular occurrence all over the country.