November 11, 2010
Things are moving incredibly fast in the NoSQL space. I am used to internet-fast — helping bring on 300 customers in a year at Akamai; going from adult bulletin boards and leased lines to hosting sites for twenty percent of the Fortune 500 at Digex (Verizon Business) in eighteen months. I have never seen a space explode like the NoSQL space.
Two weeks ago, Justin Sheehy stood on stage delivering a rousing and thoughtful presentation to the NoSQL East Conference that was less about Riak and more about a definition of first principles that underpinned Riak: what it REALLY means when you claim such terms as scalability (it doesn’t mean buying a bigger machine for your master DB) and fault-tolerance (it has to apply to writes and reads and is binary; you either always accept writes and serve reads or you don’t). The conference was a bit of a coming out party for Basho, which co-sponsored the event with Rackspace, Georgia Tech, and a host of other companies. We had been working on Riak for 18 months or so in relative quiet and it was nice to finally see what people thought, first hand.
There were equally interesting presentations about Pig and MongoDB and a host of other NoSQL entrants, all of which will make for engrossing viewing when they finally get posted. We were told this wasn’t quite as exciting as the NoSQL conference out West but none of us seemed to mind. Home Depot, Turner Broadcasting, Weather.com, and Comcast had all sent folks down to evaluate the technology for real, live problems and the enthusiasm in the auditorium spilled out into the Atlanta bars. Business cards were exchanged, calls set up, even a little business discussed. Clearly, NoSQL databases were maturing fast.
No sooner had we returned to Cambridge than news of Flybridge’s investment in 10Gen came out. Hooray! Someone was willing to bet a $3.4 million dollars on a company in the space. Chip Hazard, ever affable, wrote a nice blog post explaining the investment. According to him, every developer they talked to had downloaded some NoSQL database to test. Brilliant news. He said Flybridge invested in 10Gen because they liked the space and knew the team from their investment in Doubleclick, from whose loins the management team at 10Gen issued. No more felicitous reason exists for a group of persons to invest $3.4 million than that previous investments in the same team were handsomely rewarded. I would wish Chip and 10Gen the best if I had time.
Because contemporaneous with the news of Flybridge’s investment, and almost as if the world had decided NoSQL’s time had come, we began to field emails and calls from interested parties. Trials, quotes, lengthy discussions about features and uses of Riak — the week was a blur. Everyone was conducting a bakeoff: “I have a 4TB database and customers in three continents. I am evaluating Riak and two other document datastores. Tell me about your OLAP features.”
Heady times and, frankly, of somewhat dubious promise, if you ask me. Potential clients that materialize so quickly always seem to disappear just as fast. Really embracing a new technology requires trials, tests, new features, and time. Time most off all. These “bluebirds” would fly away in no time, if my experience held true.
Except, this time it didn’t happen. Contracts were exchanged. Pen nibs were sharpened. It is as if the entire world decided to not wait for the everyone else to jump on the bandwagon and instead, decided to go NoSQL. Even using this last week as the sole example, I think the reason is plain — people have real pain and suddenly the word is out that they no longer have to suffer.
Devs are constrained by what they can build, rich features notwithstanding. Ask the company that had to choose between Riak and a $100K in-memory appliance to scale. And Ops is getting slaughtered — the cost of scaling poorly (and by poorly I mean pagers going off during dinner, bulk updates taking hours and failing all the time, fragmented and unmanageable indices consuming dozens of machines) is beginning to look like the cost of antiquated technology. Good Ops people are not fools. They look for ways to make life easier. Make no mistake — all the Devs and Ops folks came with a set of tough questions and a list of new features. They also came with an understanding that companies that release open source software still have a business to run. They are willing to spend on a real company. In fact, having a business behind Riak ended up mattering as much as any features.
So, I suspect, we are at the proverbial “end of the beginning.” Smart people in the NoSQL movement have succeeded in building convincingly good software and then explaining the virtues convincingly (all but one of the presentations at NoSQL East demonstrated the virtues of the respective approaches). Now these people are connecting to smart people responsible for building and running web apps, people who are decidedly unwilling to sit around hoping for Oracle or IBM to solve their problems.
In the new phase — which we will cleverly call the “beginning of the middle” — great tech will matter even more than it does now. It won’t be about selling or marketing or any of that. If our numbers are any indication of a larger trend, more people will download and install NoSQL databases in the next month than the combined total of the three months previous. More people in a buying frame of mind will evaluate NoSQL technology not in terms of its coolness but in terms of its ability to solve their real, often expensive problems. The next phase will be rigorous in a way this phase was not. People have created several entirely new ways to store and distribute data. That was the easy part.
Just as much as great tech, the people behind it will matter. That means more calls between us and Dev teams. That means more feature requests considered and, possibly, judiciously, agreed to.
That also means lots of questions answered. People care about support. They care about whether you answer their emails in a timely fashion and are polite. People want to do business with NoSQL. They want to spend money to solve problems. They need to know they are spending it with responsible, responsive, dedicated people.
Earl tweets about it all the time and I happen to agree: any NoSQL success helps all NoSQL players. I also happen to feel that any failure hurts all NoSQL players. As NoSQL rapidly ages into its adolescence, it will either be awkward and painful or exciting and characterized by incredible growth.
When I was a kid on the Navy base in Alameda, my babysitter watched soaps all afternoon, leaving me mostly to my own devices. If I stopped in, I always got roped in to hearing her explain her favorite stories. Most of all she loved how ridiculous they were, though she would never admit this exactly. Instead, adopting an attitude of gleeful incredulity, she would point out this or that attractive young actor and tell me how just a year ago, she was a little baby. “Soap people have to grow up quick, I guess,” was her single (and to her, completely satisfactory) explanation. “If they don’t, they get written out of the story.”
November 11, 2010
We announced recently on the Riak Mailing List that Basho was switching to git and GitHub for development of Riak and all other Basho software. As stated in that linked email, we did this primarily for reasons pertaining to community involvement in the development of Riak. The explanation on the Mailing List was a bit terse, so we wanted to share some more details to ensure we answered all the questions related to the switch.
Riak was initially used as the underlying data store for an application Basho was selling several years ago and, at that time, its development was exclusively internal. The team used Mercurial for internal projects, so that was the de-facto DVCS choice for the source.
When we open-sourced Riak in August 2009, being Mercurial users, we chose to use BitBucket as our canonical repository. At the time we open-sourced it, we were less concerned with community involvement in the development process than we are now. Our primary reason for open-sourcing Riak was to get it into the hands of more developers faster.
Not long after this happened, the questions about why we weren’t on GitHub started to roll in. Our response was that we were a Mercurial shop and BitBucket was a natural extension of that. Sometime towards the beginning of May we started maintaining an official mirror of our code on GitHub. This mirror was our way of acknowledging that there is more than one way to develop software collaboratively and that we weren’t ignoring the heaps of developers who were dedicated GitHub users and preferred to look at and work with code on this platform.
GitHub has the concept of “Watchers” (analogous to “Followers” on BitBucket). We started accumulating Watchers once this GitHub mirror was in place. “Watchers” is a useful, but not absolute, metric for measuring interest and activity in a project. They bring a tremendous amount of attention to any given project through their use of the code and their promotion of it. They also, in the best case scenario, will enhance the code in a meaningful way by finding bugs and contributing patches.
This table shows the week on week of growth of BitBucket Followers vs. GitHub Watchers since we put the official mirror in place:
|Number of Followers/Watchers at Time of Switch||97||145|
|Avg. Week on Week Growth (%)||0.74||7.2|
Since putting the official mirror in place, the number of Watchers on the GitHub repo for Riak has grown at steady ready, averaging just over 7% week on week. This far outpaced the less than 1% growth in Followers on the canonical Bitbucket repository for Riak.
With this information it was clear that Riak on GitHub as a mirror was bringing us more attention and driving more community growth than was our canonical repo on BitBucket. So, in the interest of community development, we decided that Riak needed to live on GitHub. What they have built is truly the most collaborative and simple-to-use development platform there is (at least one well-respected software analyst has even called it “the future of open source”). Though Mercurial was deeply ingrained in our development process, we were willing to tolerate the workflow hiccups that arose during the week or so it took to get used to git in exchange for the resulting increase in attention and community contributions.
The switch is already proving fruitful. In addition to the sharp influx in Watchers for Riak, we’ve already taken some excellent code contributions via GitHub. That said, there is much left to be written. And we would love for you to join us in building something legendary in Riak, whatever your distributed version control system and platform preference may be.
October 26, 2010
Basho is hosting one event this week and participating in another. Here are the details to make sure everyone is up to speed:
A NOSQL Evening in Palo Alto
Tonight there will be a special edition of the Silicon Valley NoSQL Meetup, billed as “A NOSQL Evening in Palo Alto.” Why do I say “special”? Because this month’s event has been organized by the one and only Tim Anglade as part of his NoSQL World Tour. And this is shaping up to be one of the tour’s banner events.
There are almost 200 people signed up to see this discussion as it’s sure to be action-packed and informative. If you’re in the area and can make it out on short notice, I would recommend you attend.
October San Francisco Riak Meetup
On Thursday night, from 7-9, we are holding the October installment of the San Francisco Riak Meetup. Like last month, the awesome team at Engine Yard has once again been gracious enough to offer us their space for the event.
We have two great planned talks for this month. The first will be Basho hacker Kevin Smith talking about a feature of Riak that he has had a major hand in writing: MapReduce. Kevin is planning to cover everything from design to new code demos to the road map. In short, this should be exceptional.
For the second half of Thurday’s meetup we are going to get more interactive than usual. Articulation of use cases and database applicability is still something largely unaddressed in our space. So we thought we would address it. We are inviting people to submit use cases in advance of the meetup with some specific information about their apps. The Basho Developers are going to do some work before the event analyzing the use cases and then, with some help from the crowd, determine if and how Riak will work for a given use case – and if Riak isn’t the right fit, we might even help you find one that is. If you are curious whether or not Riak is the right database for that Facebook-killer you’re planning to build, now is your chance to find out. We still have room for one or two more use cases, so even if you’re not going to be able to attend the Thursday’s meetup I want to hear from you. Follow the instructions on the meetup page linked above to submit a use case.
That said, if you are in the Bay Area on Thursday night and want to have some beer and pizza with a few developers who are passionate about Riak and distributed systems, RSVP for the event. You won’t be disappointed.
Hope to see you there!
Basho poised to establish and lead new commercial category of enterprise-grade, fully distributed, fault tolerant, low cost data management software.
CAMBRIDGE, MA – October 19, 2010 – Basho Technologies, Inc., the distributed data storage and retrieval company, today announced Anthony S. Thornley has agreed to join Basho’s Board of Directors. Mr. Thornley served as President and Chief Operating Officer of QUALCOMM Inc. from February 2002 to July 2005. Previously he had served as Chief Financial Officer of QUALCOMM starting in 1994. He currently serves on the Board of Directors of Golf equipment manufacturer Callaway Golf Co., semiconductor technology company Cavium Networks and a development stage pharmaceutical company Transdel Pharmaceuticals as well as several private technology companies.
“By joining our Board, Tony sends the strongest message to date that Riak, our distributed data storage technology, is uniquely poised to displace traditional database technologies in the enterprise,” said Earl Galleher, Chairman and CEO of Basho Technologies.
“Enterprises, mobile providers, and early stage companies alike face the same challenge,” said Mr. Thornley. “They struggle to realize the opportunity presented by Big Data because of the ‘Big Price Tag’ of traditional database vendors like Oracle, IBM, and Microsoft. I am excited to work closely with the Basho team to create a market-leading company. They have proven customer traction and they have a strong team and growth plan. I feel I can be very helpful to the company.”
“Again Basho stands apart from the crowd,” said Mr. Galleher. “Tony brings the unique perspective of an executive who has worked with multiple companies that have successfully transformed industries. He understands why and when enterprises invest in change. His experience will translate immediately into accelerated market traction for Riak.”
Prior to joining Qualcomm, Mr. Thornley was employed at Nortel Networks for 16 years in various financial and information systems management positions, including Vice President of Finance and Corporate Controller. Mr. Thornley received his bachelor of science in Chemistry from the University of Manchester, England and is a chartered accountant.
About Basho Technologies
Basho Technologies, Inc., founded in January 2008 by a core group of software architects, engineers and executive leadership from Akamai Technologies, Inc. (Nasdaq: AKAM) is headquartered in Cambridge, Massachusetts. Basho produces Riak, a distributed data store that combines high availability, easily-scalable capacity and throughput, and ease of use. Riak’s high availability data store means that applications built using Riak remain both read and write available under almost any operational conditions and without requiring intervention. Available in both an open source and a paid commercial version, Riak provides unprecedented read- and write availability to web, mobile, and enterprise applications.
CEO, Basho Technologies, Inc.
September 29, 2010
Basho hackers will be giving quite a few presentations between now and the end of the week. And they all happen to be in Baltimore! Here is a quick rundown (in no particular order) of where we will be, who will be there, and what we will be talking about:
Rusty Klophaus at CUFP
Rusty Klophaus will be at the Commercial Users of Functional Programming (CUFP) Event taking place this weekend in Baltimore, Maryland. His talk is called “Riak Core: Building Distributed Applications Without Shared State” and it should be downright amazing.
From the talk’s description: Both Riak KV (a key-value datastore and map/reduce platform) and Riak Search (a Solr-compatible full-text search and indexing engine) are built around a library called Riak Core that manages the mechanics of running a distributed application in a cluster without requiring a central coordinator or shared state. Using Riak Core, these applications can scale to hundreds of servers, handle enterprise-sized amounts of data, and remain operational in the face of server failure.
All the details on his talk can be found here.
And, in case you haven’t been following your Riak Core developments, check out Building Distributed Systems with Riak Core.
Dave Smith at the Ninth ACM SIGPLAN Erlang Workshop
Dave Smith (a.k.a “Dizzyd”) will be keynoting the ACM SIGPLAN Erlang Workshop taking place on Friday, Sept 30th, also in Baltimore. Dave’s talk is called “Rebar, Bitcask and how chemotherapy made me a better developer.”
Rebar and Bitcask are both pieces of software that Dizzy had a major hand in creating and they have played a huge role in Riak’s adoption (not to mention that Rebar has quickly become an indispensable tool for Erlang developers everywhere). Dave was also fighting follicular lymphoma while writing a lot of this code. Needless to say, this one is sure to be memorable and of immense value.
More details on his talk can be found here.
It should also be noted that newly-minted Basho Developer Scott Fritchie is the Workshop Chair for this event. He is an accomplished Erlang developer and Riak is not the only distributed key/value store about which Scott is passionate – he will also happily talk your ear off about Hibari.
Justin Sheehy at Surge
Just when you thought they couldn’t fit another conference in Baltimore on the same weekend… And this one is big: it’s the Surge Conference put on by the team at OmniTI. Basho will be there in the form of CTO Justin Sheehy.
Justin will be giving a talk about concurrency at scale, something about which every distributed systems developer should care deeply. Additionally, he will be taking part in a panel discussion – I haven’t seen an official name for it yet but rumor has it that it’s something along the lines of “SQL versus NoSQL.”
Check out the Surge site for more conference details.
As you can see, Baltimore is the place to be this weekend. Get there at all costs. And then go download Riak.
September 1, 2010
We added a new Community Editor this week. His name is Mårten Gustafson.
Aside from being involved in putting a Riak application into production, Mårten has been active, knowledgeable and helpful on the Riak Mailing list and in the IRC Room (where he goes by the unassuming “chids”). He recently came forward and expressed interest in being a Community Editor, and based on his Riak credentials, the team here at Basho was more than happy to bring him aboard.
Welcome, Mårten! We are looking forward to your contributions.
If you’re interested in being a Community Editor for the Riak Wiki, let us know. We would love to talk to you.
Funding will accelerate already rapid market adoption of Riak, Basho’s industry transforming distributed database software.
CAMBRIDGE, MA – August 9, 2010 – Basho Technologies, Inc. today announced it has secured $2 million from a diverse collection of angel investors in a Series C preferred equity financing. Eschewing traditional early-stage funding sources in the venture capital industry, Basho instead cultivated a network of entrepreneurial angels with an appetite for disruptive technologies like Riak who were eager to fill the post- economic collapse investment gap. The company forecasts this financing will carry it to profitability by mid-2011.
“We have an innovative product in Riak so it is only fitting we have pursued a similarly innovative financing strategy,” said Earl Galleher, Chairman and CEO of Basho. “We have been fortunate to find a network of angels who understand our market and value what we have accomplished to date. Our investors have given us the time and opportunity to fully develop our product vision and to prove our market plan.”
Founded in 2008 by former Akamai Technologies (NASDAQ: AKAM) executives and senior engineers, Basho designed Riak to free corporations from the constraints of traditional, highly centralized database architectures (like those offered by Oracle, IBM, Microsoft, etc.). Enterprises like Comcast, Wikia Inc., Mochi Media, and Vibrant Media use Riak not just as an alternative to traditional databases but as the cornerstone for previously uncontemplated application architectures.
“In the late 1990s, the founders of Akamai solved the complex problem of a growing Internet by allowing anyone with a website to efficiently distribute content around the world on any network,” continued Mr. Galleher. “Ten years later, we are doing something similar but with a much more complex and expensive problem, the database
itself. We and our investors believe Riak will rapidly transform the entire database industry.”
Applications built with Riak can sustain catastrophic server, data center, and network failures without outages while avoiding the complexity and expense that characterize applications built using traditional, highly centralized databases. Optimized for the emerging class of “cloud” infrastructure, organizations building applications using Riak can scale out dynamically to handle sudden surges in load and scale back when load decreases to reduce expenses.
About Basho Technologies
Basho Technologies, Inc., founded in January 2008 by a core group of software architects, engineers and executive leadership from Akamai Technologies, Inc. (Nasdaq: AKAM) is headquartered in Cambridge, Massachusetts. Basho produces Riak, a distributed data store that combines high availability, easily-scalable capacity and throughput, and ease of use. Riak’s high availability data store means that applications built using Riak remain both read and write available under almost any operational conditions and without requiring intervention. Available in both an open source and a paid commercial version, Riak provides unprecedented read- and write-availability to web, mobile, and enterprise applications.
CEO, Basho Technologies, Inc.
August 8, 2010
This is a huge day for Basho Technologies, Riak, and our growing community of users.
We are thrilled to announce Basho’s partnership with Joyent to bring our community hosted Riak on Joyent’s Smart platform. With both open source and enterprise versions available, anyone can quickly spin up a Riak cluster and start building applications.
When we first began talking to Jason and David and the rest of the Joyent team early this year, we realized we shared a common vision for the future of infrastructure. The past several months have been spent finalizing the details, and in just a few weeks you’ll be able to go to my.joyent.com and, with a few clicks, purchase and deploy as many nodes of Riak you want, need, and can handle.
Making pre-configured Riak SmartMachines available in the Joyent cloud will enable developers to combine all the benefits of Riak with the proven, advanced hosting platform that businesses like LinkedIn, Gilt, and Backstage rely on every day.
Mark your calendar, because hosted Riak is here!
CAMBRIDGE, MA – August 3, 2010 – Basho Technologies today announced Wikia, Inc. has selected Riak, Basho’s next-generation distributed data store, as the foundation for a new set of global services. Wikia is the 70th largest site on the Internet according to Quantcast and brings millions of people together daily to create and discover engaging content. Wikia selected Riak over traditional databases and other emerging data storage technologies to distribute its data around the world and bring it closer to its global audience.
“Riak has allowed us to do something that was impossible before,” said Artur Bergman, Wikia’s Vice President of Engineering and Operations. “With Riak we can break through the ceiling on performance imposed by traditional database technologies and continue to improve the experience of our users. We invest in technology that benefits Wikia’s growing user base, therefore Riak made perfect sense. Riak is fast, easy to run, and extremely resilient to the failure scenarios anyone with real operational experience knows are all too common.”
Founded in 2008 by former Akamai Technologies (NASDAQ: AKAM) executives and senior engineers, Basho designed Riak to provide the same high availability and rapid scaling properties provided by leading content delivery networks. Applications built with Riak can sustain catastrophic server, data center, and network failures without outages, while avoiding the complexity and expense that characterize applications built using traditional databases.
“Basho is excited to have a respected and forward-looking client like Wikia so readily embrace Riak,” said Earl Galleher, Basho’s Chairman and CEO. “More and more, we see companies reject the limitations of traditional databases like Oracle and MySQL in favor of Riak’s flexibility and ease of use. Riak doesn’t just solve problems for organizations running applications on old database architectures; it frees them to build entirely new classes of applications.”
Wikia intends to deploy a replicated user session service running simultaneously in three data centers in the U.S. and Europe, replacing its current solution which is restricted to a single data center. Mr. Bergman has already contributed a file system adapter to the Riak open source community which will be used in the Wikia production environment.
“We did not set out to build a disruptive technology. We simply wanted to solve a problem faced by anyone running old database technologies,” said Mr. Galleher. “We have only scratched the surface of what Riak can do.”
Wikia, founded by Wikipedia founder Jimmy Wales and Angela Beesley, is the place where millions of passionate people come to discover, create, and share an abundance of information on thousands of topics. Wikia sites are written by community members that are deeply excited and knowledgeable about subjects ranging from video games, television shows, and movies to food, fashion, and environmental sustainability. With over four million pages of content and 150,000 enthusiast communities, Wikia attracts more than 30 million unique global visitors per month and has been listed in the Quantcast top 100 sites on the Internet since early 2009.
About Basho Technologies
Basho Technologies, Inc., founded in January 2008 by a core group of software architects, engineers, and executive leadership from Akamai Technologies, Inc. (Nasdaq:AKAM – News), is headquartered in Cambridge, Massachusetts. Basho produces Riak, a distributed data store that combines extreme fault tolerance, rapid scalability, and ease of use. Designed from the ground up to work with applications that run on the Internet and mobile networks, Riak is particularly well-suited for users of cloud infrastructure such as Amazon’s AWS and Joyent’s Smart platform and is available in both an open source and a paid commercial version. Current customers of Riak include Comcast Corporation, MIG-CAN, and Mochi Media.
CEO, Basho Technologies, Inc.
July 28, 2010
Sometimes you need more than words to illustrate a point. Here is Basho’s humble attempt to clarify the difference between “Dynamo-Style” systems (like Riak) that use consistent hashing to achieve fault tolerance, simple scaling, and prevent data loss, and systems that use techniques like sharding.