Tag Archives: data migration

Talend and Basho Partner for Streamlined Data Migration for Customers

March 19, 2014

When implementing Riak, our customers often need to migrate their data from their existing architecture. Depending on the setup, this can cause some pain points during the transition process. Basho has partnered with Talend to make it faster and more cost-effective for customers to migrate their data from existing infrastructure to Riak.

Distributed NoSQL databases like Riak are perfect for big data projects, which require large volumes of data to be stored with the ability to predictably scale and flexible data storage for a wide variety of constantly changing data. Legacy relational systems can’t keep up with big data needs. Through this partnership, customers looking to move from a relational system to Riak can take advantage of Talend’s powerful integration technology through its native big data integration solutions, so they can quickly transition their data to Riak.

In addition to the ability to migrate their data to Riak, users are also able to easily migrate Riak data to Hadoop for big data analytics. Riak and Hadoop are fundamentally different solutions and address different challenges (for more information, check out How is Riak Different from Hadoop). Talend makes it easier for them to work together for storage and analytics.

For more information about our Talend partnership, check out the full release.

For a complete list of partners, or to become a partner, visit our Partnerships Page.


Riak Data Migration

November 19, 2013

Implementing a database for a new project is a relatively straightforward process. However, when challenges of scalability are encountered in existing applications or workflows, it may be necessary to migrate data from an existing database solution to Riak. Our Professional Services team specializes in this type of engagement (Contact Us if you need help) and has put together a general set of considerations and guidelines when migrating to Riak.

When migrating data to Riak, we recommend a staged approach – migrating specific areas to Riak while continuing to run any existing data storage architecture. For each stage, pick a standalone logical unit of data, convert it to a storage format appropriate to Riak, consider how the data will be accessed, and write the migration scripts.

You should start with areas of data that have a one-to-one relationship, which makes them easier to model as a pair of keys and values (such as sessions, user preferences or profiles, logs, or straight content). This type of data can be easy to identify, as it usually will have a readily available key, such as a user id or session id.

Once you have isolated this data, you need to plan how it will be stored in Riak. In most cases, the keys will be dictated by the existing application data (the format of the session id or user id will be already be defined) and these objects can be reused as Riak object keys. The format of your object payload will also help dictate how it’s stored in Riak. Small binaries (PDFs or small images) can be stored as binary blobs, structured tables or other data can be stored as JSON or XML, and accompanying metadata can be stored as custom Riak headers.

Once the data model is defined, the act of migration is straightforward. Extract the relevant data from the existing system, create appropriate Riak objects, and upload the data. It’s hard to get much simpler than writing keys and values.

As you continue to migrate more difficult relational data, or need help during any step of the way, we have extensive documentation at docs.basho.com, the Riak users mailing list, and the Professional Services team is always available to answer questions or even help manage your transition.