Apatar data migration framework

As I mentioned in my last post, I've run across a couple of open source projects that have really jumped out of the pack with their attention to customer service.  This first of these is Apatar.  I found these guys as I was searching for a way to get data from a Salesforce.com system migrated to SugarCRM

Anyone who's ever done data migration for a project knows how quickly this task can suck the life out of a schedule.  Apatar's product looked to be just what I needed: an open data migration framework with end-point plug-ins for all sorts of systems, including my "from" and "to" systems, so I gave it a try. 


Their product is a Java-based canvas that hosts migration steps, where the steps perform any number of functions, including mapping between two data sources.  Using the pre-built connectors for SalesForce and Sugar (both using these products' SOAP interfaces), I was able to wire up a test migration harness very quickly.

I really liked the fact that I'd be able to construct a migration job that I could run again and again, and I loved the availability of all the plug-ins.  The company is a mixed-model organization - Apatar is available in open source and "Enterprise" variants.  As with other companies in this model (SugarCRM comes to mind), it is important to gauge the vitality of the open source community if you're interested in that variant.  In Apatar's case, they've got a really nice developer portal (good), but the featured content is anywhere from a couple weeks to several months old (not great).  If you're interested in Apatar, I would definitely suggest testing the response time and accuracy of the community support before betting the farm on it.

It turns out I wasn't able to use Apatar for my project -- partly due to problems in my infrastructure.  The production SalesForce system I was migrating from didn't have the SOAP API enabled, which really eliminated Apatar as a migration tool.  I tried using Apatar to read a CSV file and upload to Sugar, but I ran into problems there, as well.  To be fair to Apatar, I've used the Sugar SOAP API separately, and it is pretty touchy.  When it encounters a problem, it throws exceptions first and asks questions later. 

Error handling is one area where Apatar could use some improvement, though.  Instead of just aborting the whole job or logging the fact that x-thousand records had errors, it would be nice if Apatar would log the problem in a form that lets me go back and examine the input and output rows.  After all, if more than 5-10% of my records are failing, there's probably something in my mapping I'm going to have to fix, and that's going to be a whole lot easier if I can see what was coming in and what was going out.  For bonus points, give me an interactive debugger to let me look at the data values.

Early in my evaluation, I got an email from Apatar asking if I needed any help, and another asking for my feedback.  When I got to the point where I was able to give some constructive comments, I got in touch with them and exchanged emails with one of their techs.  There was really nothing to be done about my #1 problem (no access to SalesForce's SOAP API), but I offered my feedback about error handling, and it was well-received.

I'll definitely keep Apatar on my radar to see if they continue to grow and improve.  They've got the potential to be a great tool to have in the chest.

8 Replies to “Apatar data migration framework”

  1. Hello, David!
    Thanks fromHello, David!

    Thanks from the whole Apatar community. We do our best to improve the project. =)

    I will definitely send your ideas about handling errors to Apatar development team.

    If you have any more feedback or suggestions, don’t hesitate to email us in the future!

    Alex Khizhnyak
    Apatar Evangelist

  2. Thanks, Alex – I lookThanks, Alex – I look forward to a lot more innovation from Apatar. This isn’t always the flashiest product segment, but its a job that really eats you alive without good tools, so I appreciate the work you’re doing.

  3. David,
    If you’re looking atDavid,
    If you’re looking at migrating data from Salesforce to Sugar, I’d suggest you also try Talend Open Studio (http://www.talend.com), another open source data integration solution. Talend Open Studio is certified by SugarCRM, so your chances are better that it will work with the Sugar APIs. And if you take a look at the Talend communities, you’ll find them to be pretty active.

    Yves de Montcheuil

  4. Yves,
    Thanks for the info -Yves,

    Thanks for the info – I hadn’t appreciated the breadth of your product when I came across the bits on the SugarForge site. I’ve already done the migration in this case, but I may be presented with more opportunities in the future. In the mean time, I’ll try to find time to download Talend and take a look at its capabilities. There may be other integration needs that this would fill.


Comments are closed.