Scribe Adapter for Web Services and complex structures

If you're looking at migrating data into Salesforce or Dynamics CRM then Scribe Insight is one of your options. Its an integration system like SSIS and many others but its known for its particularly good support for cloud CRM systems.

Scribe Insight has various 'adapters' that lets it connect to different data sources/targets. One of the interesting ones is the Scribe Adapter for Web Services (SAFWS). It allows you to connect to an arbitrary web service and then push or pull data to/from it.

Can it pull data from any web service? That would be hard to achieve, and indeed it does have some limitations - here's a few that I encountered:

It doesn't like .NET DataSets

Despite Scott Hanselman's assertion that Returning DataSets from WebServices is the Spawn of Satan and Represents All That Is Truly Evil in the World, sometimes you do encounter web services that do, er, return DataSets.

Imagine you've got a strongly typed DataSet with several tables in it like this:

Strongly-typed DataSets are a bit ... clunky. This simple example (3 tables with 2 columns each, no relationships) generates 1319 lines of code. If you return that from a Web Service, the WSDL gets pretty complicated. And it turns out, if you have such a web service, Scribe wont be able to connect to it. The current version of the SAFWS simply hangs when you try to connect to such a service (ver 1.5, Dec 2012).

It is awkward with multiple tables

OK, so instead lets say you go for a nice clean DTO representation of the same structure:

namespace StructureDummy.Model
{
    [DataContract]
    public class Umbrella
    {
        [DataMember]
        public List Alphas { get; set; }

        [DataMember]
        public List Betas { get; set; }

        [DataMember]
        public List Gamma { get; set; }

    }

    [DataContract]
    public class Alpha
    {
        [DataMember]
        public string AlphaOne { get; set; }

        [DataMember]
        public int AlphaTwo { get; set; }
    }

    [DataContract]
    public class Beta
    {
        [DataMember]
        public string BetaOne { get; set; }

        [DataMember]
        public int BetaTwo { get; set; }
    }

    [DataContract]
    public class Gamma
    {
        [DataMember]
        public string GammaOne { get; set; }

        [DataMember]
        public int GammaTwo { get; set; }
    }
}

That is, an overall wrapper that has three separate 'tables' of two columns each. If you return something like that from a web service, SAFWS can connect to it, but it handles the multiple tables in a strange way. Here is a screenshot of how it 'sees' the output:

It has merged the three tables into one table of six columns. That means your data would come out looking something like this:

This combined table would then have to be processed through Scribe, one whole row at a time. That is not going to be easy.

Scribe is designed to process tabular data streams, so this limitation is not that surprising, but still, something to look out for.