Skip to main content

Scribe Adapter for Web Services and complex structures

If you're looking at migrating data into Salesforce or Dynamics CRM then Scribe Insight is one of your options. Its an integration system like SSIS and many others but its known for its particularly good support for cloud CRM systems.

Scribe Insight has various 'adapters' that lets it connect to different data sources/targets. One of the interesting ones is the Scribe Adapter for Web Services (SAFWS). It allows you to connect to an arbitrary web service and then push or pull data to/from it.

Can it pull data from any web service? That would be hard to achieve, and indeed it does have some limitations - here's a few that I encountered:

It doesn't like .NET DataSets

Despite Scott Hanselman's assertion that Returning DataSets from WebServices is the Spawn of Satan and Represents All That Is Truly Evil in the World, sometimes you do encounter web services that do, er, return DataSets.

Imagine you've got a strongly typed DataSet with several tables in it like this:


Strongly-typed DataSets are a bit ... clunky. This simple example (3 tables with 2 columns each, no relationships) generates 1319 lines of code. If you return that from a Web Service, the WSDL gets pretty complicated. And it turns out, if you have such a web service, Scribe wont be able to connect to it. The current version of the SAFWS simply hangs when you try to connect to such a service (ver 1.5, Dec 2012).

It is awkward with multiple tables

OK, so instead lets say you go for a nice clean DTO representation of the same structure:

namespace StructureDummy.Model
{
    [DataContract]
    public class Umbrella
    {
        [DataMember]
        public List Alphas { get; set; }

        [DataMember]
        public List Betas { get; set; }

        [DataMember]
        public List Gamma { get; set; }

    }

    [DataContract]
    public class Alpha
    {
        [DataMember]
        public string AlphaOne { get; set; }

        [DataMember]
        public int AlphaTwo { get; set; }
    }

    [DataContract]
    public class Beta
    {
        [DataMember]
        public string BetaOne { get; set; }

        [DataMember]
        public int BetaTwo { get; set; }
    }

    [DataContract]
    public class Gamma
    {
        [DataMember]
        public string GammaOne { get; set; }

        [DataMember]
        public int GammaTwo { get; set; }
    }
}

That is, an overall wrapper that has three separate 'tables' of two columns each. If you return something like that from a web service, SAFWS can connect to it, but it handles the multiple tables in a strange way. Here is a screenshot of how it 'sees' the output:


It has merged the three tables into one table of six columns. That means your data would come out looking something like this:


This combined table would then have to be processed through Scribe, one whole row at a time. That is not going to be easy.

Scribe is designed to process tabular data streams, so this limitation is not that surprising, but still, something to look out for.







Comments

Popular posts from this blog

Copying data to Salesforce Sandboxes using TalenD

A common problem with Salesforce Developer Sandboxes is that they are blank. Really you're going to want some data in there, so there are various strategies for copying data from your live instance to the Sandbox. There are some paid-for solutions - SFXOrgData , Salesforce Partial Data Sandboxes - but if you've got a decent ETL tool you can build your own. There are a bunch of free ETL tools for Salesforce: JitterBit Data Loader is good for quick ad-hoc tasks but the free version makes it difficult to manage specific ETL projects or share projects with other users Pentaho Community Edition - an open source edition of the enterprise version Apatar was a free open source Salesforce ETL which still works but development seems to have stopped since 2011 TalenD Open Studio is an open source ETL tool For the task of copying data from live to a Sandbox, either Pentaho or TalenD Open Studio could be used, depending on preference. Here's a good comparison of the dif

SSRS multi-value parameters with less fail

SSRS supports multi-value parameters, which is nice, but there are a few issues with them. This is how I deal with them. Two of the problems with SSRS multi-value parameters are: You have to jump through a few hoops to get them to work with stored procedures The (Select All) option, as shown above The reason the (Select All) option is a problem is that it is a really inelegant way of saying 'this parameter does not matter to me'. If you have a list with hundreds of values, passing all of them as a default option just seems wrong. Also, if your report shows the user which items they selected, printing the whole list when they choose (Select All) is excessive. So in this post I'm going to show my particular way of: Jumping through the hoops to get Multi-Value params in a stored procedure Adding a single '--All--' value that the report interprets as meaning all the options. Getting Multi-Value params to work with Stored Procedures This is

Remote Desktop on High DPI screens

Scott Hanselman wrote a nice blog post back in January about some of the issues you might face running Windows on a High DPI screen like that of a Surface Pro or Lenova Yoga. I'm kindof mystified that he didn't mention Remote Desktop though because thats been the number one problem for me on High DPI screens. That said, if you remote into very recent Server OS's like Windows Server 2012 R2, then apparently Remote Desktop will sort out the DPI scaling automatically. Perhaps Scott hadn't noticed the Remote Desktop issue because he only remotes into Server 2012 R2. Certainly if I was Scott, I wouldn't remote into anything less than that. But, in practice, I regularly have to remote into Server 2008 machines and yes even Server 2003. If you do that from a high DPI screen, the remote desktop is rendered at regular pixel size, which makes everything tiny. Its hard to show screenshots of High DPI screens that correctly convey the pixel size, so I've photoshoppe