Data Exchange Studio: a Comprehensive Tool for Dynamics AX Integration

10/09/2014

Today, I will walk you through the decisions we took to build the Dynamics AX integration solution. If you’re interested on a prequel to this post, please take a look at Face Data Integration With No Fear.

The data transfer process is a routine. However, a solution should not be like that. That’s why Data Exchange Studio (DES) has been designed with “3 Easy To Do” in mind: “Easy to use”, “Easy to control”, and “Easy to develop”.

DES consists of three main entities: party, interface and media. To say we are about to pass some data from the AX database (a source party) into the XML container (a target party will consume an XML file in its environment afterwards). We start configuration of a new interface and specify the necessary media at each end point. DES works with plain text, Excel files, XML and the Dynamics AX database. All these media are supported out of the box. The user is free to develop additional media to use proprietary data formats.

As a result, the transmitter and receiver sides form a link between each other. In other words, construct an interface between one system and another. Please take a look at the picture below.

DES-scheme

At the moment, we have specified the source media (to comprehend an inbound data) and the type of target media (to pass it to the receiver). The framework is ready to validate data according to format constraints. To transfer data, it needs information how the source data element is related to the target one. That’s an essential part of the interface configuration (so-called field mapping). The mapping consists of data description and the logic behind it.
Let’s elaborate what the mapping is. It’s the way the interface consumes the building blocks of source media and relays the building blocks of target media to the receiver. The user has a wide range of tools to construct mapping:

    • Obtain and set metadata (like the Excel properties, or the CSV header).
    • Apply the data transformation (e.g. register the new Customer if the new name requirements have been met).
    • Perform the data conversion (for instance, if the block is missing, fallback to the default value).
    • Manage data types (helps to validate the input values and warns the user in case of inconsistency threat).

We proceed with our sample and populate the mapping will field pairs (types, names, etc.) From now on the framework is ready to generate the code. DES will generate the necessary classes according to the user configuration. The user adds the custom code if necessary and invokes the transfer process.

Being a framework, the Data Exchange solution brings clarity in the way the interface should be implemented. The open architecture serves as a basement for a developer, easy to see, easy to understand and easy to extend. All stages of data processing can be replaced with custom ones. There is a great set of entry points that the user can tune.

In order to be comprehensive, the solution should provide a wide range of information tools, so the user has a full control over what’s going on. In that regard, DES tends to be more verbose rather than to be laconic:

    • Solution accumulates info about each transmission. The user always knows when interface has been started and by who.
    • Each processed file is logged. The user is always provided with folder names, file names, directories that the file has been archived to.
    • The transmission status is described in detail: there are counters of errors, successes, highlights of the failed data.
    • The messaging system clarifies the overall process of the transfer. It indicates the reasons of failed attempts, warns the user about suspicious data, specifies the initiator of errors, helps to trace all the way to a cause of a certain issue.

To achieve the main goal (which is conducting a robust data transfer), each media has been developed carefully to inform the user about sensitive places in data format, data validation, and missing data. The implemented interface, working in the production environment with a schedule, is set to embrace the small errors by a certain error tolerance strategy—that’s what DES is working hard on.

The DES solution proved its purpose on live projects. Data exchange between a stack of companies has been developed on the DES foundation. As a result, dozens of interfaces had interconnected databases of different nature. Data Exchange Studio provides guidance for successful integration. However, it doesn’t restrain developers to a certain path. It can be leveraged by integrators to easily create mapping and by developers to modify its classes, methods, etc. The implemented interfaces can be maintained years after they’ve been created.