SOA Data integration and small or mid size companies

We are often asked if undertaking a data integration project makes sense for small and mid size companies. In particular, does it make sense to implement this using a Service Oriented Architecture?

The answer is that it makes as much sense for a small or medium sized company as it does for a large company. In fact it makes more sense in many ways as such companies can achieve this more easily as they are likely to have less data in different places. Just consider that large companies were once small or medium sized and ended up undertaking data integration projects due to the fact that they didn’t integrate their data sources when they were smaller.

This does not have to happen as part of a ‘big bang’ approach. A decision should be taken to do this and as these IT projects progress, use each project to implement SOA data services to represent the business objects that are needed. This will provide the following benefits:

-         It can help to eliminate duplicate data in your organization. Duplication of data leads to large and expensive problems down the road as it gets out of date.

-         It will help your organization’s IT systems to scale as your organization and its data grows.

-         It will enable you to react more quickly to your business requirements as your data services can be immediately available to new systems.

So the time to start implementing data services in your organization is as soon as possible! The SOA Gateway can help any size organization to create data services for its data.

Deploying ‘Connected’ Excel Applications with SOA

Microsoft’s Excel has become an extremely popular tool for collating data, preparing reports and then deploying these spreadsheets throughout an organization. Given the power of the technology, this is not a surprise; however, many of these reports are out of date within hours (or minutes) as the data is included in the spreadsheet without the ability to get the latest copy directly.

Standards based SOA provides the potential to enable an Excel spreadsheet to get the latest set of the data into a spreadsheet with the press of a button. This ensures that when an Excel spreadsheet is delivered to an authorised user, they can immediately ensure they have the latest data on an hourly, daily or weekly basis by simply requesting the latest data.

This is achieved by setting up a simple REST based data service to make the data available to authorised users and using it as follows:

-         Most if not all PCs in today’s networks are connected to server systems using TCP/IP.

-         Excel has a standard ability to use a REST based service to import data which can be executed over TCP/IP. Unlike ODBC connections, this requires NO software installation on the PC.

-         The service can deliver an XML document with the relevant data and Excel can take this updated data and recreate pie charts, reports, pivot tables etc. with the most up to date data.

-         This is all done automatically without the need for any manual intervention to collect the data for Excel.

So your Excel spreadsheets need never be out of date and again and only need to be updated when the charts or reports need to change. Consider also that other technologies such as MS Access can also get at data in a similar way so the same technique can be used for other technologies such as Web Browsers. The SOA Gateway can create such services for you from existing applications or database tables and files.

REST or SOAP Services – When do I use which ?

Our customers often ask if it is better to use REST services or SOAP services and much like other similar questions, it depends very much on what you are trying to achieve.

REST based services lend themselves very well to simple, stateless requests for information or for updating a single resource that is updated in isolation. Due to the statelessness of REST requests, each one must work or fail in isolation so it must be clear that recovery of multiple actions undertaken with REST can only be backed out using a compensating transaction. In other words, your application needs to manually undo any changes that it has made. REST also sometimes requires that a number of different formats of errors may be returned which may make error recovery a little convoluted.

SOAP servers lend themselves better to more complex tasks where requests and services tend to depend on each other. For example where two services need to be updated together or not at all. Programmatically it offers a more controlled way to work with the service through service specific SOAP headers for example. It also tends to produce errors or problems in a consistent way using SOAP Faults.

If  I were to hazard some simple rules of thumb:

-         Use REST when simple unrelated updates are required or where the latest set of a specific set of data is required.

-         Use SOAP where more complex related updates are required or where information needs to be gathered and aggregated from multiple sources.

....but be warned, like many other rules of thumb it’s the exception that makes the rule so choose whatever is best for your particular requirement to satisfy the business need. The SOA Gateway gives you the ability to have both a REST or SOAP based access to the same service and thus gives you the ability to see which one fits best for your specific requirement.

SOA “RESTful” Web Services for simple integration

There is a term going around which talks about ‘RESTful’ services which our customers often ask about. It gives the impression of something that can provide great ‘peace and tranquillity’ and in a way, it can.

The ‘REST’ in this expression refers to the REpresentational State Transfer “protocol”.  While this may sound like a heavy duty standard that must be studied in depth, nothing can be further from the truth. Essentially this is the URL based protocol that we are using on a day to day basis in our browsers.

The idea is quite simple in that each URL is unique and is stateless so that it can be invoked at any time to get the data represented by that URL. In a heavy duty REST implementation, one URL will deliver data with links to other URLs to allow you to drill down to the data you want.

In the complex world of integration, a REST Web Service is the simplest way to get data from somewhere to somewhere; consider how easily we move data around the internet ?  For example, a REST service could represent the latest set of data from a data source such as a database.

Any software you have today, be it a simple editor or MS Excel or MS Word, can ‘open’ a URL so can in fact issue a REST request to retrieve  this latest data. Consider a service that delivers information on employees and their locations; MS Word could run a mail merge to write to all of these employees. If the REST service delivers stock quotes, MS Excel could use the data to produce pie charts. Java and .Net can all invoke these RESTful services so what easier way to integrate with your existing data and applications.

The ultimate test of this simplicity is that the service can be tested very easily by using a browser to open the URL. While the data coming back will be raw XML, it’s a very simple way to see if the service is working and doing what you expect. Linked with a stylesheet, a simple way to deliver data to a browser is with a RESTful service linked with a stylesheet that makes it presentable.

With such simplicity, why would you want to integrate in any other way ? The SOA Gateway can create RESTful Web Services in seconds from existing database tables or applications.