SOA Data integration and small or mid size companies

We are often asked if undertaking a data integration project makes sense for small and mid size companies. In particular, does it make sense to implement this using a Service Oriented Architecture?

The answer is that it makes as much sense for a small or medium sized company as it does for a large company. In fact it makes more sense in many ways as such companies can achieve this more easily as they are likely to have less data in different places. Just consider that large companies were once small or medium sized and ended up undertaking data integration projects due to the fact that they didn’t integrate their data sources when they were smaller.

This does not have to happen as part of a ‘big bang’ approach. A decision should be taken to do this and as these IT projects progress, use each project to implement SOA data services to represent the business objects that are needed. This will provide the following benefits:

-         It can help to eliminate duplicate data in your organization. Duplication of data leads to large and expensive problems down the road as it gets out of date.

-         It will help your organization’s IT systems to scale as your organization and its data grows.

-         It will enable you to react more quickly to your business requirements as your data services can be immediately available to new systems.

So the time to start implementing data services in your organization is as soon as possible! The SOA Gateway can help any size organization to create data services for its data.

Deploying ‘Connected’ Excel Applications with SOA

Microsoft’s Excel has become an extremely popular tool for collating data, preparing reports and then deploying these spreadsheets throughout an organization. Given the power of the technology, this is not a surprise; however, many of these reports are out of date within hours (or minutes) as the data is included in the spreadsheet without the ability to get the latest copy directly.

Standards based SOA provides the potential to enable an Excel spreadsheet to get the latest set of the data into a spreadsheet with the press of a button. This ensures that when an Excel spreadsheet is delivered to an authorised user, they can immediately ensure they have the latest data on an hourly, daily or weekly basis by simply requesting the latest data.

This is achieved by setting up a simple REST based data service to make the data available to authorised users and using it as follows:

-         Most if not all PCs in today’s networks are connected to server systems using TCP/IP.

-         Excel has a standard ability to use a REST based service to import data which can be executed over TCP/IP. Unlike ODBC connections, this requires NO software installation on the PC.

-         The service can deliver an XML document with the relevant data and Excel can take this updated data and recreate pie charts, reports, pivot tables etc. with the most up to date data.

-         This is all done automatically without the need for any manual intervention to collect the data for Excel.

So your Excel spreadsheets need never be out of date and again and only need to be updated when the charts or reports need to change. Consider also that other technologies such as MS Access can also get at data in a similar way so the same technique can be used for other technologies such as Web Browsers. The SOA Gateway can create such services for you from existing applications or database tables and files.

REST or SOAP Services – When do I use which ?

Our customers often ask if it is better to use REST services or SOAP services and much like other similar questions, it depends very much on what you are trying to achieve.

REST based services lend themselves very well to simple, stateless requests for information or for updating a single resource that is updated in isolation. Due to the statelessness of REST requests, each one must work or fail in isolation so it must be clear that recovery of multiple actions undertaken with REST can only be backed out using a compensating transaction. In other words, your application needs to manually undo any changes that it has made. REST also sometimes requires that a number of different formats of errors may be returned which may make error recovery a little convoluted.

SOAP servers lend themselves better to more complex tasks where requests and services tend to depend on each other. For example where two services need to be updated together or not at all. Programmatically it offers a more controlled way to work with the service through service specific SOAP headers for example. It also tends to produce errors or problems in a consistent way using SOAP Faults.

If  I were to hazard some simple rules of thumb:

-         Use REST when simple unrelated updates are required or where the latest set of a specific set of data is required.

-         Use SOAP where more complex related updates are required or where information needs to be gathered and aggregated from multiple sources.

....but be warned, like many other rules of thumb it’s the exception that makes the rule so choose whatever is best for your particular requirement to satisfy the business need. The SOA Gateway gives you the ability to have both a REST or SOAP based access to the same service and thus gives you the ability to see which one fits best for your specific requirement.

SOA “RESTful” Web Services for simple integration

There is a term going around which talks about ‘RESTful’ services which our customers often ask about. It gives the impression of something that can provide great ‘peace and tranquillity’ and in a way, it can.

The ‘REST’ in this expression refers to the REpresentational State Transfer “protocol”.  While this may sound like a heavy duty standard that must be studied in depth, nothing can be further from the truth. Essentially this is the URL based protocol that we are using on a day to day basis in our browsers.

The idea is quite simple in that each URL is unique and is stateless so that it can be invoked at any time to get the data represented by that URL. In a heavy duty REST implementation, one URL will deliver data with links to other URLs to allow you to drill down to the data you want.

In the complex world of integration, a REST Web Service is the simplest way to get data from somewhere to somewhere; consider how easily we move data around the internet ?  For example, a REST service could represent the latest set of data from a data source such as a database.

Any software you have today, be it a simple editor or MS Excel or MS Word, can ‘open’ a URL so can in fact issue a REST request to retrieve  this latest data. Consider a service that delivers information on employees and their locations; MS Word could run a mail merge to write to all of these employees. If the REST service delivers stock quotes, MS Excel could use the data to produce pie charts. Java and .Net can all invoke these RESTful services so what easier way to integrate with your existing data and applications.

The ultimate test of this simplicity is that the service can be tested very easily by using a browser to open the URL. While the data coming back will be raw XML, it’s a very simple way to see if the service is working and doing what you expect. Linked with a stylesheet, a simple way to deliver data to a browser is with a RESTful service linked with a stylesheet that makes it presentable.

With such simplicity, why would you want to integrate in any other way ? The SOA Gateway can create RESTful Web Services in seconds from existing database tables or applications.

Avoid Work Station maintenance with SOA

Many years ago when dumb terminals were the order of the day, upgrading those terminals meant changing a fuse or simply replacing them all together. They have by and large been replaced by a PC on every desk which has enabled unparalleled access to applications by users in all forms of organization.

With progress there are always pain points which only became evident as work stations were rolled out across organizations. One of the main pain points with work stations all over an organization is the requirement to upgrade software on each and every work station when specific server software must be updated. This can range from a few hundred PCs to many thousands and results in thousands of person days to keep such software up to date.

Consider then a solution which avoids the need to upgrade software on every PC when server software is updated. Imagine a situation where the software is installed on the server and the work stations just start using the new server software out of the box. Doesn’t this sound like a very attractive scenario ?

Well the wait is over as SOA standards like SOAP and REST mean that this can happen today. When a server communicates using the SOAP or REST standards, there is no requirement to install a work station specific component. This is achieved because the work station software either knows how to call a REST based service already or can learn how to call a SOAP service by exchanging a few messages with the server.

This has the potential to revolutionise how often you need to upgrade the work stations in your organization. The SOA Gateway will enable you to create services on your server systems that require no software footprint on your workstations.

“Response times” vs “Delivery times” - how can a SOA help ?

Those of us of a particular vintage will remember green screen applications which were essentially character based screens to allow you to communicate with your applications. We will also remember the fact that these applications generally had response times of less than a second from the time you hit ‘enter’. There was a general rule of thumb that if the response took any longer than 1 second, the user’s attention would be lost and thus reduced productivity was the outcome.

Moving forward we now have visually quite stunning graphical interfaces with which to communicate with our applications, however, how many of these provide “delivery times” of 10’s of seconds between interactions ? This is generally because the screen being displayed is made up of many parts and takes many round trips to the system to complete the screen.

This does not have to be the case. With a well designed Service Oriented Architecture, one or a limited number of services can generally provide all the information required for one screen. This means that the data to build the screen can be retrieved in less than a second. The additional ‘beautification’ that admittedly makes the screen look good is normally static in that it doesn’t change from display to display and can be cached locally.

Using a Service Oriented Architecture (SOA) and caching of graphical images, it is therefore possible to have the benefit of a fully graphical interface with response times of less than a second reminiscent of the green screen era. Many studies have shown that such responsive applications improve the productivity and job satisfaction of people using those applications many times over.

The SOA Gateway will enable you to create the services that will provide such an architecture.

How can a Service Oriented Architecture (SOA) help with Quality ?

For many projects and implementations, the cost of implementing a good quality assurance suite can cost as much as and even more than the implementation of the required functionality for the project itself. This is primarily due to the fact that with traditional project implementation, the interfaces and unit testing all involve proprietary interfaces.

When using a standards based SOA throughout your organization, it is possible to significantly reduce the costs associated with the creation of a fully functional QA suite. This is possible because when services are implemented using SOAP or REST based standards, there are a multitude of tools available freely or at minimal cost on the web.

Many of these tools enable testing of services using a configuration based approach. This facilitates the creation of QA suites without having to use programmers to code the test suites. The tools will import and understand how to call a service and then non programming resources can be used to drive the tests. This in itself represents the potential for massive savings.

Where more obscure tests are required, it is possible to drive services implemented using SOAP and REST using scripting languages like PHP or RUBY. There are many resources available that can program in these languages.

Finally a major issue with QA is that it can sometimes impact on the execution profile of a system because it runs on the same platform as the system. When standards are used, it’s possible to drive services built using these standards from a totally different system thus emulating more accurately what the real users will do to the system.

The SOA Gateway enables the creation of such services from base data access services to complex front end services to be made available to your customers