Solving data access problems in SOA environments

Data access to relational data sources such as Oracle, SQL Server, DB2, and Sybase is dominated by standards and interfaces such as ODBC, JDBC, ADO.NET, SQL, and XQuery. Traditional applications that use these interfaces and standards all perform the same basic set of functions in an architecture where the data access logic and business logic are integrated together in each application. This integrated approach is referred to as tightly coupled. This tightly coupled approach can lead to significant data access problems in a Service Oriented Architecture (SOA) environment.

Data access problems in SOA environments

• Performance/scalability issues
• Deployment headaches
• Higher total cost of ownership (TCO)

Performance and scalability issues can emerge when an application service designed for 50 users eventually supports 10 times that number. Deployment headaches materialise when a simple, lightweight service requires the installation and configuration of large and complex client libraries for data access. Higher TCO occurs when organisations seek to reduce heterogeneity within their SOA environment – a process which is always more time-consuming and difficult than it initially seems.

These problems can emerge for different reasons, but usually because the designer of the SOA environment is almost always a SOA architect, not a data access expert. SOA architects usually repeat a tightly coupled approach in their SOA environment design because it seems easiest or because it builds on what the architect is most comfortable with. In order to avoid these problems it is important for IT organisations to consider these ‘best practices’ for design and implementation of data access in a SOA environment and to leverage them wherever possible.

Figure 1: A tightly coupled design

Decouple data access logic and business logic

The SOA model presents a very different approach over the traditional data access model – a model that is said to be loosely coupled, or decoupled. In this scenario, the data access logic and business logic are no longer integrated and instead are maintained separate of one another.

Decoupling business logic and data access logic can be accomplished today with any standards-based data access interface. In some cases, technologies exist to facilitate the decoupling process. Microsoft provides a set of enterprise libraries which include the data access application block (DAAB) for developing a decoupled architecture using the ADO.NET interface and ADO.NET data providers. The DAAB, and technologies like it, allow IT organisations to migrate from an approach where each application contains its own data access logic to one where each application accesses a separate, but common data access layer (DAL). Building a DAL offers IT organisations the ability to develop applications faster and better. This is accomplished through developer specialisation, where application developers focus on business logic and data access developers develop and maintain data access logic.

It is possible to wrap a web service around a DAL for SOA environments. Doing so allows SOA-enabled applications to access data as XML – the standard way to exchange information in a SOA environment. Creating and sharing such a ‘data access service’ across a set of applications increases overall scalability, ease of deployment, and reusability.

No matter what interface or technology is chosen, a successful data access service depends on selecting the right database middleware. Choosing database middleware for use in a data access service is no different than choosing high quality database middleware for traditional applications, but the factors that should be considered are even more important in a SOA environment.

Figure 2: A loosely coupled design

Characteristics of high quality database middleware for SOA

• Reliability
• Consistency across data sources
• High performance & scalability
• True conformance to standards
• Robust feature and functionality support

Without use of high quality database middleware, the ability to build a robust data access service will be severely compromised as will any of the SOA-enabled application services that rely upon it.

Optimise the use of heterogeneous environments

There is a tendency for IT organisations to reduce system complexity within their environment by standardising and consolidating their systems down to as few as possible. While this approach is often seen as a way to cut IT costs, it can actually lead to higher costs as well as other problems within a SOA environment.

Figure 3: A heterogeneous environment can be difficult

Problems with standardised and consolidated systems in a SOA environment

• Reduced flexibility and reliability
• Difficulty achieving performance benchmarks
• Inconsistency of implementation due to system limitations

Cost estimates for standardisation and consolidation cost are almost always orders of magnitude more than originally estimated. Additionally, the most intractable system problems occur only after a commitment to standardise and consolidate has been made. While many of these problems can be solved with time and effort, it exacts real costs in terms of IT man-hours as well as hidden costs such as delayed system roll-outs and increased system maintenance.

A better approach for dealing with heterogeneous environments is to optimise, particularly if the existing system works and is complex. Optimising disparate applications and data sources can be achieved through the use of a data access service. With a data access service, application services can access any data source through a common, XML-based interface. Support of a new data source can be easily added through integration of an additional high quality database middleware to the data access service. Optimisation not only avoids the problems that can occur when standardising and consolidating systems in SOA environment, but it also provides a measure of future-proofing that ensures that as time passes and the system changes, the IT organisation can adapt simply by adding to an infrastructure, rather than trying to adapt something new to fit the existing, standardised paradigm.

Figure 4: SOA can make heterogeneous environments easier

Address persistent data access performance problems

Understanding the lifecycle of data access performance problems is the key to solving them. Performance bottlenecks often start in the database. Through efficient schema design and good database tuning, the performance bottleneck shifts to the application or service. Simple application code changes can help avoid performance bottlenecks and even result in significant performance improvements. Consider the following SQL statement example that executes a stored procedure where the parameters are passed as literal arguments:

{call getWineInventory
	('Cain Vineyard',
		'Cain Five',1998)}

The literal arguments can be changed to parameters, like this example:

{call getWineInventory (?,?,?)}

Doing so will result in the second example executing over 100% faster than the first.

After tuning the database and writing better performing application code, the bottleneck shifts to the database middleware. Use of high quality database middleware will resolve many of the common performance problems that plague applications of all types.

Figure 5: Even SOA can have complexity problems

Choosing of high quality database middleware for performance

• Confirm that high performance will scale up for any number of users
• Verify the existence of runtime tuning options for adjusting to any environment
• Measure CPU and network resource use to determine efficiency

Database middleware vendors publish benchmarks of all kinds, but no vendor benchmark is a substitute for testing how a product actually performs with a specific application or environment. Careful research, full-load testing, and appropriate middleware tuning are all essential to get a true sense of how well a particular set of middleware will perform in production.

Streamline data access with data integration techniques

Most SOA environments begin with simple services performing simple tasks, such as a service looking up an order ID from a single data source. As the environment expands and the comfort level of the IT organisation increases, however, the complexity of the services increases. Complex application services may encapsulate business logic that consumes a request submitted in an XML format, retrieves and integrates data from multiple data sources, and then publishes an XML-based response.

This complexity presents a problem for developers of these services: What is the quickest and most efficient way to build complex application services that require data access? Because SOA relies upon XML for data exchange, a natural fit can be to use XQueries versus a series of SQL queries. The result of an XQuery is an XML-based result, rather than the tabular result returned by SQL queries. Obtaining a result that is already in XML format significantly reduces the amount of coding required to build complex services and greatly simplifies data access. It also provides a performance boost as marshalling and unmarshalling of data is not required.

XQuery statements can be combined with a data access layer to simplify the process of creating a data access service. Unfortunately not all XQuery implementations are alike, which can prevent realisation of the benefits of streamlined code with this approach. To avoid this, verify that the XQuery implementation that you have chosen supports all of the relational and XML-based data sources that are required. Additionally, perform full-load performance testing to get a sense of how robust the scalability of the implementation is.

Figure 6: XQuery is simpler

The big picture

The future holds many exciting possibilities for data access in SOA environments. Emerging standards such as Service Data Objects (SDO) will transform data access services into robust data application services (DAS) which IT organisations will regard as key to their SOA environment. While not yet mature, some DAS solutions are already being discussed by database vendors and other third party vendors. Unfortunately none of these solutions offers a complete solution for IT organisations today. IT organisations would do well to focus on making data access a first class citizen of their SOA environment today by implementing each of the best practices presented here while keeping an eye on emerging standards and products that can form the basis of a strong data access infrastructure tomorrow.

---

Mike Frost leads the strategic marketing efforts for a variety of DataDirect Technologies’ data access technologies. He has more than 7 years of experience working with enterprise data connectivity and is currently involved in the development of data connectivity components including ODBC, JDBC, ADO.NET, and XML.