Integration Strategies

Types of integration
Data-level

1. Share data; not behaviour
2. Minimal change (if any) to both source and target systems
3. Can be database or file-based
4. ETL
5. File-data transfer
6. Direct database access
   1. Bypass business logic (may need to duplicate)
   2. Overall data integrity may be compromised
   3. If writable, may lead to data corruption and referential integrity violations
7. Capabilities include
   1. Data transformation
   2. Data validation
   3. Data access
   4. Schema definition
   5. Mapping
   6. Schema recognition

Application-level

1. Share functionality – business logic
2. Based on API
3. Composite applications

Business Process level

1. Share business processes
2. Specified using BPMN
3. Glued together using BPEL(4WS) and BPML
4. Start by defining business processes; then specify logical integration within it
5. Capabilities include:
   1. Rules processing
   2. Business transaction management
   3. Workflow
   4. Orchestration
   5. Event processing
   6. Schedule

Presentation

1. Share views
2. Using a portal
3. Non-invasive

Application Integration methods

1. Web services - Application integration
2. ETL
   1. Data integration
   2. Consolidation of multiple data sources
3. Communication message protocol - E.g. HTTP, TCP/IP, FTP
4. Screen-scraping
5. Program calls - Application integration
6. Direct data access - Data integration
7. File transfer - Unidirectional batch file transfer
8. Human intervention

Enterprise Integration Patterns

Deciding factors

1. Application coupling
2. Intrusiveness
3. Technology selection
4. Data format
5. Data timeliness – Latency
6. Data or functionality required
7. Remote communication – synchronous or asynchronous
8. Reliability

Choices

Order is in increasing sophistication and complexity

1. File transfer –
   1. Simplicity, applications are decoupled (availability does not matter), platform and implementation independence
   2. lacks timeliness (data integrity caused by stale data); may have data semantic dissonance; huge dataset duplicated
2. Shared database –
   1. enforce agreed upon data format and allow speedy implementation, no semantic dissonance; no data replication; more timely
   2. difficult to design shared schema; maybe dependent on software upgrade; allowing writes may cause deadlocks; may result in performance issue; applications are coupled to shared database; no collaboration
3. Remote procedure invocation –
   1. shared functionality, maintain data integrity, no semantic dissonance
   2. tight coupling between applications (availability matters), prone to failure & difficult to maintain without management infrastructure
4. Messaging –
   1. frequent exchanges of small messages, storage schema can be changed, asynchronous with retries; less decoupled than remote procedure; more reliable
   2. semantic dissonance still occurs; difficult to test & debug

Integration Styles and Strategies
These are the few integration styles that are widely used.

When deciding the appropriate style to use, always consider the pros and the cons based on the various criteria. The implication is this: one size doesn't fit all.

The following table summarises the points for consideration:

 

	File Transfer	Maintain Data Copies	Shared Database	Remote Invocation	Messaging	UI Integration
Complexity	low	medium	medium	high	high	medium
Effort to implement	medium	low	low	high	high	low
Data timeliness (latency tolerance)	low	low	high	high	medium	high
Application coupling	low	medium	high	high	low	medium
Push vs. pull	push	pull	pull	push	push	pull
Intrusiveness	low	low	medium	high	medium	medium
Testing ease	easy	easy	medium	medium	difficult	easy
Performance	separate resources	separate resources	deadlock due to sharing	resource needs to be available	separate resources	resource needs to be available
Share data/ functionality	data	data	data	functionality	functionality	functionality
Synchrony	asynch	asynch	asynch	synch	asynch	synch
Integration reliability	medium	medium	high (dependent on database)	low	high (dependent on message bus)	medium
Encapsulation	high	medium	low	high	high	low
Extensible functionality	medium	high	high	low	low	low
Dataset size catered to	big	big	big	small	small	small
Dataset volume transferred	medium	high	low	low	medium	medium
Data integrity	medium	medium	low (if writable)	high	high	high
Implementation	CSV, XML, Fixed width files	SSIS, DTS, ETL, SqlLoad		PL/SQL, T-SQL, W3C web service, .NET remoting, EJB, RMI	MSMQ, JMS, MQ	WebPart, Portlet, WRPS