Introduction
This first article of a three-part series presents practical solution patterns that you can implement to govern, enforce, and monitor business service policies. This solution is based on using a Service Level Agreement (SLA) Control File with the IBM WebSphere DataPower SOA appliance (such as XI50, XI52, XB60, XB62, XG45, and so on), which is hereafter referred to as "the appliance".
The article demonstrates the various types of business service policies that you can manage using this solution's framework and how to easily associate these policies to service consumers and service providers for individual enforcement. It details the implementation of a simple policy registration and binding solution (based on an XML SLA Control File) and a policy enforcement point (the appliance) with the ability to manage different versions of a service, its associated SLA policies, endpoint routing, and any data mediation (as needed).
You can apply this solution pattern to any type of service that the appliance must proxy and protect: XML, REST or Web-Services (SOAP). Therefore, this article describes a way to configure this solution by using the most widely used DataPower service gateways: Multi-Protocol Gateway and Web-Service Proxy.
Figure 1 presents the different components of the solution pattern.
Figure 1. Components of the SLA policy enforcement solution
Part 2 in this series discusses the introduction of WebSphere Service Registry and Repository (WSRR) to this solution pattern and how objects and metadata in the registry are used to replace the SLA Control File. This second approach provides for better integrated SOA governance and management of SLA policies and better adoption and reuse of enterprise-wide policies.
Part 3 documents the WSRR solution's DataPower configuration artifacts and provides step-by-step testing and validation instructions.
Finally, in the Download section, you can download the appliance configuration archive for a DataPower XI50/XI52 appliance implementing solution pattern, a sample application to test it, and any other related files.
The DataPower services described in this article (including
EchoService
) can be imported on a DataPower appliance using the following zip archive, SLAEnforcement_Part1_DataPower_Config_Export.zip.
Other XML files of the article are found in the SLAEnforcement_Part1_ReferenceMaterial.zip zip archive. It consists of the following files:
- The XSD schema files. The XSD schema of the SLA Control File is also provided:
ibm.devworks.GWParams.xsd.
- Request files posted on the DataPower services.
- SLA control files used on the gateways or to illustrate examples provided in the article.
- XSL stylesheet used for data mediation:ibm.devworks.productTransform.xsl.
- Errors parameter file (
ibm.devworks.error.xml
): You can upload This file in the local file of the DataPower domain, which contains the service gateways. It defines custom error codes and messages returned to HTTP client in case of an error.
SLA, SLD, and SLM policies
First, you must define common terminology used when building SOA policy solutions. This section presents different terms, which are often used while dealing with policy and governance of business services. The most important are the following:
- Service Level Agreements (SLA)
- Service Level Definitions (SLD)
- DataPower configuration artifacts (such as Service Level Management policies)
What is a Service Level Agreement?
A Service Level Agreement (SLA) is a negotiated and formally defined agreement between two parties, where one is the (service) consumer and other is the (service) provider.
It records a common understanding about the following areas of an agreement:
- Services
- Priorities
- Responsibilities
- Guarantees
- Warrantees
The SLA management process includes the following activities:
- SLA contract definition (basic schema with the quality of service parameters)
- SLA negotiation
- SLA enforcement, according to defined policies and metrics
- SLA monitoring
The concept of SLAs is important to DataPower because SOA appliances are regularly used as a Policy Enforcement Point (PEP) in SOA solutions.
There is no industry-wide standard today for expressing SLA agreements within SOA components (such as Policy Administration Point and Policy Enforcement Point). Therefore, custom solutions are common and can be non-portable between service gateways.
What is a Service Level Definition?
A Service Level Definition (SLD) defines the capabilities of a provider to deliver a service in compliance with conditions that its owner has defined as required for protecting the service endpoint.
An SLD is not negotiated as it is defined solely by the service provider owner. It is the responsibility of a service provider to deliver a level of service that meets all terms and conditions defined in its SLD. DataPower performs the enforcement of the SLD policy requirements and acts as a protector of the backend service endpoint.
The SLD management process includes the following activities:
- SLD definition (level and quality of service, security, and so on)
- SLD enforcement, according to defined policies
- SLD monitoring
What is a Service Level Management policy?
A Service Level Management (SLM) policy is a DataPower configuration artifact that you can configure to enforce an SLA or SLD policy requirement. This DataPower configuration policy is exclusively based on DataPower configuration property details and is usually created by the Policy Developer responsible for translating SLA and SLD policies requirements into operational configuration in DataPower.
Proposed solution pattern: A Service Gateway SLA Control File
This section discusses the proposed solution pattern that uses a Service Gateway SLA Control File to describe services and their associated SLA and SLD policies details. For the purpose of simplification, we refer to this XML parameterized file as an "SLA Control File", even though it is used to enforce both SLA and SLD policies.
Why such a file?
Many times an enterprise policy administration product (such as IBM WebSphere Registry and Repository) is not available to service deployers, where the product can centrally govern and deploy service policies (such as SLA and SLDs) to policy enforcement products (such as IBM DataPower appliances). Therefore, you must adopt a more localized and practical solution, which can still provide a balance between policy management and policy agility for enforcing SLA and SLD business requirements.
The XML control file approach can serve as an "easy to deploy" solution that includes simple to update file-based pattern and management process. The control file can quickly and simply describe the following service and SLA properties and relationships:
- Services (used as a top-level service registry)
- Service versions (used for version mediation)
- Service endpoint (used for dynamic routing)
- SLA related to a consumer (used for SLM policy enforcement)
- SLD related to a provider (used for SLM policy enforcement)
- Schema URLs (used for message data validation)
- Transformation (used for message content translation or any other possible customizable policy enforcement actions needed by service gateway solution requirements)
What can be done with this control file?
You can use the control file to execute any action deemed necessary for a specific service gateway solution. In the solution provided with this article, we have selected the most common actions applied with DataPower as a Policy Enforcement Point, such as:
- SLA Check Enforcement: This action consists of verifying that an SLA (contract) exists between a consumer and a provider.
- SLA SLM Enforcement: This file contains the name of the SLA SLM Policy that is configured on the DataPower device and used in the relation between a consumer and a service provider.
- SLD SLM Enforcemen5: This file contains the name of the SLD SLM Policy that is configured on the DataPower device and used in the relationship between a service version and an endpoint.
- SLD Routing Enforcement: This file contains the endpoint URLs of the different versions of a service.
- SLD Validation Enforcement: The URLs of the schemas are provided for each service endpoint.
For different enforcement action requirements, the provided solution control file and configuration can be extended to include additional enforcement support.
Structure and sections of the SLA Control File
The SLA Control File is a simple, robust, and efficient solution that you can implement very quickly. It does not replace a proper and formal registry solution (such as WebSphere Service Registry and Repository), but it can definitely be the first step to adopt SOA Governance best practices.
The XML grammar used on the SLA Control File uses the terms defined in the WSSR's Governance Enablement Profile (GEP). This choice has been made to facilitate the possible future migration path from the SLA Control File to WSRR's GEP runtime governance.
Figure 2 provides a view of the SLA Control File grammar structure.
Figure 2. SLA Control File: XML Schema
The SLA Control File is composed of a service catalog (/
BusinessServices
) that can be qualified by the domain and version information. The service catalog contains multiple services (/BusinessServices/BusinessService)
. Every business service (identified by a unique name) supports one or more service versions (ServiceVersion
).
Each service version is identified by a unique URI. Individual service versions contain a single SLD along with any number of SLAs. The SLD defines a service endpoint (
EndPoint
) and a specific SLM Policy to protect the backend server. A service endpoint must include the required URL, whereas other elements are optional.
Table 1 shows the details of the different elements of the SLA Control File.
Table 1. Elements of the SLA Control File
Element | Description |
---|---|
/BusinessServices | The catalog of business services for which SLA and SLD must be enforced. |
/BusinessServices /@domain | The DataPower domain on which the SLA Control File is applied. |
/BusinessServices /@version | The current version of the file or its creation date. |
/BusinessServices /BusinessService | A business service instance. |
/BusinessServices /BusinessService /@name | Name (identifier) of a business service. |
/BusinessServices /BusinessService /ServiceVersion | A specific business service version instance. |
/BusinessServices /BusinessService /ServiceVersion /@uri | Service version URI. Every service version must have a unique URI. |
/BusinessServices /BusinessService /ServiceVersion /SLAs | Unbounded list of SLAs that must be enforced when the business service is requested. |
/BusinessServices /BusinessService /ServiceVersion /SLAs /SLA | Declaration of an SLA instance. |
/BusinessServices /BusinessService /ServiceVersion/SLAs /SLA /@consumerId | Identifier of the consumer associated with SLA. This value is required. |
/BusinessServices /BusinessService /ServiceVersion /SLAs /SLA /@contextId | Identifier of the context associated with SLA. This The value is optional. |
/BusinessServices /BusinessService /ServiceVersion /SLAs /SLA /Policy | Name of the SLA SLM Policy, which must be enforced for a specified consumer. |
/BusinessServices /BusinessService /ServiceVersion /SLD | SLD instance declaration, which must be enforced for service version. |
/BusinessServices /BusinessService /ServiceVersion /SLD /Policy | Name of the SLD SLM Policy, which must be enforced when service version is requested. |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint | Declaration of the service version endpoint. |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /URL | Endpoint URL of the service version backend. |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Request | This optional element is used to define additional processing, which can be applied during transaction request processing. Additional processing actions consist of validations and transformations, but the solution implementation can be extended to support more. |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Request /InputSchemaURL | This optional element references a schema or WSDL file (in case of a SOAP message validation), which is used to validate the incoming message during request processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example, local:///schema.xsd ). |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Request /XForm | This optional element references an XSLT stylesheet, which is used to execute a transformation (for incoming message translation or any other application) during request processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example,local:///xform.xsl ). |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Request /OutputSchemaURL | This optional element references a schema or WSDL file (in case of a SOAP message validation), which is used to validate the outgoing message during request processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example, local:///schema.xsd ). |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Response | This optional element is used to define additional actions, which can be applied during response processing, such as message validations and transformation. |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Response /InputSchemaURL | This optional element references a schema or WSDL file (in case of a SOAP message validation), which is used to validate the incoming message during response processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example, local:///schema.xsd ). |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Response /XForm | This optional element references an XSLT stylesheet, which is used to execute a transformation (for incoming message translation or any other application) during response processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example,local:///xform.xsd ). |
/BusinessServices /BusinessService /ServiceVersion /SLD /EndPoint /Response /OutputSchemaURL | This optional element references a schema or WSDL file (in case of a SOAP message validation), which is used to validate the outgoing message during response processing. Use the local:/// prefix if the file is local to the DataPower domain specified in /BusinessServices/@domain (for example, local:///schema.xsd ). |
You can apply the input and output schemas and transformations on requests and responses.
You can attach a set of SLAs to the service version to associate consumers to a specific service version. Each SLA instance represents a specific consumer. You can define an SLA SLM Policy for each SLA instance, and therefore enforce it for a specific consumer and provider pairing.
A required "contextId" and optional "consumerId" attributes are defined at the SLA level and can uniquely identify a specific SLA within a Service Version instance. You can reuse identical context and consumer IDs to access different versions of the same service.
The SLA Control File and its components are defined for a specific domain (using the @domain attribute on the <BusinessServices> element) and a specific version (using the @version attribute on the <BusinessServices> element).
Figure 3 shows an example of a SLA Control File, describing two service versions (XML and SOAP) of the same Business Service (
MathServer
), where:- The XML service @uri is
/xml/mathserver
. - The SOAP service @uri is
/mathserver
.
Figure 3. SLA Control File sections
Processing of the SLA Control File
The SLA Control File is consumed by custom XSL stylesheets, executed on both request and response processing rules of the DataPower services. The
document()
function is used to access nodes in an XML document (the XML Gateway Control File), which must be well-formed and valid.
The following algorithm shown in Figure 4 describes the logic of these custom templates (consumer identifier and client URI are input data).
Figure 4. Processing of the SLA Control File
Example applications of the SLA Control File
This section presents some examples of using the SLA Control File in different scenarios.
Figure 5 shows the different nodes that may be created using the SLA Control File.
Figure 5. Example of multiple service versions
Optional nodes are:
- Request
- Response
- InputSchema
- OutputSchema
- XForm
Example 1. One business service with one service version
In this first example, you can see components of an SLA Control File shown in Figure 6, which contains data related to one business service (
MathServer
) with only one version of this service (/services/MathServer/V1
).Figure 6. Example 1 - SLA Control File
The service is named "MathServer" and its service version URI is "/services/MathServer/V1".
Two consumers can use this service. Their consumer identifiers are:
- APPLICATION1 with a Silver contract defined through a specific SLA
- APPLICATION2 with a Gold contract defined through a specific SLA
The endpoint of the service is "http://demoserver:9080/services/MathServer/V1".
Neither schemas nor transformations have been defined at the endpoint level of the business service.
This service control file performs the following actions on the DataPower request processing rule:
- SLA check
- SLA SLM enforcement
- SLD SLM enforcement
- SLD routing
The SLA Control File linked to Example 1 is found in the SLAEnforcement_Part1_ReferenceMaterial.zip archive:
ibm.devworks.GWParams_One-Service-Version_1.0.xml
.Example 2. One business service with two service versions
In this second example, the SLA Control File contains data related to one business service with two versions of a service, as shown in Figure 7.
Figure 7. Example 2 - SLA Control File
The service in version V1 is accessed through the URI "/services/MathServer/V1", whereas the service in version V2 is accessed through the URI "/services/MathServer/V2".
Each service version has its own endpoint and SLD (SLD1 for service in version V1, SLD2 for service in version V2, and different consumers can use the two versions of the "MathServer" business service:
- APPLICATION1 consumes version V1 with a level of service defined in SLA Bronze.
- APPLICATION3 consumes version V2 with a level of service defined in SLA Silver.
- APPLICATION4 consumes version V2 with a level of service defined in SLA Gold.
Neither schemas nor transformations have been defined on the two endpoints of the business service.
This service control file performs the following tasks on the DataPower request processing rule:
- SLA check
- SLA SLM enforcement
- SLD SLM enforcement
- SLD Routing Example 3: one business service with two service versions (versioning use case, where the backend is shared and data format must be mediated)
The SLA Control File linked to Example 2 is found in the SLAEnforcement_Part1_ReferenceMaterial.zip archive:
ibm.devworks.GWParams_Two-Service-Versions_1.0.xml
.
You can use the SLA Control File to implement a service policy solution that mediates between different data versions supported by a service and validates the request and response message before and after the data translation step.
Figure 8. Example 3 - SLA Control File
In Figure 8, the service in version V1 is accessed through the URI "/services/MathServer/V1", whereas the service in version V2 is accessed through the URI "/services/MathServer/V2".
As the same service endpoint is accessed from the two service versions, SLD1 and SLD2 must be identical. This means SLD1 and SLD2 reference the same DataPower SLM Policy.
In case MathServer service is accessed through service version "V1", the gateway must perform message content adaptation from a message in version "V1" format into the message in version "V2" format during request processing. Therefore, a transformation (Xform1-2) is defined at the Request level to complete this adaption.
A validation of the message in version "V1" is executed before message transformation. Once transformed, the message version "V2" is also validated before being sent to the backend application.
The same adaptation (from version "V2" to version "V1") is also defined while processing the response and a validation of message before and after transformation is executed during response processing as well.
In case MathServer service is accessed through service version "V2", a validation messages is defined and executed while processing request and response.
This service control file performs the following tasks on the DataPower request processing rule:
- SLA check
- SLA SLM enforcement
- SLD SLM enforcement
- SLD routing
- SLD message validation (input)
- SLD message transformation (if necessary)
- SLD message validation (output)
The DataPower response processing rule:
- SLD message validation (input)
- SLD message transformation (if necessary)
- SLD message validation (output)
The SLA Control File linked to Example 3 is found in the SLAEnforcement_Part1_ReferenceMaterial.zip archive:
ibm.devworks.GWParams_Service-Versioning_1.0.xml
.Uploading an SLA Control File to a DataPower device
This section discusses different possibilities to upload an SLA Control File on a DataPower appliance and how to flush the XML document cache related to this file on a specific domain.
This is important because the original SLA Control File might be managed off the appliance. When a modification is made to this file, it is necessary to upload it onto different appliances.
The document cache must be flushed after each upload or update of the SLA Control File since a stale version might be cached and will be re-fetched into the cache during the processing of the next transaction.
Here are the different methods to upload an XML file onto an appliance and to flush the XML document cache:
- Manual upload and manual cache flushing.
- SOAP Management (SOMA) API through the XML management interface of a DataPower SOA appliance (to upload the file and flush the cache).
- Command Line Interface (CLI) wrapped into a script (to upload the file and flush the cache).
Listing 1 and Listing 2 are samples of SOMA request messages used to upload a file on a DataPower appliance and to flush a document cache using the same approach.
Listing 1. Sample SOMA call to upload a file
<?xml version="1.0" encoding="UTF-8"?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <dp:request domain="someBank-Sample" xmlns:dp="http://www.datapower.com/ schemas/management"> <dp:set-file name="local:///Testfile"> Base64_encoded_File </dp:set-file> </dp:request> </soapenv:Body> </soapenv:Envelope>
Listing 2. Sample SOMA call to flush an XML document cache
<?xml version="1.0" encoding="UTF-8"?> <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"> <env:Body> <dp:request domain="someBank-Sample" xmlns:dp="http://www.datapower.com/schemas/management"> <dp:do-action> <FlushDocumentCache> <XMLManager>someBankXML</XMLManager> </FlushDocumentCache> </dp:do-action> </dp:request> </env:Body> </env:Envelope>
Note that in this case, the XML management interface accessed through the default domain of the appliance must be enabled and must support the SOMA format.
Creating the SLM policies (SLA and SLD) referenced in an SLA Control File
You can use both SLA and SLD policies in an SLA Control File by using three different approaches:
- Manual creation: The SLM policies are created manually. In this case, each SLM policy defined in the same domain must have a unique name.
- Off-line creation process: The SLM policies are created during a SLA Control File upload. This requires implementation of a specific service (an XML firewall for instance) that parses the SLA Control File to retrieve names of the different SLA and SLD and to configure these policies. The name of these SLA and SLD policies must contain the required information, which is used to create DataPower SLM policies.For instance, the SLA Policy "SLA_5MessagesPer10Seconds_Throttle" contains information to create the right SLM Policy:
- SLA: This information indicates the SLM Policy is enforced per consumer, which is not the case for an SLD Policy.
- 5 Messages Per 10 Seconds: This information defines the threshold level (5), threshold interval length (10 seconds), and threshold interval type (moving).
- Throttle: This information defines the SLM action (throttle), which is executed when the threshold has been reached.
Details of the SLM Policy main configuration and statement are presented in Figure 9 and Figure 10.Figure 9. SLM Policy configuration
Figure 10. SLM Statement definition
- On-line creation process: The SLM policies are created when needed before an SLA and SLD enforcement. In this case, it is important to test the SLM Policy existence before creating and using it on the fly. The names of the SLM policies used in the SLA Control File must contain required information as mentioned in Step 2 above.
To avoid configuration errors, you must use off-line and on-line creation processes when an important number of SLM policies implementation has to be performed. You can also use these processes to check existence of files (XSD schemas, XSL stylesheets) that can be referenced in the SLA Control File.
Manual creation of SLA and SLD SLM policies has been used for this article. The process for off-line and on-line creation of SLM policies is outside the scope of this article.
Using SLA control files with DataPower hosted services
The SLA Control File is used to define properties of your business services, in terms of SLA, SLD, validation, routing, and transformation. This section describes the minimal service configuration that is required to consume an SLA Control File to perform the following:
- SLA check
- SLA and SLD policies enforcement
- Routing
- Content validation and transformation
You can use the SLA Control File on DataPower services to expose and protect RESTful, XML, and SOAP Web-Services, which are defined in the control file. Therefore, the SLA Control File can use two types of DataPower:
- Multi-Protocol Gateway
- Web-Service Proxy
The next sections detail a generic configuration, which consumes the XML Gateway Control file.
There are two important data required during processing:
- Consumer identifier (Consumer ID): This information is required to identify the consumer application. You can transmit the Consumer ID using various mechanisms:
- "X-CONSUMERID" HTTP Header
- Basic Authentication: In this case, the username is the consumer ID.
- Client certificate: In this case, the client DN extracted from the client certificate is the consumer ID.
- Service identifier: This information is required to identify the requested service version. The URI is used to identify a service version.
Basic authentication is the method used to transmit the consumer ID in the examples of this section.
An AAA Policy is configured on the first Multi-Protocol Gateway. This policy extracts identity from a basic authentication header and uses the basic authentication's username as the consumer ID. Authentication is performed through an AAA Info file and any authenticated consumer is authorized to consume the business service. Therefore, the SLA policies must reference a credential class with the following properties shown in Figure 11 and Figure 12.
Figure 11. SLM Credential Class based on extracted identity
If the consumer ID is retrieved from the X-CONSUMERID HTTP header and no AAA Policy is necessary, then the credential class properties of the SLA policies must be defined with values presented in Figure 12.
Figure 12. SLM Credential Class based on HTTP Request Header
Multi-Protocol Gateway (Web applications, RESTful services)
This section describes the generic DataPower configuration that is required to consume an SLA Control File when XML services must be exposed, protected, and controlled.
The solution is based on two Multi-Protocol Gateways:
- The first gateway (mpg_xml_sla-proxy) is used to perform the following tasks:
- SLA check controls if there is a relation between a service consumer and a provider.
- Consumer request validation
- Request message transformation
- Validation of the transformed message
- SLA enforcement (per consumer)
- The second gateway (mpg_sld) is used to perform the following tasks:
- SLD enforcement
- Routing
- Service response validation
- Service response message transformation
- Validation of the transformed message
Note: The need to implement two gateways is due to the fact that, as of the current firmware version 4.0.2, it is not possible to configure more than one SLM Policy on the same processing rule. This restriction might change in the future, so the solution configuration could be adapted to use a single gateway (check your target firmware documentation for details).
Figures 13 shows an overview of this solution.
Figure 13. Multi-Protocol Gateway to consume an SLA Control File
Main properties of the mpg_xml_sla-proxy Gateway
This section presents the main properties of the first Multi-Protocol Gateway. The Request Type is defined as XML, as shown in Figure 14.
Figure 14. Request Type
If the REST services must be exposed and controlled, you have to use the "Non-XML" request type.
To improve internal communications between the two Multi-Protocol Gateways, the Persistent Connections property is set to "Off". Moreover, as backend errors are not processed by the first Multi-Protocol Gateway, the Process Backend Errors property is set to "Off", as shown in Figure 15.
Figure 15. Advanced settings
Main properties of the mpg_sld Gateway
This section presents the main properties of the second Multi-Protocol Gateway. The Request Type is defined as a non-XML as different types of SLA gateways (REST, XML, SOAP) can preceed the mpg_sld gateway, as shown in Figure 16.
Figure 16. SLD enforcement
Request processing rule
This section describes the generic request processing rule of the two Multi-Protocol Gateways used to enforce SLA and SLD as defined in the SLA Control File.
Figure 17 presents the request processing actions of the two gateways.
Figure 17. Request processing rules details
The SLA check executes the following treatments:
- Retrieve the consumer identifier from the basic authentication header or client certificate (DN) or from the "X-CONSUMERID" HTTP header.
- Retrieve the URI, which defines the service version in the SLA Control File.
- Perform the SLA check using the SLA Control File to verify that there is a contract between the consumer and the service provider. If no SLA is found, the message is rejected.
- Retrieve other information, such as:
- Validation schema used to validate the incoming XML message.
- XSL transformation used to translate the incoming XML message.
- Validation schema used to validate output of the previous transformation.
- The SLA Policy that must be enforced.
The first conditional action (Validation 1) is used to test if a validation of the incoming XML message must be executed or not.
The transformation action is used to adapt the incoming XML message or to execute any required treatments. If no transformation is defined in the SLA Control File, then the identity (
store:///identity.xsl
) transformation is executed.
The second conditional action (Validation 2) is used to test if a validation of the transformed message must be executed or not. Once the SLA Policy is retrieved, an SLM Policy is used to enforce it (SLA enforcement).
SLD retrieval is performed on the second Multi-Protocol Gateway. The endpoint URL is also retrieved during execution of this action. Once the SLD Policy is retrieved, it is possible to enforce it (SLD enforcement).
Dynamic routing is finally executed (routing) to post the XML content to the appropriate service provider, as defined in the SLA Control File.
SLA check and SLD retrieval processing actions use the SLA Control File. The content of this file is cached using an XML Manager with a dedicated Document Cache Policy, as shown in Figure 18.
Figure 18. Document Cache Policy to cache an SLA Control File
The URL match expression used on this XML Manager allows caching content of the SLA Control File as well as an AAA Info File.
You can add an optional "Set Variable" action at the beginning of the request processing rule of the mpg_sld Multi-Protocol Gateway, as shown in Figure 19.
Figure 19. Using a Set-Variable action to specify an SSL profile
This action is used to specify the SSL profile (such as
MySSLProfile
) that must be used in case an HTTPS connection has to be established between the mpg_sld Gateway and backend services.
Figure 20 shows the configuration of this optional "Set Variable" action.
Figure 20. Details of the Set-Variable action
The name of the variable that must be set is
var://context/wsxml/default-ssl-profile
.Response processing rule
This section describes the generic response processing rule of the two Multi-Protocol Gateways used to enforce the SLA and SLD as defined in the SLA Control File. Figure 21 shows the response processing actions of the two gateways.
Figure 21. Response processing rules details
The response processing action executed on the mpg_sld gateway is responsible for retrieving the following elements:
- Schema used to validate the server response.
- XSL transformation used to translate the server response (XML).
- Schema used to validate the result of the previous transformation, before the message is sent back to the consumer application.
The first conditional action (Validation 1) is used to test if a validation of the XML response message must be executed or not.
The transformation action is used to adapt the XML response message or to execute any required treatments. If no transformation is defined in the SLA Control File, then the identity (
store:///identity.xsl
) transformation is executed.
The second conditional action (Validation 2) is used to test if a validation of the transformed message must be executed or not.
The mpg_xml_sla-proxy Gateway is in charge of performing optional tasks like the following:
- Business or technical information asynchronous logging
- Record monitoring data using the DataPower WS-Management agent
The response processing action uses the SLA Control File.
Error processing rule
This section describes error processing rules of the two Multi-Protocol Gateways used to enforce SLA and SLD. Figure 22 shows the error processing rules and actions of the two gateways.
Figure 22. Error processing rules details
The same error handling is performed on the two gateways. The mpg_sld gateway does not create any error content, it just sends error messages and error codes (using respectively, DP.ERROR and DP.RC HTTP headers) to the mpg_xml_sla-proxy gateway. This second gateway is responsible for creating a SOAP Fault or for sending dedicated status code and reason phrase. If an error occurs during execution of the mpg_xml_sla-proxy gateway, the processing rule deals with the error directly.
Web-Service Proxy (SOAP/Web services)
This section describes the generic DataPower configuration that is required to consume an SLA Control File when SOAP Web-Services must be exposed, protected, and controlled.
The mpg_sld Multi-Protocol Gateway must also be used when a Web-Service Proxy has been configured to expose a Web-Service.
Note: The need to implement two gateways is due to the fact that, as of the current firmware version 4.0.2, it is not possible to configure more than one SLM Policy on the same processing rule. This restriction might change in the future, so a solution configuration might be adapted to use a single gateway (check your target firmware documentation for details).
The solution is based on one Web-Service Proxy and one Multi-Protocol Gateway, as shown in Figure 23:
- The Web-Service Proxy (wsp_sla-proxy) is used to perform the following tasks:
- SLA check controls if there is a relation between service consumer and provider.
- Consumer request validation is performed using validation capabilities of a Web-Service Proxy.
- Request message transformation
- Validation of the transformed message
- SLA Enforcement (per consumer)
- The second gateway (mpg_sld) is used to perform the following tasks:
- SLD enforcement
- Routing
- Service response validation
- Service response message transformation
- Validation of the transformed message before it is sent back to the Web-Service Proxy
Figure 23. Web-Service Proxy to consume an SLA Control File
Here is an overview of this solution. The advanced properties (see Figure 24) of wsp_sla-proxy are identical to those used when a Multi-Protocol Gateway is used to expose an XML service:
- No persistent connections between the Web-Service Proxy and the Multi-Protocol Gateway to improve internal communications.
- No HTTP error processing, as errors are processed on the mpg_sld Multi-Protocol Gateway.
Figure 24. Advanced settings of Web-Service Proxy
These properties are configured in the advanced settings of the Web-Service Proxy as shown in Figure 24.
The request, response, and error processing rules are the same as those detailed in the previous section related to the Multi-Protocol Gateway configuration. The only differences are the following:
- It is possible to define the request and response and error processing rules at different levels on a Web-Service Proxy, as shown in Figure 25.
Figure 25. Processing rules of the Web-Service Proxy based solution
As you can see, the processing rules are defined at the proxy level.
- The validation actions on the request and response processing rules are not based on the Schema URL, but the WSDL URL. Figure 26 shows a configuration example of such a validation action.
Figure 26. Validate action based on WSDL URL
When dealing with SOAP Web-Services in the SLA Control File, you must reference WSDL files and not XSD schemas, using
<inputSchemaURL>
and <outputSchemaURL>
elements.Example application using a Multi-Protocol Gateway (productInfo)
This section discusses the processing of a Multi-Protocol Gateway used to control an XML service (ProductInfo). This gateway implements the controls defined through an SLA Control File.
The ProductInfo XML service acts as an echo service. It receives an XML document (getProductInfoRequest.xml) and sends it back to the consumer application.
SLA Control File
The SLA Control File (
ibm.devworks.GWParams_1.1.xml
) used in the next example of this article contains the following information shown in Figure 27.Figure 27. SLA Control File to enforce the ProductInfo service
There is only one Business Service defined in the SLA Control File, ProductInfo, which is a simple echo service used in this article.
For this Business Service, there is only one Service Version,
URI: /xml/
productInfo
.
The XML service is accessed using a:
- Consumer identifier App1 with SLA Silver. There are no more than two messages per 10 seconds and per authenticated consumer. The throttleaction is executed if the threshold has been reached.
- Consumer identifier App2 with SLA Gold. There are no more than five messages per 10 seconds and per authenticated consumer. The throttleaction is executed if the threshold has been reached.
A consumer application with an identifier (consumerId) different from App1 or App2 cannot consume the ProductInfo service. Indeed, the consumer's messages are rejected during SLA check in this case.
The ProductInfo service has its own SLD and endpoint URL. The SLD specifies no more than ten messages per 60 seconds. The throttle action is executed if the threshold has been reached.
During the request processing, the XML message must be validated using version 1.0 of the "simpleSchema" XSD. Then, the XML message is transformed and validated using version 1.1 of the "simpleSchema" XSD. The transformation consists of decoding a base64 encoded string and modifying the XML structure of the XML document. During response processing, a validation must be performed using version 1.1 of the "simpleSchema" XSD.
Request processing
This section provides details of the request processing. Consumer identifiers are extracted from the basic authentication header. This token is also used to authenticate consumers on the DataPower appliance using an AAA Policy.
The authentication step of this policy is based on an AAA Info file. Therefore, you need to transmit the basic authentication header when requesting the mpg_xml_sla-proxy Multi-Protocol Gateway.
The message that is posted on the gateway is getProductInfoRequest.xml as shown in Listing 3.
Listing 3. getProductInfoRequest.xml
<?xml version="1.0" encoding="utf-8"?> <product-info> <product-id>XI52</product-id> <brand>WebSphere DataPower</brand> <encoded-description>RGF0YVBvd2VyIFhJNTIgU09BIEFwcGxpYW5jZQ ==</encoded-description> <benefits>Security;Integration;Performance</benefits> </product-info>
Request processing on mpg_xml_sla-proxy
This sections describes details the request processing rule on the mpg_xml_sla-proxy gateway.
On the mpg_xml_sla-proxy gateway probes, you can see that the routing of the request is made on the second Multi-Protocol Gateway (mpg_sld), which is listening on port 9000 (see Figure 28). This gateway is configured to be only accessible on the loopback interface (dp.loopback is the host alias of 127.0.0.1).
Figure 28. Viewing mpg_xml_sla-proxy transaction in service probes
Figure 29 shows the incoming XML message that has been posted and the processing actions, which are executed on the request processing rule.
Figure 29. Request message in probes
Figure 30 shows the incoming HTTP headers. You can notice the presence of a Basic Authentication header with the value of "App2:foobar" encoded in base 64, which is required by the AAA Policy identity extraction and authentication steps.
Figure 30. Request headers in probes
After the SLA check action, you can see that a specific XML content has been created. This content provides validation data (schema to be used) to the conditional actions shown in Figure 31.
Figure 31. Conditional schema validations on request processing
After the SLA check, specific context variables are created as shown in Figure 32.
Figure 32. Context variables values
These context variables are used by different actions to dynamically execute specific actions.
For instance, the variable "var://context/wsxml/sla" is used by the SLM action to enforce the right SLA Policy (SLA_5MessagesPer10Seconds_Throttle). The XSL transformation, which must be applied on the XML request message, is defined through the "var://context/wsxml/xform-req" multi-step variable. The consumer identifier App2 is set into the "var://context/wsxml/consumerId" context variable.
The transformed message is presented as shown in Figure 33 (before being validated).
Figure 33. Request message after transformation
The ibm.devworks.productTransform.xsl template (Listing 4) looks specifically for two different patterns:
- When an <encoded-description> tag is encountered, it changes it into a <description> tag, and then decodes the original tag's value. The dp:decode() function is a DataPower extension function.
- When a <benefits> tag is encountered, it uses the str:tokenize() function to tokenize the list of benefits (delimited by semicolons) into a small XML tree.
Listing 4. Partial listing of file productTransform.xsl
<xsl:template match="encoded-description"> <description> <xsl:value-of select="dp:decode(.,'base-64')"/> </description> </xsl:template> <xsl:template match="benefits"> <xsl:variable name="benefits" select="str:tokenize(.,';')"/> <benefits> <xsl:for-each select="$benefits"> <benefit><xsl:value-of select="."/></benefit> </xsl:for-each> </benefits> </xsl:template>
Before the XML content is sent to the mpg_sld gateway, the SLA Policy is enforced as shown in Figure 34.
Figure 34. SLA enforcement in probes
Request processing on mpg_sld
This section describes details of the request processing rule on the mpg_sld gateway.
On the mpg_xml_sld gateway probes shown in Figure 35, you can see that the routing of the request is made on the ProductInfo service, using the URL
http://demoserver:5080/ProductInfo.
Figure 35. Viewing mpg_sld transaction in service probes
Figure 36 presents different actions executed on the request processing rule of the mpg_sld Multi-Protocol Gateway.
Figure 36. Processing actions of the mpg_sld Gateway
After the first processing action, the SLD Policy that must be enforced is retrieved, as shown in Figure 37.
Figure 37. Context variable for SLD Enforcement
The SLD Policy is then enforced and the XML message is routed to the ProductInfo service using a "SetVar" processing action.
Response processing on mpg_sld
This section describes details of the response processing rule on the mpg_sld gateway.
The ProductInfo service echoes the XML content it receives. Figure 38 presents different actions executed on the response processing rule of the mpg_sld Multi-Protocol Gateway.
Figure 38. Response processing actions of the mpg_sld Gateway
After the first response processing action, you can see that a specific XML content has been created. This content provides validation data (schema to be used) to the conditional actions, as shown in Figure 39.
Figure 39. Conditional validation on response processing rule
Here, you can see that only one validation is performed on the response rule. This is confirmed when displaying the context variables created after the first action and presented in Figure 40.
Figure 40. Context variables used during response processing
The validation is performed using the value of the "var://context/wsxml/input-schema-res" context variable.
As no transformation has been defined in the SLA Control File, the identity transformation is executed.
Then the message is sent back to the mpg_xml_sla-proxy gateway, as shown in Figure 41.
Figure 41. Response message of the mpg_sld Gateway
Response processing on mpg_xml_sla-proxy
This section describes details of the response processing rule on the mpg_xml_sla-proxy gateway.
The response processing rule of the mpg_xml_sla-proxy gateway only consists of a custom template used to log information using an "HTTP Access log like" format.
This action is executed asynchronously as shown in Figure 42 (see the "Asynchronous = on" property).
Figure 42. Asynchronous logging on the mpg_xml_sla-proxy Gateway
Figure 43 shows an extract of the HTTP Access log file created on the appliance by
ibm.devworks.http-accesslog.xsl
.Figure 43. HTTP access logs
Error processing examples
This section provides examples of error processing encountered during the following execution steps, even though other errors may occur during request or response processing:
- SLA check
- SLA enforcement
- SLD enforcement
- Validation of a request message
SLA check
To test the SLA check error processing, you need a consumer identifier that is authenticated by the AAA Policy but that is not referenced as a possible consumer of the ProductInfo service. "App3" is such a consumer.
Figure 44 shows that the request processing is stopped on the SLA check action.
Figure 44. SLA check error
Focusing on the output context of the AAA Policy, you can see that the consumer identifier retrieved from the Basic Authentication header is "App3", as shown in Figure 45.
Figure 45. Consumer identifier in debug probe
As "App3" is not defined in the SLA Control File as a consumer that is allowed to request the ProductInfo service, the message is rejected with specific HTPP headers (
DP.RC
and DP.ERROR
), which identify an error code (403) and a message (Error: SLA Check), as shown in Figure 46.Figure 46. SLA check - Custom error code and message
SLA enforcement
To test the SLA enforcement throttling, it is necessary to exceed the threshold defined in the SLA for a specific consumer when accessing the ProductInfo service. Figure 47 is an overview of the SLA Control File that is used for tests.
Figure 47. SLA Control File of the ProductInfo service
As defined in this SLA Control File:
- Consumer App1 can consume two messages per 10 seconds (Silver SLA Policy)
- Consumer App2 can consume five messages per 10 seconds (Gold SLA Policy)
The test is performed here using Consumer App1. The third message in the same time interval of 10 seconds is rejected, as shown in the probes in Figure 48.
Figure 48. SLA enforcement (throttling)
Figure 49 shows the details of the SLA Policy rejection (throttling).
Figure 49. SLA enforcement context variable
The enforced SLM Policy is defined via the context variable
var://context/wsxml/sla
.
The value of this variable is
SLA_2MessagesPer10Seconds_Throttle
, as defined in the SLA Control File.
As an SLA has been reached, the message is rejected with specific HTTP headers (
DP.RC
and DP.ERROR
), which identify an error code (503) and a message (Error: an SLM Policy has rejected the request), as shown in Figure 50.Figure 50. SLM Policy rejection - Custom error code and message
SLD enforcement
The same error message and code is returned by the Multi-Protocol Gateway when an SLD is reached and exceeded.
In this case, the rejection is performed on the mpg_sld Multi-Protocol Gateway. Figure 51 shows an overview of the SLD Policy defined in the SLA Control File.
Figure 51. SLD section of an SLA Control File
The SLD Policy defined by the service provider does not accept more than ten messages in a time interval of 60 seconds.
The test is performed here using Consumer App2. On the mpg_xml_sla-proxy Gateway, the eleventh message has not been rejected, as shown in Figure 52.
Figure 52. SLA enforcement on the mpg_xml_sla-proxy Gateway
Figure 53 shows the request processing rule of the mpg_xml_sla-proxy Gateway, showing that there is no rejection on the eleventh message.
Figure 53. No rejection on the mpg_xml_sla-proxy Gateway
The eleventh message in the same time interval of 60 seconds is rejected, as shown in the probes of the mpg_sld Gateway in Figure 54.
Figure 54. SLD enforcement (throttling)
Figure 55 shows the SLD Policy rejection (throttling).
Figure 55. SLD enforcement context variable
The SLM Policy that is enforced is defined via the context variable
var://context/wsxml/sld
.
The value of this variable is
SLD_10MessagesPer60Seconds_Throttle
, as defined in the SLA Control File.
Validation of a request message
To complete this test, an invalid message is posted on the gateway,
getInvalidProductInfoRequest.xml, as shown in Listing 5. This file is also named
bad-products.xml
in the provided examples.Listing 5. getInvalidProductInfoRequest.xml
<?xml version="1.0" encoding="utf-8"?> <product-info> <product-id>XC10</product-id> <brand>WebSphere DataPower</brand> <encoded-description>RGF0YVBvd2VyIFhJNTIgU09BIEFwcGxpYW5jZQ ==</encoded-description> <benefits>Security;Integration;Performance</benefits> </product-info>
XC10
is not a valid string for the /product-info/product-id
element. When a request message is not valid, the message is rejected on the mpg_xml_sla-proxy gateway, as shown in Figure 56.Figure 56. XML validation error
The error message in the system logs clearly identifies the validation error, as shown in Figure 57.
Figure 57. XML validation error details
As validation of the request message has failed, the message is rejected with specific HTPP headers (
DP.RC
and DP.ERROR
), which identify an error code (500) and a message (Error: XML Validation), as shown in Figure 58.Figure 58. XML validation error - Custom error code and message
Monitoring SLA Control File metrics
This section describes the DataPower configuration, which you can use to add monitoring capabilities on the response and error processing rules of DataPower services that are configured to work with an SLA Control File.
The aim is to provide traffic monitoring information that you can retrieve by using a client application that implements the WS-Management specification. An example of such a client is the IBM Tivoli Composite Application Manager for SOA DataPower Data Collector, as shown in Figure 59.
Figure 59. Monitoring DataPower SOA Appliance using ITCAM for SOA
This section deals with transaction traffic monitoring. Other types of monitoring, such as health monitoring (generally based on SNMP polling and traps), are not covered in this article.
Tivoli Composite Application Manager data collector deployment steps
To deploy the DataPower data collector in your environment, complete the following general steps:
- Configure your DataPower SOA appliances for monitoring.
- Enable the DataPower data collector.
- Run the
startDPDC
script to start the data collector or configure the data collector to run in the background and start the background task.
Configuring your DataPower SOA appliances for monitoring
This section describes the steps to configure a DataPower SOA appliance for traffic monitoring.
- Assuming that you do not work with an XI52 appliance, you first need to upgrade the DataPower firmware version to 3.7.1.4 or later.
- Configure a user account on the DataPower SOA appliance by using the admin account or any privileged account.
- Enable the XML Management Interface. Access the XML Management Interface configuration from the Control Panel by selecting Objects > Device Management > XML Management Interface. For WS-Management Endpoint, select "on" as shown in Figure 60.
Figure 60. Enabling WS-Management on the XML Management Interface
- Check for additional optional setting. Access the Web Services Agent configuration from the Control Panel by selecting Services > Miscellaneous > Web Services Agent, as shown in Figure 61.
Figure 61. Web-Services Management Agent configuration
- If you want the DataPower SOA appliance to save metrics records when the DataPower data collector is not running, set BufferMode to "buffer". With this setting, the appliance reports the saved metric data when the data collector starts.
- Adjust the values for MaxRecords and MaxMemoryKB.
- If you want the data collector to record message content in addition to summary metrics, change CaptureMode from "faults" to "all-messages".
Configuring DataPower processing rules
In addition to upgrading your DataPower firmware, you might need to add XSL transforms to the action rules in each processing rule for each affected Web-Services Proxy or Multi-Protocol Gateway Proxy object.
If you are monitoring a WS-Proxy without topology support, no transforms are necessary.
If you are monitoring a Multi-Protocol Gateway, you must create certain transforms.
Configuring processing rules for DataPower Multi-Protocol Gateways
At the end of each transaction, typically on response rules and error rules, you must call the
dp:wsm-agent-append()
function by using a custom XSL stylesheet. This function "sends" the monitoring data about the current transaction to the Tivoli Composite Application Manager for SOA data collector.
Configuring processing rules on the SLA and SLD control gateways
This section presents the configuration to monitor the traffic on the mpg_xml_sla-proxy and mpg_sld Multi-Protocol Gateways.
To avoid data monitoring redundancy, you must record traffic monitoring information only on one Multi-Protocol Gateway, mpg_xml_sla-proxy, on the response and error processing rules before any messages are sent back to the consumer application.
Figure 62 presents the response processing rule configuration.
Figure 62. Custom template to submit monitoring data on response processing
A transformation action (Xform) is added before the log action.
This transformation uses the
dp:wsm-agent-append()
function, which adds a record for a request-response message sequence to a Web Services Management Agent. The record contains transaction data that was collected during a request-response message sequence. Each application domain can contain an instance of a configured Web Services Management Agent. Depending on the configuration, a record can include the following processing information about requests and responses:- IP address
- Client identification
- Authenticated identity
- Target service or operation
- Message size
- Processing duration
- Fault messages
- SOAP headers
- Full message
- And so on Figure 63 shows the error processing rule configuration.
Figure 63. Custom template to submit monitoring data on error processing
A transformation action (Xform) is added between the error and log actions. The error action is the first transformation to be processed on the error rule. It is necessary to position the monitoring action after the creation of the error because specific error codes and messages are recorded in the monitoring data.
You can retrieve error codes and messages on the monitoring action in the dedicated context variables, which are set during the error processing action of the rule.
Migration to WSRR
The migration from the SLA Control File approach to WebSphere Service Registry and Repository (WSRR) consists of defining the XML data currently defined in the SLA Control File as WSRR object's metadata properties.
The goal is to not use an SLA Control File. You modify the DataPower configuration artifacts referenced by the file (validate actions, routing, SLA/SLD policies, transformations) instead of fetching such data directly from WSRR using WSRR APIs. The next article in the series will cover best practices and approaches to migrate the pattern described in this article to WSRR.
No comments:
Post a Comment