ESQL best practices

ESQL is a powerfull language allowing to provide fast and flexible development into IIB/ACE/CP4I

The compute node:

When adding an ESQL Node the compute mode is a crucial choice for performance.
Different options are available:

  • Message: This option is often the most commonly used
  • LocalEnvironment: This option is usefull when you need to change the behavior of next nodes
  • LocalEnvironment and Message: This option is usefull when you need to change the behavior of next nodes with the message included
  • Exception: This option is usefull for error management only
  • Exception and Message: This option is usefull for error management only and to log the message or part of the message
  • Exception and LocalEnvironment: This option is usefull for error management only and log the local variable of nodes information
  • All: Try to avoid this option as much as possible.
  • Between Java compute node and ESQL compute node, for performance matter, try to prioritize the ESQL usage.

LocalEnvironment and Environment:

Difference between LocalEnvironment and Environment.

LocalEnvironment is an important option to help next node to know what they will have to do.
Examples:

  • To change the behavior of an HTTP Request node, i will override the variables
    • OutputLocalEnvironment.Variables.HTTP_Method
    • OutputLocalEnvironment.Destination.HTTP.RequestURL
  • To change the destination queue of an MQ Output node, i will override the variable
    • OutputLocalEnvironment.Destination.MQ.DestinationData[1].queueName
  • To change the folder and file to read of a File Input node, i will override the variables
    • OutputLocalEnvironment.Destination.File.Directory
    • OutputLocalEnvironment.Destination.File.Name

Many middleware developers are mistaking the usage of LocalEnvironment and Environment.
Never use the LocalEnvironment to store your own variable, this could lead to a possible bad behavior if the values of a known variable is changed that a node needs to use later. If this happens, the debug becomes quite diffuclt to find and you may waste hours before you find this root cause.

Things to keep in mind when developing:

  • Never use CARDINALITY on a table to check if the table is not empty for a performance matter. CARDINALTY will browse the whole table and return the size of the table. While the EXISTS function will stop at the first occurence.
  • Try to avoid browsing sequences elements with integer index and cardinality, try to user reference pointers and browse next elements with MOVE NEXTSIBLING.
  • When manipulating XML elements, try to use XMLNSC domain instead of XML or XMLNS.
    • Difference between XML, XMLNS, XMLNSC:
    • XML: XML is deprecated and should not be used anymore. It is still present for compatibility with older flows coming from WMB.
    • XMLNS: is a proframmatic parser and considers the XML message as a STRING, validation is not possible either for generation or parse. It is namespace aware. XMLNS will parse the message and will keep the information in memory, so it will use lees cpu and mode memory to handle the message. So XMLNS is more suitable if XPath access is needed.
    • XMLNSC: is the new optimzed way to treat XML messages. It is namespace aware. XMLNSC will parse only sequentially portion of message to finally create a compact tree. This is more optimized to use less memory than XMLNS but may consume a little more CPU.
  • When manipulating json elements, take care about float elements needing scientific representation or not. This must be changed in the Environment variable of the NODE/Integration server.

Shared variables and external variables:

  • A shared variable is a very usefull type of object where the variables keeps living between all the different threads of the application flow.
    • The variable keeps living until the application flow of the integration server is stopped.
  • An external variables is a final object. After assigning a value, it can not be changed anymore.
    • This variable can be usefull for security matter when the values must never be changed.
    • This variable can also be usefull to elevate a user defined property from a flow into the ESQL code.

CICD with IIB / ACE / CP4I

This post is intended to help you understand how an automated build, deployment and test pipeline can be done based on different schemas and videos.

To provide a better understanding, here is a video showing a demo in french (An english version will be done later).

This schema explains the steps to manage the CICD build and deploy parts for IIB, ACE, CP4I.

  • 1. The toolkit already contains maven.
    • You must install the ACE Maven plugin into your m2 repository and upload it also to nexus.
    • Convert your application, policy to a maven project.
    • Insert the necessary information such as ace server binary location, compile esql enabled or not.
    • Build your application, this will create a compressed file.
  • 2. Push your code to GIT.
  • 3. Go to Jenkins, create a Maven project ITEM.
    • Enter the git repository, enable the maven deploy.
    • Your settings.xml with the necessary information.
    • Run your job which will upload the binary to Nexus.
  • 4. Create a new jenkins ITEM.
    • Provide the GroupId, ArtifactID
    • Point the search binaries to nexus
    • The settings.xml should have the information about the ACE Integration server URL, credentials of the Web Admin API.
    • At the run job, select the version to deploy.
    • The job will deploy the version by passing by the web admin API of the integration server.

This schema explains the steps to manage the CICD test part for IIB, ACE, CP4I.

  • 1. To test a message to post to MQ service, create a soapui hermes jms project first and test it manually then create soapui maven project with soapui hermes jms project.
  • 1. To test a message to post to SOAP service, create a soapui project first and test it manually then create soapui maven project with soapui project.
  • 1. To test a message to post to a REST service, create a soapui project first and test it manually then create soapui maven project with soapui project OR create a postman collection first and test it manually then create Newman maven project with your postman collection.
  • 1. !!! Don’t forget to ensure that you have an assert for each reponse you will receive for your services. !!!
  • 1. I suggest to create a maven profile for each environment (for instance DEV/TEST, QA, UAT/ACCEPTENCE, PRODUCTION).
  • 2. Push your Testing project to GIT.
  • 3. Create a jenkins maven ITEM
    • Point to your GIT repository.
    • Add each profile environment available as a combo box.
  • 4. Run the test and wait the job for be in state success or failure.

IBM Integration Bus, App Connect, CP4I

In the section, i will share different topics related to the ESB of IBM.

To get a small historic context, here is the list of versions of the product:

  • MQ Series 1.0
  • MQ Series Integrator 2.0
  • Websphere MQ Integrator 2.1
  • Websphere Business Integration Message Broker 5.0
  • Websphere Message Broker 6.0
  • Websphere Message Broker 7.0
  • Websphere Message Broker 8.0
  • Wesphere Integration Bus 9.0
  • Wesphere Integration Bus 10.0
  • IBM App Connect 11.0
  • IBM App Connect 12.0
  • Cloud Pak for Integration (CP4i) 2020
  • Cloud Pak for Integration (CP4i) 2021