How to delete a random element from the XML payload with the use of Script mediator in WSO2 ESB

In WSO2 ESB, we can use the Script Mediator manipulate a XML payload. Here I have used JavaScript/E4X for accessing/manipulating the elements.

Example XML payload;

<breakfast_menu>
   <food>
      <name>Belgian Waffles</name>
      <price>$5.95</price>
      <calories>650</calories>
   </food>
   <food>
      <name>Strawberry Belgian Waffles</name>
      <price>$7.95</price>
      <calories>900</calories>
   </food>
   <food>
      <name>Berry-Berry Belgian Waffles</name>
      <price>$8.95</price>
      <calories>900</calories>
   </food>
</breakfast_menu> 

Lets assume we want to remove the last food element (Berry-Berry Belgian Waffles); In this scenario, breakfast_menu is the root element and the set of children elements will be food.

The length of the child elements (food) can be obtained as follows;

var payload = mc.getPayloadXML();
var length = payload.food.length();

Then delete the last element as follows; Here the index of the last element would be length-1

delete payload.cuidInfo[length-1];

Complete Script Mediator configuration would be as follows;

<script language="js">
    var payload = mc.getPayloadXML();
    var length = payload.food.length();
    delete payload.food[length-1];
    mc.setPayloadXML(payload);
</script>

The output of the script mediator would be as follows;

<breakfast_menu>
   <food>
      <name>Belgian Waffles</name>
      <price>$5.95</price>
      <calories>650</calories>
   </food>
   <food>
      <name>Strawberry Belgian Waffles</name>
      <price>$7.95</price>
      <calories>900</calories>
   </food>
</breakfast_menu>

Likewise  we can delete the required elements from the payload.

A Simple way to log time durations in WSO2 ESB

In this post I will explain a simple mechanism to log a time duration in WSO2 ESB. Already there is a great and powerful way of getting statistics of WSO2 ESB which is capable of providing runtime statistical information from sequences, endpoints, proxies and view it through the ESB management console. This will help you to analyze response times, counts and many more performance related options very effectively.

But lets assume, for debugging purposes (or any other purpose) you need a quick and simple way of measuring time durations. For example imagine that you have a proxy service which calls to a back end and returns the response from the back end. Lets say you need to get a rough idea on the time taken to call the back end and get the response. This kind of a scenario can be simply achieved by the use of property mediator, script mediator and a log mediator.

There is a Synapse Message Context Property which can retrieve the current time is milliseconds (i.e. the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC). We can set this value to a property (at the desired point of the message flow) and access it when ever we need.

For example, following is how to set the time to the property 'TIME' and log it later;

<property name="TIME" expression="get-property('SYSTEM_TIME')" scope="default" type="LONG"/>
...
<log>
 <property name="Time : " expression="get-property('TIME')"/>
</log>
...

Like wise we can store the current system time in milliseconds, at desired points in the message flow into different properties. After that we can use a Script mediator to read these values and perform the required calculations.

For example if we have set two time values in propeties 'TIME_1' and 'TIME_2' we can get the difference between those as follows;

<script language="js">
 var time1 = mc.getProperty("TIME_1");
 var time2 = mc.getProperty("TIME_2");
 var timeTaken = time2 - time1;
 print("Time Duration :  " + timeTaken + " ms ");
 mc.setProperty("RESPONSE_TIME", timeTaken);
</script>

Also we can set the result to another property (mc.setProperty("RESPONSE_TIME", timeTaken);) which can be used in the message flow (for example to log the time with the Log mediator).

Therefore this will provide a simple mechanism to log a time in WSO2 ESB. A sample proxy with this kind of a scenario is as follows;

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="SampleTimeProxy"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <property name="TIME_1"
                   expression="get-property('SYSTEM_TIME')"
                   scope="default"
                   type="LONG"/>
      </inSequence>
      <outSequence>
         <send/>
         <property name="TIME_2"
                   expression="get-property('SYSTEM_TIME')"
                   scope="default"
                   type="LONG"/>
         <script language="js">
              var time1 = mc.getProperty("TIME_1");
              var time2 = mc.getProperty("TIME_2");
              var timeTaken = time2 - time1;
              print("--------------  " + timeTaken + " ms  -----------------");
              mc.setProperty("RESPONSE_TIME", timeTaken);
         </script>
         <log>
            <property name="Time Duration in ms: " expression="get-property('RESPONSE_TIME')"/>
         </log>
      </outSequence>
      <endpoint>
         <address uri="http://localhost:8080/axis2/services/SimpleStockQuoteService"/>
      </endpoint>
   </target>
   <publishWSDL uri="http://localhost:8080/axis2/services/SimpleStockQuoteService?wsdl"/>
   <description/>
</proxy>

Hope this will be helpfull...!

Clustering WSO2 ESB - Part 2 - Distributed worker-manager setup with one manager and two workers

In the previous post I explained how you can cluster two WSO2 ESB servers where  one ESB node will be playing a dual role as a worker as well as a manager. The setup which I am going to explain in this post is similar to the above, but it has three ESB servers where one server will be dedicated only for management purposes while the other two will be dedicated workers.

As the previous one, this setup also can be used as a Active/Passive setup where the active-passiveness can be decided by the load balancer based on a predefined mechanism(echo test or a heartbeat mechanism). 

The overall setup will contain three ESB servers (nodes), in which one server will act as the Manager while other two will act as the workers. The Manager will be used for deploying artifacts (CAR files and proxies) and managing the cluster.

The  synchronization  of artifacts between the servers will be achieved through the SVN based Deployment Synchronizer.

The overall view of the cluster will be as in the following figure.

Configuring the Manager

Download and extract the WSO2 ESB distribution (referred as MANAGER_HOME ).

axis2.xml configuration

Clustering   should  be  enabled  at  axis2  level  in  order  for  management  node  to  communicate with  the   worker   nodes.  
Open  MANAGER_HOME/repository/conf/axis2/axis2.xml  and  update the clustering configuration as follows:

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent"  enable="true">
<parameter name="membershipScheme">wka</parameter>

Specify the cluster domain as follows;

<parameter name="domain">wso2.esb.domain</parameter>

Uncomment localmemberhost element in axis2.xml and specify the IP address (or host name) to be exposed to members of the cluster. This address can be an internal address used only for clustering.

<parameter name="localMemberHost">127.0.0.1</parameter>

Define the port through which other nodes will contact this member. If several instances are running in the same host make sure to use ports which will not conflict.

<parameter name="localMemberPort">4001</parameter>

Comment out the static or well­-known members, because this will be the well­-known member. (i.e do not need to have a list of members)
Add a new property "subDomain" under properties and set it to "mgt" to denote that this node belongs to management subdomain.  

<property name="subDomain" value="mgt"/> 

carbon.xml configuration

If multiple WSO2 Carbon ­based products are running in same host, to avoid possible port conflicts, the port offset of MANAGER_HOME/repository/conf/carbon.xml should be changed as follows. Ignore this if you are using three separate hosts.

<Offset>1</Offset>

Configure the Deployment Synchronizer as follows. Make sure both AutoCommit and
AutoCheckout are ‘true’

<DeploymentSynchronizer>
     <Enabled>true</Enabled>
     <AutoCommit>true</AutoCommit>
     <AutoCheckout>true</AutoCheckout>
     <RepositoryType>svn</RepositoryType>
     <SvnUrl>http://svnrepo.example/repos/esb</SvnUrl>
     <SvnUser>USERNAME</SvnUser>
     <SvnPassword>PASSWORD</SvnPassword>
     <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

Start the ESB manager node

 ./bin/wso2server.sh

You can access the management console by: https://localhost:9444/carbon/ within the managers node. (Use the correct port according to the port offset which was defined earlier) 

Add a sample proxy though the console and check whether it is getting committed to the SVN correctly. Proxy will be committed to SvnUrl/-1234/synapse-configs/default/proxy-services/ if you created the proxy as the admin.

Configuring the Worker node 1

Download and extract the WSO2 ESB distribution(referred as WORKER_HOME).

axis2.xml configuration

Clustering   should  be  enabled  at  axis2  level  in  order  for  management  node  to  communicate with  the  worker  nodes. Open  WORKER_HOME/repository/conf/axis2/axis2.xml and update the clustering configuration as follows:

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>

Specify the cluster domain as follows;

<parameter name="domain">wso2.esb.domain</parameter>

Define  the  port  through  which  other  nodes  will  contact  this  member.  If several instances are running in the same host make sure to use ports which will not conflict.

<parameter name="localMemberPort">4002</parameter>

Add managers(Node 1) IP address (or host name) and the port as the well known member.

• hostName: defined as localMemberHost in managers axis2.xml

• port: defined as localMemberPort in managers axis2.xml.

Make  sure  this  address  can  be  accessed  through  the  workers  host. If  you  use  host name, map it to managers IP in the ‘etc/hosts’ file of the workers machine.

<members>
   <member>
   <hostName>127.0.0.1</hostName>
      <port>4001</port>
   </member>
</members> 

Add a new property "subDomain" under properties and set it to "worker" to denote that this node belongs to worker subdomain.  

<property name="subDomain" value="worker"/> 

carbon.xml configuration

If multiple WSO2 Carbon­ based products are running in same host, to avoid possible port conflicts, the port offset of WORKER_HOME/repository/conf/carbon.xml should be changed as follows. Ignore this if you are using three separate hosts.

<Offset>2</Offset>

Configure the Deployment Synchronizer as follows. Make sure AutoCommit is set to ‘false’ and AutoCheckout to ‘true’

<DeploymentSynchronizer>
     <Enabled>true</Enabled>
     <AutoCommit>false</AutoCommit>
     <AutoCheckout>true</AutoCheckout>
     <RepositoryType>svn</RepositoryType>
     <SvnUrl>http://svnrepo.example/repos/esb</SvnUrl>
     <SvnUser>USERNAME</SvnUser>
     <SvnPassword>PASSWORD</SvnPassword>
     <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer> 

Start the ESB worker node. The workerNode system property must be set to true when starting the workers. 
Therefore worker should be started as follows (in linux),

./bin/wso2server.sh -DworkerNode=true 

Configuring Worker node 2 

With the consideration of the following points, follow the exact steps as the configuration the worker node 1.

• Decide on a port Offset value and update the port Offset element in WORKER_HOME/repository/conf/carbon.xml file. If you are using separate hosts ignore this.

• Change the localMemberPort value of WORKER_HOME/repository/conf/axis2/axis2.xml to a port that is not used already.

Start the ESB worker node. The workerNode system property must be set to true when starting the workers in a cluster. Therefore worker should be started as follows,

./bin/wso2server.bat -DworkerNode=true

How to test the setup..

• Start the servers as described above. 

• Go to the Management Console https://localhost:9444/carbon

• Add a sample proxy and save it. Add a log mediator to the inSequence so that logs will be displayed in the workers terminals

• Observe cluster messages(through the terminal or log) which is send by the manager and received by the worker. Worker will then synchronize with the SVN and deploy the proxy

• Send a request to the end point through the load balancer. Load Balancer should point to the active nodes end point. For example, for external clients, the EP of the proxy should be as  

http://{Load_Balancer_Mapped_URL_for_worker}/services/{Sample_Proxy_Name}

• In Active workers logs you will see the proxy being invoked (You need to add a log mediator as in number 3 in the proxy to see this)

Clustering WSO2 ESB - Part 1 - Distributed worker-manager setup with one node acting the dual role, as a manager as well as a worker

There are several ways of clustered deployment of WSO2 ESB (or any WSO2 server). You can find more about ESB clustering from here. The deployments introduced in that document is fronted by WSO2 ELB (Elastic Load Balancer). In this blog post I will explain the minimum configurations required to configure WSO2 ESB in a worker/manager distributed setup without having a WSO2 ELB at front so that any load balancer can be used. This setup can be also used as a Active/Passive setup where the active-passiveness can be decided by the load balancer based on a predefined mechanism(echo test or a heartbeat mechanism). 

In this setup one ESB node will be playing a dual role as a worker as well as a manager. The  overall  setup  will contain  two  ESB  servers  (nodes), in which  one  server (node 1) will act as  both  Manager  and  Worker while the other server(node 2) will act as only a worker. Node 1 will be used for deploying artifacts (CAR files and proxies) and managing the cluster.

The  synchronization  of artifacts between the servers will be achieved through the SVN based Deployment Synchronizer.

The overall view of the cluster will be as in the following figure.

Configuring the Manager/Worker Node (node 1)

Download and extract the WSO2 ESB distribution (referred as MANAGER_HOME ).

axis2.xml configuration

Clustering   should  be  enabled  at  axis2  level  in  order  for  management  node  to  communicate with  the   worker   nodes.  
Open  MANAGER_HOME/repository/conf/axis2/axis2.xml  and  update the clustering configuration as follows:

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent"  enable="true">
<parameter name="membershipScheme">wka</parameter>

Specify the cluster domain as follows;

<parameter name="domain">wso2.esb.domain</parameter>

Uncomment localmemberhost element in axis2.xml and specify the IP address (or host name) to be exposed to members of the cluster. This address can be an internal address used only for clustering.

<parameter name="localMemberHost">127.0.0.1</parameter>

Define the port through which other nodes will contact this member. If several instances are running in the same host make sure to use ports which will not conflict.

<parameter name="localMemberPort">4001</parameter>

Comment out the static or well­-known members, because this will be the well­-known member. (i.e do not need to have a list of members)

carbon.xml configuration

If multiple WSO2 Carbon ­based products are running in same host, to avoid possible port conflicts, the port offset of MANAGER_HOME/repository/conf/carbon.xml should be changed as follows. Ignore this if you are using three separate hosts.

<Offset>1</Offset>

Configure the Deployment Synchronizer as follows. Make sure both AutoCommit and
AutoCheckout are ‘true’

<DeploymentSynchronizer>
     <Enabled>true</Enabled>
     <AutoCommit>true</AutoCommit>
     <AutoCheckout>true</AutoCheckout>
     <RepositoryType>svn</RepositoryType>
     <SvnUrl>http://svnrepo.example/repos/esb</SvnUrl>
     <SvnUser>USERNAME</SvnUser>
     <SvnPassword>PASSWORD</SvnPassword>
     <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

Start the ESB manager node(./bin/wso2server.sh ).

You can access the management console by: https://localhost:9444/carbon/ within the managers node. (Use the correct port according to the port offset which was defined earlier) 

Add a sample proxy though the console and check whether it is getting committed to the SVN correctly. Proxy will be committed to SvnUrl/-1234/synapse-configs/default/proxy-services/ if you created the proxy as the admin.

Configuring the worker node (Node 2)

Download and extract the WSO2 ESB distribution(referred as WORKER_HOME).

axis2.xml configuration

Clustering   should  be  enabled  at  axis2  level  in  order  for  management  node  to  communicate with  the  worker  nodes. Open  WORKER_HOME/repository/conf/axis2/axis2.xml and update the clustering configuration as follows:

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>

Specify the cluster domain as follows;

<parameter name="domain">wso2.esb.domain</parameter>

Define  the  port  through  which  other  nodes  will  contact  this  member.  If several instances are running in the same host make sure to use ports which will not conflict.

<parameter name="localMemberPort">4002</parameter>

Add managers(Node 1) IP address (or host name) and the port as the well known member.

• hostName: defined as localMemberHost in managers axis2.xml

• port: defined as localMemberPort in managers axis2.xml.

Make  sure  this  address  can  be  accessed  through  the  workers  host. If  you  use  host name, map it to managers IP in the ‘etc/hosts’ file of the workers machine.

<members>
   <member>
   <hostName>127.0.0.1</hostName>
      <port>4001</port>
   </member>
</members> 

carbon.xml configuration

If multiple WSO2 Carbon­ based products are running in same host, to avoid possible port conflicts, the port offset of WORKER_HOME/repository/conf/carbon.xml should be changed as follows. Ignore this if you are using three separate hosts.

<Offset>2</Offset>

Configure the Deployment Synchronizer as follows. Make sure AutoCommit is set to ‘false’ and AutoCheckout to ‘true’

<DeploymentSynchronizer>
     <Enabled>true</Enabled>
     <AutoCommit>false</AutoCommit>
     <AutoCheckout>true</AutoCheckout>
     <RepositoryType>svn</RepositoryType>
     <SvnUrl>http://svnrepo.example/repos/esb</SvnUrl>
     <SvnUser>USERNAME</SvnUser>
     <SvnPassword>PASSWORD</SvnPassword>
     <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer> 

Start the ESB worker node. The workerNode system property must be set to true when starting the workers. 
Therefore worker should be started as follows (in linux),

./bin/wso2server.sh -DworkerNode=true 

How to test the setup..

• Start the servers as described above. 

• Go to the Management Console https://localhost:9444/carbon

• Add a sample proxy and save it. Add a log mediator to the inSequence so that logs will be displayed in the workers terminals

• Observe cluster messages(through the terminal or log) which is send by the manager and received by the worker. Worker will then synchronize with the SVN and deploy the proxy

• Send a request to the end point through the load balancer. Load Balancer should point to the active nodes end point. For example, for external clients, the EP of the proxy should be as  

http://{Load_Balancer_Mapped_URL_for_worker}/services/{Sample_Proxy_Name}

• In Active nodes logs you will see the proxy being invoked (You need to add a log mediator as in number 3 in the proxy to see this)