Showing posts with label Message Broker. Show all posts
Showing posts with label Message Broker. Show all posts

Sunday, September 24, 2017

A problem with the SOAP logical tree format

Just one small step will complete my finding. It was the web service development on Message Broker. Ever wonder how things done and I always got an excuse to delay my finding.

Say I have a WSDL, which consist of following:
...

<wsdl:types>
  <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://helloWorld.huahsin.org/">
    <xsd:element name="Input">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="name" type="xsd:string" minOccurs="1"/>
          <xsd:element name="age" type="xsd:integer" minOccurs="0"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
    <xsd:element name="Output">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="greetings" type="xsd:string" minOccurs="1"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
  </xsd:schema>
</wsdl:types>

...
Ignore those unnecessary stuff, let's focus only on the input and output. The above WSDL shows the following request input:
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Header/>
   <soap:Body>
      <hel:Input>
         <name>LKJ</name>
         <!--Optional:-->
         <age>0</age>
      </hel:Input>
   </soap:Body>
</soap:Envelope>
And the expected response message:
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Body>
      <hel:Output>
         <greetings>Hello World</greetings>
      </hel:Output>
   </soap:Body>
</soap:Envelope>
That's all about it. Somehow, things always out of expectation. I got this:
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
   <soapenv:Body>
      <soapenv:Fault>
         <soapenv:Code>
            <soapenv:Value>soapenv:Receiver</soapenv:Value>
         </soapenv:Code>
         <soapenv:Reason>
            <soapenv:Text xml:lang="en">BIP3113E: Exception detected in message flow Main.SOAP Input (broker brk2)</soapenv:Text>
         </soapenv:Reason>
         <soapenv:Node>://localhost:7080/HelloWorld</soapenv:Node>
         <soapenv:Detail>
            <Text>BIP3752E: The SOAP Reply node 'Main.SOAP Reply' encountered an error while processing a reply message. 
An error occurred during reply message processing. 
See previous error messages to determine the cause of the error. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPReplyNode.cpp: 397: ImbSOAPReplyNode::evaluate: ComIbmSOAPReplyNode: Main#FCMComposite_1_2
BIP3605E: The SOAP logical tree cannot be serialized. 
There is a problem with the SOAP logical tree format. 
Review further error messages for an indication to the cause of the error. Check that the SOAP logical supplied is correctly formatted. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPParser.cpp: 1360: ImbSOAPParser::refreshBitStreamFromElementsInner: : 
BIP3602E: The Web service payload ''{http://helloWorld.huahsin.org/}Input'' does not match an operation described by WSDL binding ''HelloWorldBinding'' in file ''huahsin.wsdl''. 
The first child of the SOAP Body does not correspond to any of the operations defined in the specified WSDL definition. 
Check that the correct WSDL definition was deployed. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPParser.cpp: 790: ImbSOAPParser::refreshBitStreamFromElementsInner: :</Text>
         </soapenv:Detail>
      </soapenv:Fault>
   </soapenv:Body>
</soapenv:Envelope>
At first I was so struggling to work on this error, luckily I got the solution during my work, The more I work, the more knowledge I learn. This was due to the message construction happened in Transform Node:
CREATE COMPUTE MODULE Main_Compute
  CREATE FUNCTION Main() RETURNS BOOLEAN
  BEGIN
    -- CALL CopyMessageHeaders();
    CALL CopyEntireMessage();

    DECLARE hel NAMESPACE 'http://helloWorld.huahsin.org/';

    SET OutputRoot.SOAP.Body.hel:Output.greetings = 'Hello World';

    RETURN TRUE;
  END;

  ...
END MODULE;
The error is from the OutputRoot. OutputRoot is the ultimate destination that the message will be reach at the end of the flow. During the message construction, any message that going to send out a response are advisable to clear them out. Like this:
    ...

    DELETE FIELD OutputRoot.SOAP.Body;
    SET OutputRoot.SOAP.Body.hel:Output.greetings = 'Hello World';

    ...
Leave the SOAP tree empty before any new message is constructed.

Sunday, September 10, 2017

Restart is required after switching to broker-wide listener

Just to share some though when I am using mqsicreateexecutiongroup command. First thing first, this command was so powerful that it will create a whole new world in the broker. Each world has a separate memory allocation that wouldn't affect to others when it is failing to operate. This is some sort of web instances in JAVA sense. When it was first created, I did a quit checked on its status, it shows following details.
>mqsireportproperties brk -e ceres -o ExecutionGroup -r

ExecutionGroup
  uuid='6f8ad861-5e01-0000-0080-cb62575bf054'
  userTraceLevel='none'
  traceLevel='none'
  userTraceFilter='none'
  traceFilter='none'
  label='ceres'
  unnamedUserTraceLevel='none'
  unnamedTraceLevel='none'
  processorArchitecture='64'
  consoleMode='off'
  traceNodeLevel='on'
  activeUserExitList=''
  inactiveUserExitList=''
  shortDesc=''
  longDesc=''
  httpNodesUseEmbeddedListener='false'
  soapNodesUseEmbeddedListener='true'

BIP8071I: Successful command completion.
Assuming I have a perfect program without any error being deployed, I can load up the WSDL through HTTP. But this time I receive a page cannot be found error, this indicate that the execution group is still in the embedded listener mode. In order to receive the request through HTTP, I am required to switch to broker-wide listener from embedded listener. Otherwise, my program could not be reached by the consuming system.
>mqsichangeproperties brk -e ceres -o ExecutionGroup -n soapNodesUseEmbeddedListener -v false
BIP8071I: Successful command completion.

>mqsireportproperties brk -e ceres -o ExecutionGroup -r

ExecutionGroup
  uuid='6f8ad861-5e01-0000-0080-cb62575bf054'
  userTraceLevel='none'
  traceLevel='none'
  userTraceFilter='none'
  traceFilter='none'
  label='ceres'
  unnamedUserTraceLevel='none'
  unnamedTraceLevel='none'
  processorArchitecture='64'
  consoleMode='off'
  traceNodeLevel='on'
  activeUserExitList=''
  inactiveUserExitList=''
  shortDesc=''
  longDesc=''
  httpNodesUseEmbeddedListener='false'
  soapNodesUseEmbeddedListener='false'

BIP8071I: Successful command completion.
I did a quick check on its status right after the configuration. Notice that the soapNodesUseEmbeddedListener have a value false. This is good ,but not there yet. I still will get the page cannot be found error when I send in the URL request. One trick is to restart the execution group, by now the WSDL should load up in the browser.

Monday, July 27, 2015

Continuous build on Message Flow with Jenkin 3

Since the build was so successful on last time. My colleague, Ming Kuan, doesn't want to let Jenkins stops. He is pushing Jenkins to next level – deploy to Message Broker after the build. Not much information could be found on the Internet for this issue. But he was successfully implemented it on Jenkins. He is a ranger, warrior, a real fighter among the team member. What he did was he created a BATCH script for the deployment job and then this BATCH script was triggered from ANT script.

This is the ingredient for ANT, its job only to trigger BATCH:
< target name="deploy">
   <exec executable="cmd">
      <arg value="/c"/>
      <arg value="${deploy_file_path}/deploy.bat"/>
      <arg value="${bar_file}.bar"/>
   </exec>
</target>

This is the ingredient for BATCH:
call "C:/Program Files/IBM/MQSI/8.0.0.0/bin/mqsiprofile.cmd"

SET EXE_PATH="C:/Program Files/IBM/MQSI/8.0.0.0/classes/*;C:/Program Files (x86)/IBM/WebSphere MQ/java/lib/*"

SET PATH=%PATH%;C:/Program Files/IBM/MQSI/8.0.0.0/jre16/bin;C:/Program Files/IBM/MQSI/8.0.0.0;

SET BAR_FILE =%1

java -cp %EXE_PATH% com.ibm.broker.config.util.Deploy -i 192.168.1.8 -p 1818 -q Q1 -e E1 -a %BAR_FILE% -w 600

java -cp %EXE_PATH% com.ibm.broker.config.util.MessageFlowControl -stop -i 192.168.1.8 -p 1818 -q Q1 -e E1

java -cp %EXE_PATH% com.ibm.broker.config.util.MessageFlowControl -start -i 192.168.1.8 -p 1818 -q Q1 -e E1

*Alert! Notice there is a path, C:/Program Files/IBM/MQSI/, being pre-code in the BATCH. This path contains various raw materials to feed the deployment process. Thus, make sure WebSphere MQ was installed on the system. If it doesn’t, C:/Program Files/IBM/MQSI/ will never exist on the system.

The com.ibm.broker.config.util.Deploy is just like the regular mqsideploy command use in ours daily job for deployment purpose. And com.ibm.broker.config.util.MessageFlowControl is just like mqsistartmsgflow and mqsistopmsgflow use to restart the execution group. The reason for this is to make sure the deployment will take effect immediately after deployment.

Nice work from our warrior, Ming Kuan.

Keep on building, Jenkins!

Tuesday, July 14, 2015

Continuous build on Message Flow with Jenkin 2

Not long ago, I’d done it once, but failed. That was happened on past 2 months ago. But now he is back again. Thanks to my colleague on his effort to discover this feature. This round ANT is used in the mission. First we try on this approach:
    < target name="create_bar">
     <mkdir dir="${dist.path}"/>
     <exec dir="." executable="C:/Program Files (x86)/IBM/WMBT800/mqsicreatebar.exe" failonerror="false" >
      <redirector output="error.log" alwayslog="true"/>     
      <arg line="-data . "/>
      <arg line="-b ProjectA/BARs/ProjectA.bar"/> 
      <arg line="-cleanBuild"/>
      <arg line="-l ProjectA"/> 
     </exec> 
    </target>
Pay attention on the argument –data and –b, the value might be different depends on the file structure. For my case, I put the build.xml inside the project folder. There is a con on this approach where it takes approximately 3 minutes to complete a build. Why does it take so long? The text below was extract from this document for easy reference.

3.2 Execution time It will be noted that when the BAR file is built from the command line, the time taken is often noticeably longer than time taken when built from the WMB ToolKit. This is because the mqsicreatebar command actually runs the Eclipse client itself in a 'headless' mode. The Eclipse front end probably keeps track of various file names, project references and lots of other parameters that speeds up the build time. As this does not happen from command line, the time taken is noticeably longer.
The second approach we did was this:
    < target name="package_bar">
     <mkdir dir="${dist.path}"/>
     <java classname="com.ibm.broker.config.appdev.FlowRendererBAR" failonerror="true" fork="true">
      <arg line="-a ${dist.path}/ProjectA.bar"/>
      <arg line="-w ../"/>
      <arg line="-y commonJava commonESQL ProjectA"/>
      <classpath>
       <fileset dir="C:/Program Files/IBM/MQSI/8.0.0.3/classes">
        <include name="*.jar"/>
       </fileset>
      </classpath>
     </java>
    </target>
With this approach, the execution time is damn fast. I’m feeling much better with this approach.

Reference resource

Saturday, July 4, 2015

Reading Message Broker property value has cause memory leak

This is my first time encountering memory leak happened in Java code. I’m feeling so ridiculous about this because I always believe that Java is a language that I should not worry about memory leak. There is one web service I was working on have spotted an abnormal behaviour usage and eventually will force the Execution Group stop functioning. This abnormal behaviour is hard to spot on my own until I run a load test on it (the load test was execute with SOAP UI).

But later have found out the root cause was happened when reading the properties values from configurable service defined in Message Broker. As according to the documentation, whenever reading the properties values with BrokerProxy as shown in the following code, it is advisable to invoke brokerProxy.disconnect() to free up resources, otherwise the memory will keep piling up causing memory leaks.
    public class JavaNode extends MbJavaComputeNode {
    
     @Override
     public void evaluate(MbMessageAssembly inAssembly) throws MbException
     {
      BrokerProxy brokerProxy;
      try {
       brokerProxy = BrokerProxy.getLocalInstance();
       brokerProxy.getConfigurableService("USERDEFINED", "key");
      } catch (ConfigManagerProxyLoggedException e1) {
       // TODO Auto-generated catch block
       e1.printStackTrace();
      } catch (ConfigManagerProxyPropertyNotInitializedException e) {
       // TODO Auto-generated catch block
       e.printStackTrace();
      }
    
      ...
     }
    }

Thus what I did is to invoke the disconnect() in finally clause. But have to be aware that after the resources had been freed, brokerProxy will no longer usable and have to be renew. Below are some useful resources I have read in solving the issue:
  1. Important facts about garbage collection in WebSphere Message Broker
  2. Class BrokerProxy
  3. Connecting to a broker from a CMP application

Thursday, June 4, 2015

Message Broker not able to start

ArghhhhHH~ my broker is jam. What a bad day! I was spending hours diagnosing this problem, until my colleague lent me a hand, then only I realize it was due to the recent update of my Windows logon password. This is my first time encounter this issue, I don’t mean to delay my work, please bear with me.

Whenever the Windows logon password is updated, I have to update IBM MQSeries service component immediately in order to start Message Broker. This component is accessible from:
  1. Start > Run > type dcomcnfg to bring up Component Services.
  2. From the Component Services, go to Computers > My Computer > DCOM Config, look for IBM MQSeries service.
  3. Right click on the service to bring up properties dialog.
  4. Under the Identity tab, ensure This user option is selected, fill in the currently logon user and password details.
That’s all about it.

Friday, May 29, 2015

How could I execute SQL query in ESQL

Usually I used to call stored procedure rather than executing SQL query directly in message flow, but this round the use case is a bit different. Is it possible to submit query in ESQL just like the prepare statement we did in regular programming language? The answer is yes. I have to put a mark mention that I have no knowledge about message flow programming, this skill was taught by my senior. This is how I make the query call:
    DECLARE query CHARACTER 'select tabA.colA, tabA.colB from Table_A tabA';
    DECLARE totalRow INTEGER;
    DECLARE I INTEGER;
    SET Environment.Variable.ResultSet[] = PASSTHRU(query);
    SET totalRow = CARDINALITY(Environment.Variable.ResultSet[]);
    IF (totalRow = 1) THEN
     SET OutputRoot.SOAP.Body.ns1:ServiceAResponse.ns1:Category.ns1:FieldA = Environment.Variable.ResultSet.colA;
     SET OutputRoot.SOAP.Body.ns1:ServiceAResponse.ns1:Category.ns1:FieldB = Environment.Variable.ResultSet.colB;
    ELSE
     SET I = 1;
     WHILE I < totalRow DO   
      SET OutputRoot.SOAP.Body.ns1:ServiceAResponse.ns1:Category[I].ns1:FieldA = Environment.Variable.ResultSet[I].colA;
      SET OutputRoot.SOAP.Body.ns1:ServiceAResponse.ns1:Category[I].ns1:FieldB = Environment.Variable.ResultSet[I].colB;
      SET I = I + 1;
     END WHILE;
    END IF;
The statement at line 4, PASSTHRU(query), is how I execute the query. The result will be store in ResultSet[]. It is an array. And then at line 11 is how I loop through the result set from DB

Take note also if the result set return just 1 row of record, I have to explicitly tell the program how it should be execute. Notice the difference between line 7 and line 12? This is because in ESQL, the assignment of only 1 value can't be done to a variable of array type.

Feeling stunt? Yes, I am.

Establish Informix connection in Message Broker

In order to allow WebSphere Message Broker (WMB) establish connection to Informix database, first I must have data source configure in Windows. The configuration was done through ODBC. The very first step is to have Informix driver install in Windows, otherwise there is no room for WMB connect to Informix. Following are the summary steps for the setup:
  1. Install Informix driver.
  2. Go to Control Panel > Administrative Tools > Data Sources (ODBC) > Drivers tab > Verify driver is there.
  3. In system DSN tab > Add new data source.
  4. Setup necessary details and choose 'olsoctcp' protocal.
Once done, the data source has to be register into WMB. Open up WMB command prompt and fire following command:

mqsisetdbparms Broker_Name -n Data_Source_Name -u username -p password

mqsicvp Broker_name -n Data_Source_Name

The last command is to verify the data source is successfully registered into message broker. And for each command entered, BIP8071I: Successful command completion. should be seen. Do take note also if the Windows command prompt is used instead of WMB command prompt, issue C:\IBM\MQSI\8.0.0.3\bin\mqsiprofile.cmd before any mqsi_xxx command can be used.

Continuous build on Message Flow with Jenkin

Every time there is a new release, I am required to get the latest source from SVN, make the build and then deploy using WebSphere Message Broker Toolkit. This is so manual way. Then later something cool pops up in my mind: “Could it be better if I automate the process in Jenkins”? Since Jenkins do support Windows batch command and WebSphere Message Broker able to accomplished task through command line, thus both tools might integrate together. Here are the steps to mimic the whole process from start to end:
  1. Clean the workspace.
  2. Get latest source from SVN.
  3. Build the bar file.
  4. Deploy to server.
Let’s start coding on batch script.
del /s /q MessageFlowProject\*.*
    for /d %%x in (.\MessageFlowProject\*) do @rd /s /q "%%x"
    FOR /f "tokens=*" %%G IN ('DIR /b /ad /s MessageFlowProject\*.svn*') DO RMDIR /s /q "%%G"
    
    svn checkout --force SVN_URL module_A
    svn checkout --force SVN_URL module_B
    svn checkout --force SVN_URL module_C
    svn checkout –-force SVN_URL MessageFlowProject
    
    C:\"Program Files"\IBM\MQSI\8.0.0.3\bin\mqsiprofile.cmd
    
    mqsipackagebar -a MessageFlowProject.bar -w . -y module_A module_B module_C MessageFlowProject
    
    mqsideploy -i 192.168.1.2 -p 1418 -q QueueA -e GRP.A -a MessageFlowProject.bar -w 600
Notice the 3rd is required to clean .svn path as the 2nd statement unable to handle hidden path since .svn are hidden in file system. The script doesn’t work as expected in Jenkins environment. The build result was successful, and I could see following trace in the console output:
C:\Users\Vendor\.jenkins\jobs\Job1\workspace>C:\IBM\MQSI\8.0.0.3\bin\mqsiprofile.cmd
    MQSI 8.0.0.3
    C:\IBM\MQSI\8.0.0.3
I notice the last 2 statements, which is mqsi*, doesn’t get invoke after mqsiprofile.cmd were executed. I think this is due to mqsiprofile.cmd has additional environment configure to the system where Jenkins not able to read from there.

I’m running out of clue how could I continue, I am stuck!