New implementation of clearing recursive intrusive list

There is still memory leak happens at the end of each test case. Urrhhh! Now only I got to remember this is a new implementation of loading up the file path into memory. A new thing. And the clearMemory() is still implementing the old code.

void FileBot::clearMemory()
{
    for(FileNode &T : fileList) {
        T.sibling.erase_and_dispose(T.sibling.begin(), T.sibling.end(), DisposeFileNode());
    }
 
    fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
}

In the new implementation, one I can think of is a recursive function. The recursive function usage is like this: As long as there are file objects inside the sibling, dive into the sibling and look for any other file objects inside the sibling. If it doesn't contain any file object, it will return. This return will go back up one level, then only clean the sibling for the particular file object. The process will continue until it reaches to the beginning level of the path, then only remove the remaining memory from container.

void FileBot::clearMemory(FileNode *fileNode)
{
    bool beginning = false;

    if( fileNode == NULL ) {
       fileNode = &*(fileList.begin());
       beginning = true;
    }

    if( fileNode->sibling.size() > 0 ) {
        cout << "scanning current node: " << fileNode << endl;
        clearMemory(&*(fileNode->sibling.begin()));
    }
    else
        return;

    cout << "clean sibling memory: " << fileNode << " sibling size: " << fileNode->sibling.size() << endl;
    fileNode->sibling.erase_and_dispose(fileNode->sibling.begin(), fileNode->sibling.end(), DisposeFileNode());

    if( beginning == true) {
        cout << "clean root. Root size [" << fileList.size() << "]" << endl;
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
    }
}

Notice the clearMemory(FileNode* ) is expecting an argument. I have declared this argument to NULL when it is first called. Something like this:

class FileBot
{
public:
   void clearMemory(FileNode *fileNode = NULL);

   ...
   ...
};

Thanks to the C++ great feature. When it is NULL, this indicate the beginning of the file path, after the subsequent call to the clearMemory(), it is no longer NULL anymore. And when the sibling size is zero, clearMemory() wouldn't get called. Thus, no worry about that.

In addition to that, there is a memory leak happened on following test case:

/* load 1 level parent path
 *
 */
BOOST_AUTO_TEST_CASE(TL_4)
{
    BOOST_TEST_MESSAGE("TC4 : load 1 level parent path");

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:");
    FileBot fb("D:");
#else
    BOOST_TEST_MESSAGE("Test path: /home");
    FileBot fb("/home");
#endif

    string path = "";
    if( fb.initialize() == 1 )
       path = fb.verifyFileList();

#if defined(WIN32)
    BOOST_TEST(path == "D:");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home");
#endif
}

This is due to something was not being handle probably in clearMemory(), end up the fileList wasn't clear. This test case consists of a file path with only one level, such as C:\ in Windows or /home in Linux. For Linux, it is a little bit special, /home is actually 2 levels file path. One is root path, and home is under the root. For my requirement, I just treat it as 1 level. Unlike Windows, the root is represented by a label, C:\.

I can't think of a perfect solution to resolve this defect yet. For now, I make a validation check at the beginning of the function:

void FileBot::clearMemory(FileNode *fileNode)
{
    bool beginning = false;

    // this parent have no child
    if( (&*(fileList.begin()))->sibling.size() == 0 ) {
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
        return;
    }

    ...
    ...
}

When I see there is no sibling, clean up the mess and then bail out from the function.

New solution to build parent path

It took me a few weeks to work on this POC with Boost Intrusive. I was thinking to use Boost Intrusive build a directory path. Just like a tree structure, the branches representing the files and folders, spread until the end of the sub directory. Thus the setup for this class is as follows:

class FileNode : public boost::intrusive::list_base_hook<link_mode<auto_unlink> >
{
private:
    boost::filesystem::path name; // file name
    char type; // f is file, d is directory

public:
    boost::intrusive::list< FileNode, base_hook<list_base_hook<link_mode<auto_unlink> > >, constant_time_size<false> > sibling;

};

I created a FileNode class represent the file in a path, and each object of this class can neither be a file or a folder, and the a name too. These are the private member declared in the class. The sibling member is to tell whether they are any sub directory available, if the size is greater than zero, it means there are files in it.

Now come to the main dish. There are 2 part of it; first would be the load the given path into the memory by using the class structure mention above, second is to constructed of the files underneath the given path. For this POC, I'm focusing on the first part, the second part has not yet done.

I have though should I skip the first part for some while? Say when user keyed in /home/path_1, can I just treat the value as a single FileNode? Since I doesn't really care what the value are, it would be easier for me to do the job. Coming from the perfectionism view-point, I think it would be nice to load every single file entity as a separate FileNode object. Meaning to said that /home is one object, and /path_1 is another object.

For this purpose, I create a function to handle this job for me.

FileNode* FileBot::constructParentPath(string path)
{
    vector<string> words;
    FileNode *node = new FileNode();
    string unixFilePath = path;

    boost::replace_last(unixFilePath, "/", "|");
    boost::split(words, unixFilePath, boost::is_any_of("|"), boost::token_compress_on);

    node->setType('d');
    node->setName(words[1]);

    // there is a parent node
    if( words[0] != "" ) {
        FileNode *parent = constructParentPath(words[0]);

        cout << "parent address: " << parent << endl;

        parent->sibling.push_back(*node);
    }
    else {
        fileList.push_back(*node);
        cout << "node address: " << node << " " << node->getName() << endl;
    }

    words.clear();

    return node;
}

This function is as good as it works only on Linux, but failed on Windows. It can't pass the test case in Windows specific path. After many round of rework, then only I figure out a new solution that tested out on both Linux and Windows:

int FileBot::initialize()
{
    // bail out if the path doesn't exists
    if( !exists(fileNode.getName().generic_string()) )
        return 0;

    // construct the parent path first
    string path = fileNode.getName().generic_string();
    FileNode *parent = NULL;

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(path, first_finder("/", is_equal()));
         it != string_split_iterator();
         ++it)
    {
        FileNode *node = new FileNode();
        node->setType('d');
        node->setName(copy_range<string>(*it));

        cout << "value inserted: [" << copy_range<string>(*it) << "]" << endl;

        if( parent == NULL ) {
            fileList.push_back(*node);
        }
        else
            parent->sibling.push_back(*node);

        parent = node;
        cout << "parent: " << parent;
        cout << " parent value: " << parent->getName();
        cout << " sibling size: " << parent->sibling.size() << endl;
    }

    return 1;
}

The difference between the 2 solutions is that the first one is using the recursive loop and the later one is using for loop. On top of that the second solution is much easier to read than the first. To check my work is being done correctly, I have created following test case for the unit test.

/* load 1 level parent path
 *
 */
BOOST_AUTO_TEST_CASE(TL_4)
{
    BOOST_TEST_MESSAGE("TC4 : load 1 level parent path");

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:");
    FileBot fb("D:");
#else
    BOOST_TEST_MESSAGE("Test path: /home");
    FileBot fb("/home");
#endif

    string path = "";
    if( fb.initialize() == 1 )
       path = fb.verifyFileList();

#if defined(WIN32)
    BOOST_TEST(path == "D:");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home");
#endif
}


/* load 2 level parent path
 *
 */
BOOST_AUTO_TEST_CASE(TL_5)
{
    BOOST_TEST_MESSAGE("TC5 : load 2 level parent path");

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:/workspaceqt");
    FileBot fb("D:/workspaceqt");
#else
    BOOST_TEST_MESSAGE("Test path: /home/kokhoe");
    FileBot fb("/home/kokhoe");
#endif

    string path = "";
    if( fb.initialize() == 1 )
       path = fb.verifyFileList();

#if defined(WIN32)
    BOOST_TEST(path == "D:/workspaceqt");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home/kokhoe");
#endif
}


/* load 3 level parent path
 *
 */
BOOST_AUTO_TEST_CASE(TL_6)
{
    BOOST_TEST_MESSAGE("TC6 : load 3 level parent path");

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:/workspaceqt/ui1");
    FileBot fb("D:/workspaceqt/ui1");
#else
    BOOST_TEST_MESSAGE("Test path: /home/kokhoe/workspaceqt");
    FileBot fb("/home/kokhoe/workspaceqt");
#endif

    string path = "";
    if( fb.initialize() == 1 )
       path = fb.verifyFileList();

#if defined(WIN32)
    BOOST_TEST(path == "D:/workspaceqt/ui1");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home/kokhoe/workspaceqt");
#endif
}

Accessing the last element of an Intrusive list

I always thought that retrieving the last element from an intrusive list just as easy as one two three. In fact, it is not. Let's dive into the story. Here I have the intrusive list declaration:

/***** FileNode.h *****/

class FileNode : public boost::intrusive::list_base_hook<link_mode<auto_unlink> >
{
...
...

};


/***** FileBot.h *****/

typedef boost::intrusive::list<FileNode, base_hook<list_base_hook<link_mode<auto_unlink> > >, constant_time_size<false> > FileNodeListType;

class FileBot
{
private:
    FileNodeListType fileList;

...
...
};

And then I'll use the iterator as shown below go straight to access the element located at the end of the list, but somehow this a hit segmentation fault error.

        FileNodeListType::iterator it(fileList.end());

        // following piece basically doing some data manipulation
        // on the element retrieve from the list.
        FileNode *p = &*it;
        (*it).sibling.push_back(*node);

        cout << p << endl; // memory address shows 0x7ffedff8e808

The error is due to the wrong memory address is being accessed. As I verify on the element being inserted into the list (only one element is inserted in this test case) is having an address different from the one mention in the code above. Thus, according to the expert, in order to access the correct element located at the end of the list is by doing this:

        FileNodeListType::iterator it(fileList.end());
        FileNode *p = &*(--it);
        cout << p << endl; // memory address shows 0xa01bc0

Remember to clear the memory hold in Boost recursive intrusive

While I'm studying on making an intrusive container right inside an object which which is also an intrusive container. Some sort of nested intrusive container I might think of. Well, I never thought Boost really made one for us, this feature is called recursive intrusive container.

There is some issue with this container. It happens only on Windows, no problem in Linux at all. When removing an object from an intrusive container, make sure the nested intrusive container is clear off before the object is removed. I have run some test case about it.

Here is the declaration of the object for the test case, the sibling member is the recursive intrusive container:

class FileNode : public boost::intrusive::list_base_hook<link_mode<auto_unlink> >
{
public:
    boost::intrusive::list<FileNode, base_hook<list_base_hook<link_mode<auto_unlink> > >, constant_time_size<false> > sibling;

...
...
};

This is the intrusive list declaration:

typedef boost::intrusive::list<FileNode, base_hook<list_base_hook<link_mode<auto_unlink> > >, constant_time_size<false> > FileNodeListType;

class FileBot
{
public:
    FileNodeListType fileList;
 

First, an object was pumped into the intrusive container, fileList in this case. Then allocate another object into recursive list, sibling. I loop through every object in the fileList and put in another object into the recursive list.

void FileBot::addFileNode()
{
    FileNode *f1 = new FileNode();
    f1->setName("ASD");
    fileList.push_back(*f1);
}

void FileBot::addSubFileNode()
{
    FileNode *f2 = NULL;

    for(FileNode &T : fileList)
    {
        f2 = new FileNode();
        f2->setName("QWE");
        T.sibling.push_back(*f2);
    }
}

Now I want to clear everything in the fileList, without knowing is there any object saved into the container yet. Basically I just doesn't care at all, I just want to clear the container. Thus I implemented the following code:

void FileBot::clearMemory()
{
   fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
}

When I call this method, memory leak is detected. This happens only on Windows. This is my mistake. I overlook on the addSubFileNode() on how I extract the object, T from the fileList. Each object has its sibling, and each sibling has another FileNode object in it, which consume memory as well. For my case, the object inside sibling memory is not removed but the object in fileList memory has already removed. Put in other words, it just like the parent object contain the child object, when releasing the memory, the child's object must release first then only release the parent's object. And what I did here is I go straight to the release the parent object, thus the child object is now complaining about the memory leak.

So in order to play safe in this game, I'll do this:

void FileBot::clearMemory()
{
    for(FileNode &T : fileList) {
        T.sibling.erase_and_dispose(T.sibling.begin(), T.sibling.end(), DisposeFileNode());
    }

    fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
}

Clean off the memory in the recursive intrusive container before cleaning the memory in the parent intrusive container. This should be the strategy for this game.

Removing an item from the Boost Intrusive list

Over the pass few weeks, I was reading the a lot of articles about the intrusive list, its benefits of using it, and its use case example on game development. This brings me a strong message that there are so many peoples on Earth promoting this container. What was so special about this container? Due to the limitation on the container provided by STL and intrusive container's low memory footprint which could benefit to the application's performance.

Does Boost provide this container. Yes! During this great moment with a sudden impulse, I begin my journey to develop a demo program using the Boost Intrusive container. Tadaa~

So far the journey was up and down, a lot of road pump, and feel tired to continue. When trying to remove an item from the list using following method will not work. Before that, let's presume I have the following declaration in the header file:

typedef boost::intrusive::list<FileNode, base_hook<list_base_hook<link_mode<auto_unlink> > >, constant_time_size<false> > FileNodeListType;

class FileBot
{
private:
    FileNodeListType fileList;

    ...
    ...
};

Then I begin to insert something into the list, assume the object type is a FileNode, do take note that I do this purely in one method:

void FileBot::addFileNode()
{
    FileNode *f1 = new FileNode();
    f1->setName("ASD");
    fileList.push_back(*f1);

    cout << "addFileNode(): memory address of insert object: " << f1 << endl;
}

Then I have another method to remove an object from the list:

void FileBot::clearMemoryByName(string filename)
{
    cout << "before delete: " << fileList.size() << endl;

    FileNode *p = new FileNode();
    p->setName("ASD");

    fileList.erase(FileNodeListType::s_iterator_to(*p));
    delete p;

    cout << "clearMemoryByName(): memory address of remove object: " << p << endl;

    cout << "after delete: " << fileList.size() << endl;
}

But somehow this method will cause a memory leak due to the following assertion failed in boost/intrusive/list.hpp, line 1270:

Assertion failed: !node_algorithms::inited(value_traits::to_node_ptr(value)), file D:\tool\boost_1_61_0\msvc\boost/intrusive/list.hpp, line 1270
Detected memory leaks!
Dumping objects ->
...
...
...
Object dump complete.
Press <return> to close this window...

A million hours were spent on studying the weak spot of this creature has finally found a solution. The clue was lying on the memory address of the object. The memory address of the object inserted into the list in addFileNode() was different from the memory object created in the clearMemoryByName().

...

addFileNode(): memory address of insert object: 001180C8
clearMemoryByName(): memory address of remove object: 00111DE0

...

In the revise's version, the object was retrieved from the list and then perform some filtering (e.g. match the property field's value), then only perform the deletion.

void FileBot::clearMemoryByName(string filename)
{
    FileNode *p = NULL;
    FileNodeListType::iterator it(fileList.begin());

    for(; it != fileList.end(); it++)
    {
        cout << &*it << endl;

        // look for particular node
        string curName = (*it).getName().c_str();
        if( filename == curName) {
            p = &*it;
            break;
        }
    }

    cout << "claerMemoryByName(): memory address of remove object: " << p << endl;

    fileList.erase(FileNodeListType::s_iterator_to(*p));
    delete p;

The result was stunning! The object was removed from the list. And here is the output of the memory address:

...

addFileNode(): memory address of insert object: 00E2DAF0
claerMemoryByName(): memory address of remove object: 00E2DAF0

...

To conclude this, in order to remove an object from the list, O(n) algorithm is the only way to filter the object, then only come to deletion.

Convert Windows file separator into Linux format

I wrote a function to extract the parent path for a given path. An interesting thing happens here; Windows's file system was used (\) backslash as file separator, whereas Linux's using (/) slash as file separator. Thus, this will end up I have following code produce:

void FileBot::constructParentPath()
{
   ...

   string sep = string(1, boost::filesystem::path::preferred_separator);
   if( sep == "/" )
      boost::replace_last(filePath, "/", "|");
   else
      boost::replace_last(filePath, "\", "|");

   ...
}

When handling file separator with Boost library, the library will able to convert the Windows's file separator into Linux format. Therefore, my code will reduce into one line from the previous version:

void FileBot::constructParentPath()
{
   ...

   string filePath = fileNode.getName().generic_string();
   boost::replace_last(filePath, "/", "|");

   ...
}

A problem with the SOAP logical tree format

Just one small step will complete my finding. It was the web service development on Message Broker. Ever wonder how things done and I always got an excuse to delay my finding.

Say I have a WSDL, which consist of following:

...

<wsdl:types>
  <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://helloWorld.huahsin.org/">
    <xsd:element name="Input">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="name" type="xsd:string" minOccurs="1"/>
          <xsd:element name="age" type="xsd:integer" minOccurs="0"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
    <xsd:element name="Output">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="greetings" type="xsd:string" minOccurs="1"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
  </xsd:schema>
</wsdl:types>

...

Ignore those unnecessary stuff, let's focus only on the input and output. The above WSDL shows the following request input:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Header/>
   <soap:Body>
      <hel:Input>
         <name>LKJ</name>
         <!--Optional:-->
         <age>0</age>
      </hel:Input>
   </soap:Body>
</soap:Envelope>

And the expected response message:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Body>
      <hel:Output>
         <greetings>Hello World</greetings>
      </hel:Output>
   </soap:Body>
</soap:Envelope>

That's all about it. Somehow, things always out of expectation. I got this:

<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
   <soapenv:Body>
      <soapenv:Fault>
         <soapenv:Code>
            <soapenv:Value>soapenv:Receiver</soapenv:Value>
         </soapenv:Code>
         <soapenv:Reason>
            <soapenv:Text xml:lang="en">BIP3113E: Exception detected in message flow Main.SOAP Input (broker brk2)</soapenv:Text>
         </soapenv:Reason>
         <soapenv:Node>://localhost:7080/HelloWorld</soapenv:Node>
         <soapenv:Detail>
            <Text>BIP3752E: The SOAP Reply node 'Main.SOAP Reply' encountered an error while processing a reply message. 
An error occurred during reply message processing. 
See previous error messages to determine the cause of the error. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPReplyNode.cpp: 397: ImbSOAPReplyNode::evaluate: ComIbmSOAPReplyNode: Main#FCMComposite_1_2
BIP3605E: The SOAP logical tree cannot be serialized. 
There is a problem with the SOAP logical tree format. 
Review further error messages for an indication to the cause of the error. Check that the SOAP logical supplied is correctly formatted. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPParser.cpp: 1360: ImbSOAPParser::refreshBitStreamFromElementsInner: : 
BIP3602E: The Web service payload ''{http://helloWorld.huahsin.org/}Input'' does not match an operation described by WSDL binding ''HelloWorldBinding'' in file ''huahsin.wsdl''. 
The first child of the SOAP Body does not correspond to any of the operations defined in the specified WSDL definition. 
Check that the correct WSDL definition was deployed. : F:\build\slot1\S800_P\src\WebServices\WSLibrary\ImbSOAPParser.cpp: 790: ImbSOAPParser::refreshBitStreamFromElementsInner: :</Text>
         </soapenv:Detail>
      </soapenv:Fault>
   </soapenv:Body>
</soapenv:Envelope>

At first I was so struggling to work on this error, luckily I got the solution during my work, The more I work, the more knowledge I learn. This was due to the message construction happened in Transform Node:

CREATE COMPUTE MODULE Main_Compute
  CREATE FUNCTION Main() RETURNS BOOLEAN
  BEGIN
    -- CALL CopyMessageHeaders();
    CALL CopyEntireMessage();

    DECLARE hel NAMESPACE 'http://helloWorld.huahsin.org/';

    SET OutputRoot.SOAP.Body.hel:Output.greetings = 'Hello World';

    RETURN TRUE;
  END;

  ...
END MODULE;

The error is from the OutputRoot. OutputRoot is the ultimate destination that the message will be reach at the end of the flow. During the message construction, any message that going to send out a response are advisable to clear them out. Like this:

    ...

    DELETE FIELD OutputRoot.SOAP.Body;
    SET OutputRoot.SOAP.Body.hel:Output.greetings = 'Hello World';

    ...

Leave the SOAP tree empty before any new message is constructed.

Never forget qmake

Oh come on! Error again! I encounter a weird error on QT compiler today. I though I had the thing fixed on last week, wonder why the fix on last week is not working on today? What a joke. Now the error is different. The error mentions that I have multiple definition found. The root cause is coming from the Boost library.

13:12:11: Starting: "D:\tool\Qt\Tools\QtCreator\bin\jom.exe" 
 D:\tool\Qt\Tools\QtCreator\bin\jom.exe -f Makefile.Debug
 cl -c -nologo -Zc:wchar_t -FS -Zc:strictStrings -Zc:throwingNew -Zi -MDd -GR -W3 -w34100 -w34189 -w44996 -w44456 -w44457 -w44458 -wd4577 -wd4467 -EHsc /Fddebug\wqt1.vc.pdb -DUNICODE -DWIN32 -DQT_QML_DEBUG -DQT_CORE_LIB -I..\wqt1 -I. -I..\..\..\tool\boost_1_61_0 -I..\..\tool\Qt\5.7\msvc2015\include -I..\..\tool\Qt\5.7\msvc2015\include\QtCore -Idebug -I..\..\tool\Qt\5.7\msvc2015\mkspecs\win32-msvc2015 -Fodebug\ @C:\Users\HUAHS_~1\AppData\Local\Temp\main.obj.4132.31.jom
main.cpp
 link /NOLOGO /DYNAMICBASE /NXCOMPAT /DEBUG /SUBSYSTEM:CONSOLE "/MANIFESTDEPENDENCY:type='win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' publicKeyToken='6595b64144ccf1df' language='*' processorArchitecture='*'" /MANIFEST:embed /OUT:debug\wqt1.exe @C:\Users\HUAHS_~1\AppData\Local\Temp\wqt1.exe.4132.8453.jom
libboost_filesystem-vc140-mt-gd-1_61.lib(path_traits.obj) : error LNK2005: "void __cdecl boost::filesystem::path_traits::convert(char const *,char const *,class std::basic_string<wchar_t std::char_traits="" struct="" wchar_t="">,class std::allocator<wchar_t> > &,class std::codecvt<wchar_t _mbstatet="" char="" struct=""> const &)" (?convert@path_traits@filesystem@boost@@YAXPBD0AAV?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@ABV?$codecvt@_WDU_Mbstatet@@@5@@Z) already defined in boost_filesystem-vc140-mt-gd-1_61.lib(boost_filesystem-vc140-mt-gd-1_61.dll)
libboost_filesystem-vc140-mt-gd-1_61.lib(path.obj) : error LNK2005: "public: static class std::codecvt<wchar_t _mbstatet="" char="" struct=""> const & __cdecl boost::filesystem::path::codecvt(void)" (?codecvt@path@filesystem@boost@@SAABV?$codecvt@_WDU_Mbstatet@@@std@@XZ) already defined in boost_filesystem-vc140-mt-gd-1_61.lib(boost_filesystem-vc140-mt-gd-1_61.dll)
debug\wqt1.exe : fatal error LNK1169: one or more multiply defined symbols found
jom: D:\workspaceqt\build-wqt1-Desktop_Qt_5_7_1_MSVC2015_32bit-Debug\Makefile.Debug [debug\wqt1.exe] Error 1169
jom: D:\workspaceqt\build-wqt1-Desktop_Qt_5_7_1_MSVC2015_32bit-Debug\Makefile [debug] Error 2

How could I resolve them? I am spending hours experiment this error, by making a clean build or start a new project from scratch just to simulate this error have no result. Until late night only found out there is a very important step was missing during the build.

Whenever the PRO file is updated, I'm required to run qmake, because this will allow to update the Makefile generate by the QT. No wonder why sometimes I get an error on undefined reference on blah_blah_blah_functionA() when I add a new library in PRO file.

Restart is required after switching to broker-wide listener

Just to share some though when I am using mqsicreateexecutiongroup command. First thing first, this command was so powerful that it will create a whole new world in the broker. Each world has a separate memory allocation that wouldn't affect to others when it is failing to operate. This is some sort of web instances in JAVA sense. When it was first created, I did a quit checked on its status, it shows following details.

>mqsireportproperties brk -e ceres -o ExecutionGroup -r

ExecutionGroup
  uuid='6f8ad861-5e01-0000-0080-cb62575bf054'
  userTraceLevel='none'
  traceLevel='none'
  userTraceFilter='none'
  traceFilter='none'
  label='ceres'
  unnamedUserTraceLevel='none'
  unnamedTraceLevel='none'
  processorArchitecture='64'
  consoleMode='off'
  traceNodeLevel='on'
  activeUserExitList=''
  inactiveUserExitList=''
  shortDesc=''
  longDesc=''
  httpNodesUseEmbeddedListener='false'
  soapNodesUseEmbeddedListener='true'

BIP8071I: Successful command completion.

Assuming I have a perfect program without any error being deployed, I can load up the WSDL through HTTP. But this time I receive a page cannot be found error, this indicate that the execution group is still in the embedded listener mode. In order to receive the request through HTTP, I am required to switch to broker-wide listener from embedded listener. Otherwise, my program could not be reached by the consuming system.

>mqsichangeproperties brk -e ceres -o ExecutionGroup -n soapNodesUseEmbeddedListener -v false
BIP8071I: Successful command completion.

>mqsireportproperties brk -e ceres -o ExecutionGroup -r

ExecutionGroup
  uuid='6f8ad861-5e01-0000-0080-cb62575bf054'
  userTraceLevel='none'
  traceLevel='none'
  userTraceFilter='none'
  traceFilter='none'
  label='ceres'
  unnamedUserTraceLevel='none'
  unnamedTraceLevel='none'
  processorArchitecture='64'
  consoleMode='off'
  traceNodeLevel='on'
  activeUserExitList=''
  inactiveUserExitList=''
  shortDesc=''
  longDesc=''
  httpNodesUseEmbeddedListener='false'
  soapNodesUseEmbeddedListener='false'

BIP8071I: Successful command completion.

I did a quick check on its status right after the configuration. Notice that the soapNodesUseEmbeddedListener have a value false. This is good ,but not there yet. I still will get the page cannot be found error when I send in the URL request. One trick is to restart the execution group, by now the WSDL should load up in the browser.

When Boost Log meet MinGW, everything will not work!

My mission on migrating Boost code from Linux to Windows is almost failed. The error was strange enough that I have no idea why there are so many undefined reference happened in the code. I mean, since the code is designed for cross platform, it should not have any trouble if it is migrating to the other platform.

My finding on this matter was that the compiler will throw error whenever I have the following piece appear in my code.

boost::log::add_file_log
(
   keyword::file_name = "sample.log",
   keyword::rotation_size = 10 * 1024 * 1024,
   keyword::time_based_rotation = boost::log::sinks::file::rotation_at_time_point(0,0,0),
   keyword::format = "[%TimeStamp%]: %Message%"
);

Yippee! I start to see some light on the problem. That piece is belongs to Boost Log. I am pretty sure the error comes from this piece.

D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `make_absolute':
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:597: undefined reference to `boost::filesystem::absolute(boost::filesystem::path const&, boost::filesystem::path const&)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `store_file':
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:695: undefined reference to `boost::filesystem::path::parent_path() const'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `scan_for_files':
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:776: undefined reference to `boost::filesystem::path::parent_path() const'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost3log10v2s_mt_nt55sinks17text_file_backend7consumeERKNS1_11record_viewERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE':
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:1218: undefined reference to `boost::filesystem::path::parent_path() const'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost3log10v2s_mt_nt55sinks17text_file_backend30set_file_name_pattern_internalERKNS_10filesystem4pathE':
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:1265: undefined reference to `boost::filesystem::path::parent_path() const'
d:\tool\boost_1_61_0/libs/log/src/text_file_backend.cpp:1265: undefined reference to `boost::filesystem::absolute(boost::filesystem::path const&, boost::filesystem::path const&)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZNK5boost10filesystem4path15has_parent_pathEv':
d:\tool\boost_1_61_0/./boost/filesystem/path.hpp:516: undefined reference to `boost::filesystem::path::parent_path() const'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystemdvERKNS0_4pathES3_':
d:\tool\boost_1_61_0/./boost/filesystem/path.hpp:789: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem12current_pathEv':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:596: undefined reference to `boost::filesystem::detail::current_path(boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem10equivalentERKNS0_4pathES3_':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:608: undefined reference to `boost::filesystem::detail::equivalent(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem9file_sizeERKNS0_4pathE':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:614: undefined reference to `boost::filesystem::detail::file_size(boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem9file_sizeERKNS0_4pathERNS_6system10error_codeE':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:618: undefined reference to `boost::filesystem::detail::file_size(boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem15last_write_timeERKNS0_4pathE':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:637: undefined reference to `boost::filesystem::detail::last_write_time(boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem6renameERKNS0_4pathES3_':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:677: undefined reference to `boost::filesystem::detail::rename(boost::filesystem::path const&, boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem5spaceERKNS0_4pathE':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:698: undefined reference to `boost::filesystem::detail::space(boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem15system_completeERKNS0_4pathE':
d:\tool\boost_1_61_0/./boost/filesystem/operations.hpp:710: undefined reference to `boost::filesystem::detail::system_complete(boost::filesystem::path const&, boost::system::error_code*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(text_file_backend.o): In function `ZN5boost10filesystem4pathaSINS0_15directory_entryEEENS_9enable_ifINS0_11path_traits11is_pathableINS_5decayIT_E4typeEEERS1_E4typeERKS8_':
d:\tool\boost_1_61_0/./boost/filesystem/path.hpp:202: undefined reference to `boost::filesystem::path_traits::dispatch(boost::filesystem::directory_entry const&, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >&)'
collect2.exe: error: ld returned 1 exit status

Notice that all Boost Log library were failed. When things are working fine on Linux platform, but error on the other, I have a very strong feeling that this could be the compiler version issue. Base on my pass experience and common sense of programming knowledge, I make a quit decision on the MinGW to switch to the Microsoft C++ compiler.

Going through all the hassle setup on the QT + MSVC compiler, together with the same configuration on the .pro file:

...

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib64-msvc-14.0/ -lboost_system-vc140-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib64-msvc-14.0/ -lboost_system-vc140-mt-gd-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_system

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_filesystem-vc140-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_filesystem-vc140-mt-gd-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_filesystem

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_log_setup-vc140-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_log_setup-vc140-mt-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_log-vc140-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/msvc/lib32-msvc-14.0/ -lboost_log-vc140-mt-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log

...

The code was compiled successfully at least. And now I am declaring to myself to abandon and give up any use of MinGW compiler with Boost library on Windows platform. I am very serious about this matter. (>.<)

Undefined reference to WinMain@16 in Boost Test

My bad!! Ever wonder what causes this error in my Boost Test?

crt0_c.c:-1: error: undefined reference to `WinMain@16'
collect2.exe:-1: error: error: ld returned 1 exit status

I was in the midst of preparing a new test script for my unit test, but somehow I was so struggling when I got hit on that error. Googling around couldn't find any answer, I sit back and doing code review again on my unit test script. What is surprising me is that I was missing the one time setup code in the beginning of the code:

#define BOOST_TEST_DYN_LINK

#define BOOST_NO_CXX11_SCOPED_ENUMS
#define BOOST_TEST_MODULE "syncFile core function"

#include <initializer_list>
#include <boost/test/unit_test.hpp>

...

The first line is very important, this is the crucial part of the success or failure of the compilation. What if I have multiple unit test source file, and BOOST_TEST_DYN_LINK was declared in both files? I am using QT compiler, script below shows 2 unit test source files was included in the project:

...

SOURCES += \
    ../unittest/testCaptureFiles.cpp \
    ../unittest/testlab.cpp

...

This is what I got from the compiler:

D:\tool\boost_1_61_0\boost\test\unit_test_suite.hpp:338: error: multiple definition of `init_unit_test()'
D:\tool\boost_1_61_0\boost\test\unit_test.hpp:62: error: multiple definition of `main'
collect2.exe:-1: error: error: ld returned 1 exit status

So please be careful on using Boost Test.

A WSDL requires input for processing

Although I have the WSDL successfully loaded into SoapUI, but the structure was incorrect. Following is the SOAP request message generated by default when the WSDL first loaded into SoapUI:

<soap:envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
   <soap:header/>
   <soap:body>
</soap:Envelope>

I'll get an error message says line -1: null when I validate on this request message. To fix this error, I must have at least one element in the body tag. Here is the new WSDL code for the input:

...

<wsdl:types>
  <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://helloWorld.huahsin.org/">
    <xsd:element name="Input"/>
  </xsd:schema>
</wsdl:types>

< wsdl:message name="HelloWorldRequest">
  < wsdl:part name="parameters" element="tns:Input"/>
< /wsdl:message>
...

Now the SOAP request message will look like this:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Header/>
   <soap:Body>
      <hel:Input>?</hel:Input>
   </soap:Body>
</soap:Envelope>

In the real world, it is impossible to have this empty SOAP request message, some input would require to pass in for processing. For the sake of this experiment, I would need a name and an age field. The name field is a mandatory field, whereas age field is optional.

...

<wsdl:types>
  <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://helloWorld.huahsin.org/">
    <xsd:element name="Input">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="name" type="xsd:string" minOccurs="1"/>
          <xsd:element name="age" type="xsd:integer" minOccurs="0"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
  </xsd:schema>
</wsdl:types>

With this new WSDL design, the SOAP message now will be much better and complete:

< soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:hel="http://helloWorld.huahsin.org/">
   <soap:Header/>
   <soap:Body>
      <hel:Input>
         <name>?</name>
         <!--Optional:-->
         <age>?</age>
      </hel:Input>
   </soap:Body>
</soap:Envelope>

Boost Log can't live without thread library

In one of my experiment project, I have the Boost Log configure in QT's build path, see below:

...

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log

...

End up when I try to compile this program, I get these bunch of undefined error on boost::detail::set_tss_data() in my console output window:

g++ -Wl,-subsystem,console -mthreads -o debug/backupUtilUnitTest.exe debug/FileHelper.o debug/FilePath.o debug/filenode.o debug/testlab.o  -LD:/tool/boost_1_61_0/stage/lib -lboost_system-mgw53-mt-d-1_61 -lboost_filesystem-mgw53-mt-d-1_61 -lboost_timer-mgw53-mt-d-1_61 -lboost_chrono-mgw53-mt-d-1_61 -lboost_log_setup-mgw53-mt-d-1_61 -lboost_log-mgw53-mt-d-1_61 -lboost_regex-mgw53-mt-d-1_61 -lboost_unit_test_framework-mgw53-mt-d-1_61 -LD:/tool/Qt/5.7/mingw53_32/lib D:/tool/Qt/5.7/mingw53_32/lib/libQt5Cored.a 
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZN5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEED1Ev':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZNK5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEE3getEv':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZN5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEE5resetEPS5_':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `get':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `reset':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `get':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `reset':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `~thread_specific_ptr':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(severity_level.o): In function `ZN5boost11this_thread14at_thread_exitINS_3_bi6bind_tINS2_11unspecifiedENS_15checked_deleterIyEENS2_5list1INS2_5valueIPyEEEEEEEEvT_':
d:\tool\boost_1_61_0/./boost/thread/detail/thread.hpp:862: undefined reference to `boost::detail::add_thread_exit_function(boost::detail::thread_exit_function_base*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(thread_id.o): In function `at_thread_exit<boost::log::v2s_mt_nt5::aux::this_thread:: anonymous="" id_storage::deleter="" namespace="">':
d:\tool\boost_1_61_0/./boost/thread/detail/thread.hpp:862: undefined reference to `boost::detail::add_thread_exit_function(boost::detail::thread_exit_function_base*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(once_block.o): In function `ZN5boost6detail19basic_cv_list_entry4waitENS0_7timeoutE':
d:\tool\boost_1_61_0/./boost/thread/win32/condition_variable.hpp:94: undefined reference to `boost::this_thread::interruptible_wait(void*, boost::detail::timeout)'
collect2.exe: error: ld returned 1 exit status

As of my finding from the forum, they mention that it requires to compile with -pthread option. Unfortunately, that solution is only applicable to Linux. Since I'm using Boost, the only option I have is to link the Boost Thread library:

...

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread

...

And the compilation proceeds successfully.

Illegal extension attribute 'soapAction'

I got it up!! Finally, I got my first dummy WSDL, first created with notepad, load into SoapUI. Since there is no tool could allow me to do it in easy way, I have to verify the syntax with my own eyes.

On my first try, I got this stupid error when I first load into SoapUI. The cause of the error happened in wsdl:operation and soapAction is an illegal attribute? Ehrrr~ What is that means?

WSDLException (at /wsdl:definitions/wsdl:binding/wsdl:operation): faultCode=INVALID_WSDL: Encountered illegal extension attribute 'soapAction'. Extension attributes must be in a namespace other than WSDL's.    

I take a closer look into my first baby, then only I notice that I have mistakenly mess up the wsdl:operation with soapAction. This is something bad without a good tool to gauge my work.

<wsdl:binding name="HelloWorldSOAP12" type="tns:HelloWorldPort">
  <soap12:binding transport="http://schemas.xmlsoap.org/soap/http"/>
  <wsdl:operation name="HelloWorld" soapAction="http://helloWorld.huahsin68.org/helloWorld"/>
  <wsdl:input>
    <soap12:body use="literal"/>
  <wsdl:output>
    <soap12:body use="literal"/>
  </wsdl:output>
</wsdl:binding>

The fix is as follows:

<wsdl:binding name="HelloWorldSOAP12" type="tns:HelloWorldPort">
  <soap12:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
  <wsdl:operation name="HelloWorld">
    <soap12:operation soapAction="HelloWorldAction"/>
    <wsdl:input>
      <soap12:body use="literal"/>
    </wsdl:input>
    <wsdl:output>
      <soap12:body use="literal"/>
    </wsdl:output>
  </wsdl:operation>
</wsdl:binding>

Boost Intrusive container must be clear before exits

A few weeks ago when I was working on a problem involving linked list, I was stuck in an interesting test case on Boost Intrusive. The error I'm getting is coming from the comment of a source code as seen below:

void destructor_impl(Hook &hook, detail::link_dispatch<safe_link>)
{  //If this assertion raises, you might have destroyed an object
   //while it was still inserted in a container that is alive.
   //If so, remove the object from the container before destroying it.
   (void)hook; BOOST_INTRUSIVE_SAFE_HOOK_DESTRUCTOR_ASSERT(!hook.is_linked());
}

I was in debug mode, and the program was stopped right there. At first I had overlooked into this important piece, but then later only I got to realize that the comment is giving some clue over there. The comment has clearly stated that I must have the container to be clear before destroying it. So what I did in my class was I extend the list_base_hook:

class FileNode : public list_base_hook<> {
public:
    string name;
    char type;

public:
    FileNode() {}

    void setName(string name) { this->name = name; }
    void setType(char type) { this->type= type; }
};

typedef boost::intrusive::list<FileNode> FileNodeList;

And this is my test case that causes the error:

BOOST_AUTO_TEST_CASE(TL_2)
{
    FileNodeList fnl;

    FileNode fn1;
    fn1.name = "path_A";

    assert(fn1.is_linked() == false);

    FileNode fn2;
    fn2.name = "fileA";

    fnl.push_front(fn1);

    fnl.clear(); // error will be thrown if this code is removed!
}

Do take note the above class FileNode does not take any template argument. There is another version using auto_unlink as template argument, just like below:

class FileNode : public list_base_hook<link_mode<auto_unlink> > { 
    ...
    ...
}; 
 
typedef boost::intrusive::list<FileNode, constant_time_size<false> > FileNodeList;

This feature does not require me to clear the container when the container's destructor is invoked, which happened when the function is exited.

Make path-to-string utility as static function

I have a string utility that converts the boost::filesystem::path into a string. The current implementation was done in the following way.

/** header file **/

using convert_typeX = std::codecvt_utf8<wchar_t>;

class Path {

   ...
   ...

private:
   wstring_convert<convert_typeX, wchar_t> converterX;

};


/** implementation file **/

Path::Path(path thePath)
    : rootPath(thePath), searchDone(false)
{
    string filename = converterX.to_bytes(thePath.wstring());
    string rootPath = converterX.to_bytes(this->rootPath.wstring());

    ...
}

This utility is spread across in every class that require path-to-string conversion. After some time when I look back on this code I felt that the design approach is very unpractical and quite troublesome too. Since this utility is used in everywhere, I just wonder, would it be better if I make it static. I'm just trying to simplify things, complex thing is tiring me.

/** header file **/

using convert_typeX = std::codecvt_utf8<wchar_t>;

class FileHelper {

   ...
   ...

public:
   static string convertPathToString(boost::filesystem::path thePath) {
      wstring_convert<convert_typeX, wchar_t> converterX;
      return converterX.to_bytes(thePath.wstring());
   }


private:
   wstring_convert<convert_typeX, wchar_t> converterX;

};


/** implementation file **/

Path::Path(path thePath)
    : rootPath(thePath), searchDone(false)
{
    string filename = FileHelper::convertPathToString(thePath);
    string rootPath = FileHelper::convertPathToString(this->rootPath);

    ...
}

With the new approach, I centralize that piece into FileHelper (a utility class specializes in file). When Path objects need it, just direct call on convertPathToString() from the class.

Constructor pattern or code design error?

There is a minor defect happened in my program where the program only able to capture the files in the last recursive loop. Below is the code snippet of the code:

void FilePath::captureFiles(path searchPath) {

            ...
            ...

            // it is a directory
            if( is_directory(status(*it)) ) {
                BOOST_LOG_TRIVIAL(info) << "The file is a directory";

                captureFiles(*it);
            }
            // it is a file
            else {
                ...
                ...

                if( bPlus ) {
                    bPlus = false;
                    indexFile << "+" << filename << endl;

                    BOOST_LOG_TRIVIAL(info) << "FilePath::captureFiles : The file is capture into index file with '+' sign";
                }
                else if( !bMinus ) {
                    bMinus = false;
                    indexFile << filename << endl;

                    BOOST_LOG_TRIVIAL(info) << "FilePath::captureFiles : The file is capture into index file";
                }
            }

    indexFile.close();

    ...
    ...
}

The root cause happened when there is no more directory were found in the path and the program will flow into the else section. Once the file name was captured into indexFile, the indexFile were close!

Then where does this variable get initialized?

FilePath::FilePath(path thePath)
    : rootPath(thePath), searchDone(false)
{
    ...
    ...

    // capture the index file content into memory
    indexFilePath = rootPath + string(1, path::preferred_separator) + filename + ".i";

    if( exists(indexFilePath) ) {
        captureExistingFile();
        bFileExist = true;
    }

    indexFile.open(indexFilePath, ios_base::in | ios_base::out | ios_base::app);
}

Notice that the indexFile only got initialize once in the constructor, and its memory was free in the last loop of the recursive function, I have no way to reopen it inside the recursive loop.

A good programming practice for this case is that close it in the destructor since it got initialize in the constructor, then the problem will fix automatically.

FilePath::~FilePath()
{
    ...

    if( indexFile.is_open() )
        indexFile.close();
}

Using Boost.Log in the project

Boost.Log, just like other module, it must be built first before it can be used. But most of the time I will just go for the default compilation, just like below

> ./b2 --with-log 

According to this documentation, I can have a lot more option to the build. Say for example -DBOOST_LOG_DYN_LINK, below is the definition of the macro:

BOOST_LOG_DYN_LINK

If defined in user code, the library will assume the binary is built as a dynamically loaded library ("dll" or "so"). Otherwise it is assumed that the library is built in static mode. This macro must be either defined or not defined for all translation units of user application that uses logging. This macro can help with auto-linking on platforms that support it.

When I build with -DBOOST_LOG_DYN_LINK just like the sample below:

> ./b2 --with-log define=BOOST_LOG_DYN_LINK

I must have the following statement in the .pro file:

DEFINES += BOOST_LOG_DYN_LINK

Take a closer look into the console output. Notice that BOOST_LOG_DYN_LINK option is added into g++ build command:

g++ -c -pipe -g -std=gnu++0x -Wall -W -D_REENTRANT -fPIC -DBOOST_LOG_DYN_LINK -DQT_QML_DEBUG 
-DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I../../backupUtil -I. -I../../../tool/boost_1_61_0 -I../../backupUtil 
-I../../../tool/Qt5.6.0/5.6/gcc_64/include -I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtWidgets 
-I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtGui -I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtCore -I. -I. 
-I../../../tool/Qt5.6.0/5.6/gcc_64/mkspecs/linux-g++ -o main.o ../main.cpp

If the above define statement is missing from the .pro file, the compiler will throw out the link error. Tracing that error could lead to a wrong way fixing the problem because the root cause wasn't there. This documentation would give some hints on how this thing could be fixed. Below is the important text from that documentation:

<version><linkage>_<threading>_<system>

• The <version> component describes the library major version. It is currently v2.
• The <linkage> component tells whether the library is linked statically or dynamically. It is s if the library is linked statically and empty otherwise.
• The <threading> component is st for single-threaded builds and mt for multi-threaded ones.
• The <system> component describes the underlying OS API used by the library. Currently, it is only specified for multi-threaded builds. Depending on the target platform and configuration, it can be posix, nt5 or nt6.

Same to the default Boost.Log build without BOOST_LOG_LINK, link error will be throw if the define statement was declared in the .pro file.

replace multiple character in a string

As mention before, unit test is just a way to gauge my code doing the right thing. But I still miss the functional test. Take the following piece as an example:

    int seperatorPos = filename.find("/");
    if(seperatorPos > 0) {
        filename.replace(seperatorPos, 1, "_");
    }

This piece supposes to construct the index file name from the given path. For example, if the given path is path_A/path_B, then the index file name should be path_A_path_B. The unit test has done a pretty good job, but in a real world environment, the path could be path_A/path_B/path_C, in such a situation, that piece could fail. This is the drawback of unit test, it can't simulate the actual use case. To fix the defects, that piece needs to identify when it has searched through the entire string. As you guess it, I need a while loop to do this, here is the solution:

    int seperatorPos = filename.find("/");
    while( seperatorPos != string::npos ) {
        filename.replace(seperatorPos, 1, "_");
        seperatorPos = filename.find("/");
    }

Then this is the final piece that meets my requirement.