Sunday, August 13, 2017

Boost Log can't live without thread library

In one of my experiment project, I have the Boost Log configure in QT's build path, see below:
...

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log_setup

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_log

...
End up when I try to compile this program, I get these bunch of undefined error on boost::detail::set_tss_data() in my console output window:
g++ -Wl,-subsystem,console -mthreads -o debug/backupUtilUnitTest.exe debug/FileHelper.o debug/FilePath.o debug/filenode.o debug/testlab.o  -LD:/tool/boost_1_61_0/stage/lib -lboost_system-mgw53-mt-d-1_61 -lboost_filesystem-mgw53-mt-d-1_61 -lboost_timer-mgw53-mt-d-1_61 -lboost_chrono-mgw53-mt-d-1_61 -lboost_log_setup-mgw53-mt-d-1_61 -lboost_log-mgw53-mt-d-1_61 -lboost_regex-mgw53-mt-d-1_61 -lboost_unit_test_framework-mgw53-mt-d-1_61 -LD:/tool/Qt/5.7/mingw53_32/lib D:/tool/Qt/5.7/mingw53_32/lib/libQt5Cored.a 
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZN5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEED1Ev':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZNK5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEE3getEv':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(core.o): In function `ZN5boost19thread_specific_ptrINS_3log10v2s_mt_nt54core14implementation11thread_dataEE5resetEPS5_':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `get':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `reset':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `get':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `reset':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(record_ostream.o): In function `~thread_specific_ptr':
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
d:\tool\boost_1_61_0/./boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(severity_level.o): In function `ZN5boost11this_thread14at_thread_exitINS_3_bi6bind_tINS2_11unspecifiedENS_15checked_deleterIyEENS2_5list1INS2_5valueIPyEEEEEEEEvT_':
d:\tool\boost_1_61_0/./boost/thread/detail/thread.hpp:862: undefined reference to `boost::detail::add_thread_exit_function(boost::detail::thread_exit_function_base*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(thread_id.o): In function `at_thread_exit<boost::log::v2s_mt_nt5::aux::this_thread:: anonymous="" id_storage::deleter="" namespace="">':
d:\tool\boost_1_61_0/./boost/thread/detail/thread.hpp:862: undefined reference to `boost::detail::add_thread_exit_function(boost::detail::thread_exit_function_base*)'
D:/tool/boost_1_61_0/stage/lib/libboost_log-mgw53-mt-d-1_61.a(once_block.o): In function `ZN5boost6detail19basic_cv_list_entry4waitENS0_7timeoutE':
d:\tool\boost_1_61_0/./boost/thread/win32/condition_variable.hpp:94: undefined reference to `boost::this_thread::interruptible_wait(void*, boost::detail::timeout)'
collect2.exe: error: ld returned 1 exit status
As of my finding from the forum, they mention that it requires to compile with -pthread option. Unfortunately, that solution is only applicable to Linux. Since I'm using Boost, the only option I have is to link the Boost Thread library:
...

win32:CONFIG(release, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread-mgw53-mt-1_61
else:win32:CONFIG(debug, debug|release): LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread-mgw53-mt-d-1_61
else:unix:!macx: LIBS += -L$$PWD/../../../tool/boost_1_61_0/stage/lib/ -lboost_thread

...
And the compilation proceeds successfully.

Tuesday, August 8, 2017

Illegal extension attribute 'soapAction'

I got it up!! Finally, I got my first dummy WSDL, first created with notepad, load into SoapUI. Since there is no tool could allow me to do it in easy way, I have to verify the syntax with my own eyes.

On my first try, I got this stupid error when I first load into SoapUI. The cause of the error happened in wsdl:operation and soapAction is an illegal attribute? Ehrrr~ What is that means?
WSDLException (at /wsdl:definitions/wsdl:binding/wsdl:operation): faultCode=INVALID_WSDL: Encountered illegal extension attribute 'soapAction'. Extension attributes must be in a namespace other than WSDL's.    
I take a closer look into my first baby, then only I notice that I have mistakenly mess up the wsdl:operation with soapAction. This is something bad without a good tool to gauge my work.
<wsdl:binding name="HelloWorldSOAP12" type="tns:HelloWorldPort">
  <soap12:binding transport="http://schemas.xmlsoap.org/soap/http"/>
  <wsdl:operation name="HelloWorld" soapAction="http://helloWorld.huahsin68.org/helloWorld"/>
  <wsdl:input>
    <soap12:body use="literal"/>
  <wsdl:output>
    <soap12:body use="literal"/>
  </wsdl:output>
</wsdl:binding>
The fix is as follows:
<wsdl:binding name="HelloWorldSOAP12" type="tns:HelloWorldPort">
  <soap12:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
  <wsdl:operation name="HelloWorld">
    <soap12:operation soapAction="HelloWorldAction"/>
    <wsdl:input>
      <soap12:body use="literal"/>
    </wsdl:input>
    <wsdl:output>
      <soap12:body use="literal"/>
    </wsdl:output>
  </wsdl:operation>
</wsdl:binding>

Sunday, June 4, 2017

Boost Intrusive container must be clear before exits

A few weeks ago when I was working on a problem involving linked list, I was stuck in an interesting test case on Boost Intrusive. The error I'm getting is coming from the comment of a source code as seen below:
void destructor_impl(Hook &hook, detail::link_dispatch<safe_link>)
{  //If this assertion raises, you might have destroyed an object
   //while it was still inserted in a container that is alive.
   //If so, remove the object from the container before destroying it.
   (void)hook; BOOST_INTRUSIVE_SAFE_HOOK_DESTRUCTOR_ASSERT(!hook.is_linked());
}
I was in debug mode, and the program was stopped right there. At first I had overlooked into this important piece, but then later only I got to realize that the comment is giving some clue over there. The comment has clearly stated that I must have the container to be clear before destroying it. So what I did in my class was I extend the list_base_hook:
class FileNode : public list_base_hook<> {
public:
    string name;
    char type;

public:
    FileNode() {}

    void setName(string name) { this->name = name; }
    void setType(char type) { this->type= type; }
};

typedef boost::intrusive::list<FileNode> FileNodeList;
And this is my test case that causes the error:
BOOST_AUTO_TEST_CASE(TL_2)
{
    FileNodeList fnl;

    FileNode fn1;
    fn1.name = "path_A";

    assert(fn1.is_linked() == false);

    FileNode fn2;
    fn2.name = "fileA";

    fnl.push_front(fn1);

    fnl.clear(); // error will be thrown if this code is removed!
}
Do take note the above class FileNode does not take any template argument. There is another version using auto_unlink as template argument, just like below:
class FileNode : public list_base_hook<link_mode<auto_unlink> > { 
    ...
    ...
}; 
 
typedef boost::intrusive::list<FileNode, constant_time_size<false> > FileNodeList;
This feature does not require me to clear the container when the container's destructor is invoked, which happened when the function is exited.

Friday, March 17, 2017

Make path-to-string utility as static function

I have a string utility that converts the boost::filesystem::path into a string. The current implementation was done in the following way.
/** header file **/

using convert_typeX = std::codecvt_utf8<wchar_t>;

class Path {

   ...
   ...

private:
   wstring_convert<convert_typeX, wchar_t> converterX;

};


/** implementation file **/

Path::Path(path thePath)
    : rootPath(thePath), searchDone(false)
{
    string filename = converterX.to_bytes(thePath.wstring());
    string rootPath = converterX.to_bytes(this->rootPath.wstring());

    ...
}
This utility is spread across in every class that require path-to-string conversion. After some time when I look back on this code I felt that the design approach is very unpractical and quite troublesome too. Since this utility is used in everywhere, I just wonder, would it be better if I make it static. I'm just trying to simplify things, complex thing is tiring me.
/** header file **/

using convert_typeX = std::codecvt_utf8<wchar_t>;

class FileHelper {

   ...
   ...

public:
   static string convertPathToString(boost::filesystem::path thePath) {
      wstring_convert<convert_typeX, wchar_t> converterX;
      return converterX.to_bytes(thePath.wstring());
   }


private:
   wstring_convert<convert_typeX, wchar_t> converterX;

};


/** implementation file **/

Path::Path(path thePath)
    : rootPath(thePath), searchDone(false)
{
    string filename = FileHelper::convertPathToString(thePath);
    string rootPath = FileHelper::convertPathToString(this->rootPath);

    ...
}
With the new approach, I centralize that piece into FileHelper (a utility class specializes in file). When Path objects need it, just direct call on convertPathToString() from the class.

Thursday, March 2, 2017

Constructor pattern or code design error?

There is a minor defect happened in my program where the program only able to capture the files in the last recursive loop. Below is the code snippet of the code:
void FilePath::captureFiles(path searchPath) {

            ...
            ...

            // it is a directory
            if( is_directory(status(*it)) ) {
                BOOST_LOG_TRIVIAL(info) << "The file is a directory";

                captureFiles(*it);
            }
            // it is a file
            else {
                ...
                ...

                if( bPlus ) {
                    bPlus = false;
                    indexFile << "+" << filename << endl;

                    BOOST_LOG_TRIVIAL(info) << "FilePath::captureFiles : The file is capture into index file with '+' sign";
                }
                else if( !bMinus ) {
                    bMinus = false;
                    indexFile << filename << endl;

                    BOOST_LOG_TRIVIAL(info) << "FilePath::captureFiles : The file is capture into index file";
                }
            }

    indexFile.close();

    ...
    ...
}
The root cause happened when there is no more directory were found in the path and the program will flow into the else section. Once the file name was captured into indexFile, the indexFile were close!

Then where does this variable get initialized?
FilePath::FilePath(path thePath)
    : rootPath(thePath), searchDone(false)
{
    ...
    ...

    // capture the index file content into memory
    indexFilePath = rootPath + string(1, path::preferred_separator) + filename + ".i";

    if( exists(indexFilePath) ) {
        captureExistingFile();
        bFileExist = true;
    }

    indexFile.open(indexFilePath, ios_base::in | ios_base::out | ios_base::app);
}
Notice that the indexFile only got initialize once in the constructor, and its memory was free in the last loop of the recursive function, I have no way to reopen it inside the recursive loop.

A good programming practice for this case is that close it in the destructor since it got initialize in the constructor, then the problem will fix automatically.
FilePath::~FilePath()
{
    ...

    if( indexFile.is_open() )
        indexFile.close();
}

Monday, February 20, 2017

Using Boost.Log in the project

Boost.Log, just like other module, it must be built first before it can be used. But most of the time I will just go for the default compilation, just like below

> ./b2 --with-log 

According to this documentation, I can have a lot more option to the build. Say for example -DBOOST_LOG_DYN_LINK, below is the definition of the macro:

BOOST_LOG_DYN_LINK If defined in user code, the library will assume the binary is built as a dynamically loaded library ("dll" or "so"). Otherwise it is assumed that the library is built in static mode. This macro must be either defined or not defined for all translation units of user application that uses logging. This macro can help with auto-linking on platforms that support it.

When I build with -DBOOST_LOG_DYN_LINK just like the sample below:

> ./b2 --with-log define=BOOST_LOG_DYN_LINK

I must have the following statement in the .pro file:

DEFINES += BOOST_LOG_DYN_LINK

Take a closer look into the console output. Notice that BOOST_LOG_DYN_LINK option is added into g++ build command:
g++ -c -pipe -g -std=gnu++0x -Wall -W -D_REENTRANT -fPIC -DBOOST_LOG_DYN_LINK -DQT_QML_DEBUG 
-DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I../../backupUtil -I. -I../../../tool/boost_1_61_0 -I../../backupUtil 
-I../../../tool/Qt5.6.0/5.6/gcc_64/include -I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtWidgets 
-I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtGui -I../../../tool/Qt5.6.0/5.6/gcc_64/include/QtCore -I. -I. 
-I../../../tool/Qt5.6.0/5.6/gcc_64/mkspecs/linux-g++ -o main.o ../main.cpp
If the above define statement is missing from the .pro file, the compiler will throw out the link error. Tracing that error could lead to a wrong way fixing the problem because the root cause wasn't there. This documentation would give some hints on how this thing could be fixed. Below is the important text from that documentation:
<version><linkage>_<threading>_<system>
  • The <version> component describes the library major version. It is currently v2.
  • The <linkage> component tells whether the library is linked statically or dynamically. It is s if the library is linked statically and empty otherwise.
  • The <threading> component is st for single-threaded builds and mt for multi-threaded ones.
  • The <system> component describes the underlying OS API used by the library. Currently, it is only specified for multi-threaded builds. Depending on the target platform and configuration, it can be posix, nt5 or nt6.
Same to the default Boost.Log build without BOOST_LOG_LINK, link error will be throw if the define statement was declared in the .pro file.

Saturday, January 21, 2017

replace multiple character in a string

As mention before, unit test is just a way to gauge my code doing the right thing. But I still miss the functional test. Take the following piece as an example:
    int seperatorPos = filename.find("/");
    if(seperatorPos > 0) {
        filename.replace(seperatorPos, 1, "_");
    }
This piece supposes to construct the index file name from the given path. For example, if the given path is path_A/path_B, then the index file name should be path_A_path_B. The unit test has done a pretty good job, but in a real world environment, the path could be path_A/path_B/path_C, in such a situation, that piece could fail. This is the drawback of unit test, it can't simulate the actual use case. To fix the defects, that piece needs to identify when it has searched through the entire string. As you guess it, I need a while loop to do this, here is the solution:
    int seperatorPos = filename.find("/");
    while( seperatorPos != string::npos ) {
        filename.replace(seperatorPos, 1, "_");
        seperatorPos = filename.find("/");
    }
Then this is the final piece that meets my requirement.