Tuesday, November 6, 2018

Found a new usage of getLoadFile()

Initially I was thinking to sunset the getLoadFile() as it's just another dump function for debugging purpose, but now I am going to need it in the search process. The original function will accept a FileNode object and then recursively loop through each path until the last object has reached, then convert each object into the URL string.
std::list<string> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<string> mlist;

    if( fileNode == nullptr || fileNode->sibling.size() == 0 )
        return mlist;

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        FileNode *n = (&*it);

        if( n->getType() == 'd' ) {
            std::list<string> subList;
            subList = getLoadedFiles(n);

            mlist.insert(mlist.begin(), subList.begin(), subList.end());
        }
        else {
            string filePath = constructPathAddress(n);
            mlist.push_back(filePath);
        }
    }

    return mlist;
}
This was originally used in path construction process to verify the path has correctly loaded into memory by verifying the URL returned. For now, I need this to retrieve all the FileNode objects that park under a particular FileNode object rather than the URL string. Thus, I am converting the list storage to store the FileNode object instead of string:
std::list<FileNode*> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<FileNode*> mlist;

    ...

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        ...

        if( n->getType() == 'd' ) {
           ...
           ...
        }
        else {
            mlist.push_back(n);
        }
    }

    return mlist;
}
On the unit testing site, originally I have the following code to verify my desired URL was loaded into the memory. The std::find is a very useful function to lookup a value in the list.
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(false)))
{
    ...

    // verify the file name was captured
    std::list<string> files = fb.getLoadedFiles(root);

    if( std::find(files.begin(), files.end(), "/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
}
Due to the change to object type, the std::find is unusable anymore. But I found a workaround on this, to lookup a value in the object list, that is to create a new functor for this purpose. The URL is now constructed and perform matching inside functor.
struct FileNodeFinder
{
    FileNodeFinder(const std::string &s) : s_(s) {}

    bool operator() (const FileNode* obj)
    {
        FileBotUnderTest fb;

        return fb.constructPathAddress(obj).compare(s_) == 0 ? true : false ;
    }

private:
    const std::string s_;
};
Thus, the new way of doing work is like this:
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(false)))
{
    ...

    // verify the file name was captured
    std::list<FileNode*>::iterator it = find_if(files.begin(),
                                                files.end(),
                                                FileNodeFinder("/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt"));
    FileNode *n = *it;

    if( it == files.end() )
        BOOST_FAIL("nothing were found");
}

Thursday, November 1, 2018

Pretty bad design on the return of constructParentPath()

Not long ago, there was some major change on constructParentPath() in order to maintain 2 different types of request, one for local home path and another one for the remote home path. The FileListType is used to determine whether this function is generated in local home path or remote home path before it get returned. See the code snippet below:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    FileNode *parent = NULL;

    ...
    ...

    for( string_split_iterator it = make_split_iterator(path, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        ...
        ...

        if( parent == NULL )
            if( listType == local )
                localList.push_back(*node);
            else
                remoteList.push_back(*node);
        else {
            parent->sibling.push_back(*node);
            node->setParentNode(parent);
        }

        ...
        ...
    }

    ...

    if( listType == local ) {
        localHome = parent;
        return localHome;
    }
    else {
        remoteHome = parent;
        return remoteHome;
    }
}
Notice the parent is assigned back to the respective home variable, I was thinking this code might need to rework if there is another third option is coming in. Pretty bad, right?!! Thus, I just feel the return code at the end is redundant. To prevent that from happening in future, I might as well just return the parent and let the caller to decide where the value should assign.
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    FileNode *parent = NULL;

    ...
    ...

    return parent;
}
As for this change, there are 2 callers on this function. One is loadLocalFS() and another one is loadRemoteFS(). Below are the changes made to adopt this new change:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    localHome = constructParentPath(inputPath);
    constructChildPath(localHome);

    ...
}


FileNode* FileBot::loadRemoteFS(vector<string> fileList)
{
    ...

    // load up the abstract path for first entry
    remoteHome = constructParentPath(fileList[0], remote);

    ...
    ...
}

Thursday, October 4, 2018

Message Queue in Liberty Profile ​

I didn't know that Liberty Profile has come with an internal embedded messaging server? This sound interesting to me, I should to give it a try now. For this to work, it required the full profile version of Liberty Profile. When I first start, I didn’t realize the one I'm using now is a web profile version, end up I'm not able to configure the messaging features in the server. I got the Liberty full profile from here. According to the tutorial found on the web, the configuration on the server.xml is pretty straightforward. Here is the work done in PTP configuration:

The <jmsactivationspec> tag is the very crucial part of connecting the embedded messaging engine to the MDB deployed in the server. The documentation mentions here has detail explanation on how the things should work. Without this tag or miss configure the id will give a friendly error reminding you that there are some problem with the tag. Below is the sample error in my case, notice the name of the id has mentioned in the error:
[WARNING ] CNTR4015W: The message endpoint for the MessageBean message-driven bean cannot be activated because the jms2-0.0.1-SNAPSHOT/MessageBean activation specification is not available. The message endpoint will not receive messages until the activation specification becomes available.
Code snippet below is a simple MDB app for the sake of this testing purpose:


Connect to MDB without using JNDI 

Now the server is ready to accept the incoming messages. The sample below is a standalone Java program that will establish a connection to Liberty Profile, it doesn’t use the JNDI to send the message to MDB. This is the hardest part as not much information available on the web. Eventually, I have identified 3 libraries which are needed in order to build this program, there are available in the WAS installation path located at AppServer/runtimes​. Here are the 3 libraries:
  1. com.ibm.ws.ejb.thinclient_8.5.0.jar
  2. com.ibm.ws.sib.client.thin.jms_8.5.0.jar
  3. com.ibm.ws.orb_8.5.0.jar
Looking at the code above, I have no idea what the setBusName() on line 16 trying to do? It will just throw an error as seen below if I remove that line:
Caused by: com.ibm.websphere.sib.exception.SIIncorrectCallException: CWSIT0003E: No busName property was found in the connection properties.
  at com.ibm.ws.sib.trm.client.ClientAttachProperties.< init>(ClientAttachProperties.java:109)
  at com.ibm.ws.sib.trm.client.TrmSICoreConnectionFactoryImpl.createConnection(TrmSICoreConnectionFactoryImpl.java:295)
  at com.ibm.ws.sib.trm.client.TrmSICoreConnectionFactoryImpl.createConnection(TrmSICoreConnectionFactoryImpl.java:222)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createCoreConnection(JmsJcaConnectionFactoryImpl.java:711)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createCoreConnection(JmsJcaConnectionFactoryImpl.java:647)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createConnection(JmsJcaConnectionFactoryImpl.java:376)
  at com.ibm.ws.sib.api.jms.impl.JmsManagedConnectionFactoryImpl.createConnection(JmsManagedConnectionFactoryImpl.java:162)
  ... 3 more

Connect to MDB with JNDI

This is much easier to implement as compare to the previous client app just because it uses JNDI. It doesn’t require any additional library from WAS server. Long story short, here is the code:

Tuesday, October 2, 2018

Resolving conflict between nVidia and mesa when upgrading Fedora 25

What a busy week! I was doing a major upgrade to the Fedora system and I was almost getting myself killed in the action. During the upgrade process, I was stopped by this nasty error:
$ sudo dnf system-upgrade download --releasever=25
...
...
...
Error: Transaction check error:
file /usr/lib64/libGLX_indirect.so.0 from install of mesa-libGL-17.0.5-3.fc25.x86_64 conflicts with file from package nvidia-driver-libs-2:378.13-3.fc24.x86_64
file /usr/lib/libGLX_indirect.so.0 from install of mesa-libGL-17.0.5-3.fc25.i686 conflicts with file from package nvidia-driver-libs-2:378.13-3.fc24.i686

Error Summary 
-------------
I spend almost one week to tackle this error, from day till night researching on the root cause. To keep the story short, 2 things need to do. First step is to uninstall the driver, I follow the instructions from this link which is dedicated for Fedora. After this step, there are some configuration files was left over in the xorg.config.d directory and some other rpm packages were still not yet remove. This is the sample output for my case:
$ rpm –qa | grep nvidia
nvidia-driver-libs-378.13-3.fc24.x86_64
nvidia-driver-libs-378.13-3.fc24.i686
nvidia-settings-378.13-1.fc24.x86_64
nvidia-libXNVCtrl-378.13-1.fc24.x86_64
dkms-nvidia-378.13-2.fc24.x86_64
nvidia-driver-378.13-3.fc24.x86_64
Then the next step would be this:

$ sudo dnf remove nvidia-kmod

Never try to uninstall individual rpm package manually, as the command above is the professional way to do it. Clean the whole chunk in one shot. Only after this command, then only the system upgrade can continue. During the last weekend, I have successfully upgraded from Fedora 24 all the way up to Fedora 27 in one shot.

Sunday, September 23, 2018

Memory leaks again when clearing memory

Arhhh! Memory leaks again?!! How many times I have been working on this shit? It has been reviewed and rework for many, many times. Remember, there are once a major changes to the code, most probably I miss out to run the unit test on Windows then now all test cases were failed in clearMemory(). For now I am reworking on the logic for clearing the memory, I hope this is the last and final.

Friday, September 21, 2018

How Boost Property Tree works in object loop?

I am in the midst of working on the extraction of file listing captured in memory into a text file. The task was simple, I will just write each file path into a line and then follow by another path in the next line. But I felt this method does not look good, I was thinking about future expansion, there would be a possibility of rework. And I hate rework. And thus, I change my direction to keep them in XML, which I felt much more organize. Besides storing the file listing, I'll also store the date captured on this listing, the destination path and the source path.

For this purpose, I'm using Boost Property Tree. According to the tutorial on Boost Property Tree, the code was very easy to handle. For that, I separate the piece away from the core object and put them into a brand new object, where the object will take the sole responsibility in handling the reading and writing of XML content. And for the sake of the whole execution, the object will have only one instance in the memory. And then I give a name for this object called Configuration.

Brilliant! When I test out the code, the content was successfully transferred into the designate file as shown in the following.

Unfortunately, in another test case, I am not able to read it back from the file, there are funny characters shown in the output on line 9 in the following code. There is no problem with the file being generated. I have no clue how this could happen. I am totally blind now.

Many hours were spent to debug this shit, and I found the root cause at last. It was really shocking on my finding. I just rework at line 8 in the above code and make it like the code shown below at line 10 and 12.

From this experience, I wonder how this Boost Property Tree is actually works in object loop?

Friday, August 31, 2018

Git local repository concept is really a confusion

Now only I got to know that there are 2 "types" of the local repository in the Git world. The first one is used to compare it against the local repository, where the command for this is git diff ; the second one is used to compare it against the remote repository, where the command for this is git diff --cached. One thing that really confuses me is there are 2 stages in local repository, one is called Changes not staged or commit, whereas the other one is called Changes to be committed

Confuse huh? Every time when I have done something, and I need to know the changes happen to my local PC, I'll use the command git status. The following will shows up:
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

 modified:   backupUtilUnitTest/backupUtilUnitTest.pro
 modified:   core/BackupHeader.h
 modified:   core/FileBot.cpp
 modified:   core/FileBot.h
 modified:   core/FileHelper.cpp
 modified:   core/FileHelper.h
 modified:   core/FileNode.h
 new file:   unittest/FileBotUnderTest.cpp
 new file:   unittest/FileBotUnderTest.h
 modified:   unittest/testloadfiles.cpp
 new file:   unittest/testsearch.cpp

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

 modified:   backupUtilUnitTest/backupUtilUnitTest.pro.user
 modified:   unittest/testlab.cpp

Untracked files:
  (use "git add <file>..." to include in what will be committed)

 backupUtil.zip
 backupUtilUnitTest/backupUtilUnitTest.pro.user.RX2720

Take closer attention here, when I have changes to my work in my local computer, only git diff is useful here. If I use the command git diff --cached will show nothing. This experiment shows that I am comparing the changes with the work locate in the local PC with the local repository locate at the level named Changes not staged for commit. When I issue the command git add , the file will be moved from the level of Changes not staged for commit to a new level named Changes to be committed. Nothing will show up if the command git diff is used. At this stage, git diff --cached can only be used, and this will only compare the file at the level named Changes to be committed to the remote repository, not the one in local PC. If anything has been changed to the file, the changes will not reflect with this command. But if I use the command git diff , this will show on the changes I have made to the file in local PC.

Monday, August 27, 2018

constructChildPath() need to be clean

Currently the constructChildPath() is taking 2 arguments, the first argument is a pointer to a FileNode type and the second argument is a string.
void FileBot::constructChildPath(FileNode *currentPath, string inputPath)
{
   ...
   ...
}
The purpose of having these 2 arguments is to allow the method to identify where should I scan the path and where should I start parking those paths in the list. I have been thinking for so long to revamp this method so that it doesn't take any argument for its process. I'm just trying to make it as clean as possible.

Here is my thought on the finding. In this method, the string argument, is the crucial part of this method because it tells me where should I start scanning the path. If it doesn't contain any value, then nothing would be scanned, the return list would be empty. A screenshot of the piece is as follows:
void FileBot::constructChildPath(FileNode *currentPath, string inputPath)
{
    vector<path> pathList;

    // do not proceed for root path
    if( inputPath == "" )
        return;

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentPath);

            currentPath->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node, inputPath + string(1, boost::filesystem::path::preferred_separator) + node->getName().string());
        }

    ...
    ...
}
As mention in the code above, this is a recursive function, it will recursively dive into itself searching for more files when it sees a directory. Hence, the second argument of this method, inputPath, is constantly extending the directory underneath the current path. This method is only accessible through following method and it is highly dependent on the constructParentPath() because it needs a very important element for child path construction:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    if( valiateRootPath(inputPath.generic_string()) == false ) {
        BOOST_LOG_TRIVIAL(info) << "Failed to validate input path";
        return nullptr;
    }

    constructParentPath(inputPath);
    constructChildPath(searchPath, inputPath.string());

    return searchPath;
}
Notice there is a junk variable called searchPath. I called it junk because it serves for temporary purpose. This variable was used to store the home path, which was generated in constructParentPath() as mention above, and then this variable will pass down to the constructChildPath(). This is what I mention the code is highly depends on others. Duhhh~

I'm restudy the code to see what I could improve about these junk code. As I first start, I remove the inputPath argument because I felt it was a redundant. It can generate on the spot from the currentPath, below is the upgrade version of this change:
void FileBot::constructChildPath(FileNode *currentPath)
{
    vector<path> pathList;

    // for the first time, retrieve the root path
    // bail otherwise
    if( currentPath == nullptr ) {
        return;
    }

    // construct path address
    string inputPath = constructPathAddress(currentPath);

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentPath);

            currentPath->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node);
        }

    ...
    ...
}
Since the string argument has been removed, the caller doesn't require to pass down the inputPath anymore and also the searchPath is no longer there. The caller has been trimmed down as below:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    constructChildPath(constructParentPath(inputPath));

    return localHome;
}
Since searchPath has already taken away, this will have a major impact on constructParentPath() as well, thus I have the following logic append to the end of the method:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    ...

    if( listType == local ) {
        localHome = parent;
        return localHome;
    }
    else {
        remoteHome = parent;
        return remoteHome;
    }

//    return searchPath;
}

Monday, July 16, 2018

A minor mistake from make_split_iterator

Recently, I found out there are some minor mistake lie underneath the following function:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath)
{
    ...
    ...

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(path, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
       FileNode *node = new FileNode();
       node->setType('d');
       node->setName(copy_range<string>(*it));

       BOOST_LOG_TRIVIAL(info) << "value inserted: [" << copy_range<string>(*it) << "]";

       if( parent == NULL )
           fileList.push_back(*node);
       else {
           parent->sibling.push_back(*node);
           node->setParentNode(parent);
       }

       parent = node;
       ...
       ...
    }
}
This mistake would not cause any major crash, it just doesn't look "pretty" in terms of behavioral design. Before I go into the details, let's consider following 2 possible inputs:
  1. /home/user
  2. C:/home/user 
The output for entry 2 will generate a nice and beautiful link-list as shown in the image below:
But entry 1 would become like this, the first node in the link-list doesn't contain any name.
 
Why this could happen? I though the code will automatically handle the first token when it is empty? And I also realized this only happened to Linux due to Linux file system structure design behave differently from Windows. To prove my assumption, I run a series of test on make_split_iterator(), and the result shows the method does return an empty string. Another test on boost::split() also having this issue. The only one success is boost::tokenizer().

BOOST_AUTO_TEST_CASE(TL_split_words, *boost::unit_test::precondition(skipTest(false)) )
{
    string win = "D:/home/user";
    string lin = "/home/user";

    vector<string> words;
    string str, firstElement;

    cout << "***** version 1 *****" << endl;
    cout << "Windows platform: ";

    str = win;
    boost::split(words, str, boost::is_any_of(GENERIC_PATH_SEPARATOR), boost::token_compress_on);

    for(string w : words) {
        cout << "[" << w << "]";
    }
    cout << endl;

    firstElement = words.at(0);

    BOOST_TEST(words.size() == 4);
    BOOST_TEST(firstElement.compare("D:") == 0);

    words.clear();
    str = lin;
    boost::split(words, str, boost::is_any_of(GENERIC_PATH_SEPARATOR), boost::token_compress_on);

    cout << "Linux platform: ";
    for(string w : words) {
        cout << "[" << w << "]";
    }
    cout << endl;

    firstElement = words.at(0);

    BOOST_TEST(words.size() == 4);
    BOOST_TEST(firstElement.compare("") == 0);

    cout << "***** version 2 *****" << endl;
    cout << "Windows platform: ";

    str = win;

    typedef split_iterator< string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(str, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        cout << "[" << copy_range<string>(*it) << "]";
    }
    cout << endl;

    cout << "Linux platform: ";
    str = lin;

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(str, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        cout << "[" << copy_range<string>(*it) << "]";
    }
    cout << endl;

    cout << "***** version 3 *****" << endl;
    cout << "Windows platform: ";

    str = win;

    boost::char_separator<char> sep{GENERIC_PATH_SEPARATOR.c_str()};
    boost::tokenizer<boost::char_separator<char> > token{str, sep};

    for( const auto &t : token ) {
        cout << "[" << t << "]";
    }
    cout << endl;

    cout << "Linux platform: ";
    str = lin;
    token = {str, sep};

    for( const auto &t : token ) {
        cout << "[" << t << "]";
    }
    cout << endl;
}
The output of this test would be like this:
***** version 1 *****
Windows platform: [D:][home][user]
Linux platform: [][home][user]
***** version 2 *****
Windows platform: [D:][home][user]
Linux platform: [][home][user]
***** version 3 *****
Windows platform: [D:][home][user]
Linux platform: [home][user]
I admit that I have overlooked the code, it is just too detail which something I have missed. Although the test doesn't fail, but the consequent impact would cause the link-list contain a node without a name.

Friday, July 6, 2018

Rebuild gSOAP from the source

I'm making a web service in C++. Never try it before, just want to find out how hard could it be? Unlike JAVA, the web service library in C++ isn't that popular, however, I still manage to get one. It is called gSOAP, I download it from sourceForge. And the experts are putting a thumbs up on it too.

For the first time, I am doing it on the Windows platform, it was working just fine. When I move on to the Linux machine, I got the following error pop up when I'm compiling my source:
In file included from soapH.h:16:0,
                 from HuahsinServiceBindingPort.nsmap:2,
                 from main.cpp:3:
soapStub.h:24:3: error: #error "GSOAP VERSION 20828 MISMATCH IN GENERATED CODE VERSUS LIBRARY CODE: PLEASE REINSTALL PACKAGE"
 # error "GSOAP VERSION 20828 MISMATCH IN GENERATED CODE VERSUS LIBRARY CODE: PLEASE REINSTALL PACKAGE"
   ^~~~~
What's happening just now? The error seems trying to remind me that the gSoap version isn't compatible with something else. This never happened in Windows. Wouldn't it be the cause of gSOAP I downloaded is for Windows only? Could it be some other thing else? Flashing back my memory what I did wrong, I downloaded the gSOAP from sourceForge, and then at the same time, I downloaded another version from Fedora repository. Then I generate the stub objects using the following command with Fedora's version, like this:
wsdl2h -o hello.h http://localhost:8080/services/hello?wsdl -t/usr/share/gsoap/WS/typemap.dat
soapcpp2 -jCL -I/usr/share/gsoap/import hello.h
And then I compile the source code with the stdsoap2.cpp from sourceForge's version.
g++ -g -o main.exe main.cpp soapC.cpp soapHuahsinServiceBindingPortProxy.cpp 
/home/kokhoe/tool/gsoap-2.8.68/gsoap/stdsoap2.cpp 
-I/home/kokhoe/tool/gsoap-2.8.68/gsoap 
-I. -std=c++11
Is this the reason why the compilation failed? Am I correct? I wasn't sure whether my assumption makes sense. As I read on the documentation, it does mention this:
This error is caused by mixing old with new gSOAP versions of the generated code and gSOAP libraries. Make sure to include the latest stdsoap2.h and link with the latest libgsoapXX.a libraries (or use the stdsoap2.c or stdsoap2.cpp source code).
I think this has given a clear sign on my mistake. I knew that I am going to make a fresh build from source, but still trying hard to find a lazy way, without making any hassle work to rebuild from the source. I am making a deep breath... thinking... and then keep on searching... for quite some time... and I found a step by step manual guide that could help me from building the source until the installation. It is accompanied with the source I downloaded. Thus, I made a bold decision to rebuild everything, including gSOAP.

During the build process, there are many libraries were missing, among those errors I have, following error tells no clue on what was missing. The compiler just stops like that with the given command like this:
gcc -DWITH_YACC -DWITH_FLEX -DSOAPCPP2_IMPORT_PATH="\"/usr/local/share/gsoap/import\"" 
-DLINUX -g -O2 -o soapcpp2 soapcpp2-soapcpp2_yacc.o soapcpp2-soapcpp2_lex.o soapcpp2-symbol2.o soapcpp2-error2.o 
soapcpp2-init2.o soapcpp2-soapcpp2.o -ly  
/usr/bin/ld: cannot find -ly
It gave me a hard time to identify the relationship between the -DWITH_YACC, -DWITH_FLEX, and -ly. There are the Bison and Flex libraries actually. Thank god. The rebuild process continues at last and my source was built with no error.

Friday, June 29, 2018

Some flaw happened in clearing the memory

There is some amendment being done in clearMemory() for a very long time ago. I can't recall what I did last time, the code seemed likely been done in a rush and there are not test well. This is the piece I mention in this post:
void FileBot::clearMemory(FileNode *fileNode)
{
    bool lastNode = false;

    // this case will happened only when the root is supply
    if( fileList.size() == 0 || (fileList.begin())->sibling.empty() ) {
        BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << endl;
        return;
    }

    // if this is the only node in the container
    // clean up the container and then bail
    FileNode *firstNode = &*(fileList.begin());

    // test on first node is empty
    if( !firstNode->sibling.empty() && firstNode->sibling.size() == 0 ) {
        BOOST_LOG_TRIVIAL(info) << "This is the only node. Root size [" << fileList.size() << "]";
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
        return;
    }

    // if this is not the only node in the container
    // clean up the container recursively
    if( fileNode == NULL ) {
       BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << fileNode;

       fileNode = &*(fileList.begin());
       fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
       lastNode = true;
    }
    else {
        BOOST_LOG_TRIVIAL(info) << "current node has sibling: " << fileNode << " : " << fileNode->getName() << " sibling : " << fileNode->sibling.size();

        if( fileNode->sibling.size() > 0 ) {
            for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); it++ ) {
                clearMemory(&*(it));
            }
        }
        else {
            cout << fileNode << " has no more sibling." << endl;
            return;
        }

        BOOST_LOG_TRIVIAL(info) << "cleaning sibling memory for this node: " << fileNode << " node name: " << fileNode->getName();
        fileNode->sibling.erase_and_dispose(fileNode->sibling.begin(), fileNode->sibling.end(), DisposeFileNode());
    }

    if( lastNode ) {
        BOOST_LOG_TRIVIAL(info) << "Reacing the root. Root size [" << fileList.size() << "]";
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
    }
}
In the past few days I sat down just to review my test case, I found out some of the code was not covered by the test cases. Those uncover pieces are 16-20, 24-30, and 49-52. I was shocked when I see those uncovered pieces. I wonder whether those pieces are unused code since none of the test cases cover them up.

I study the code carefully, test run for few times and finally I got a fantastic conclusion. There are 2 sections are required for this function to carry on its job; first section would be the validation, this will validate whether the container, as well as, the FileNode's sibling contain any elements in it? This is the one:
    // this case will happened only when the root is supply
    if( fileList.size() == 0 || (fileList.begin())->sibling.empty() ) {
        BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << endl;
        return;
    }
Whereas the second section would be the core logic focus on cleaning the container, as well as, the sibling of each FileNode. Here is the one:
    BOOST_LOG_TRIVIAL(info) << "current node has sibling: " << fileNode << " : " << fileNode->getName() << " sibling : " << fileNode->sibling.size();

    if( fileNode->sibling.size() > 0 ) {
        for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); it++ ) {
            clearMemory(&*(it));
        }
    }
    else {
         cout << fileNode << " has no more sibling." << endl;
         return;
    }

    BOOST_LOG_TRIVIAL(info) << "cleaning sibling memory for this node: " << fileNode << " node name: " << fileNode->getName();
    fileNode->sibling.erase_and_dispose(fileNode->sibling.begin(), fileNode->sibling.end(), DisposeFileNode());
Now the code is much more readable. Don't you agree? Oh! by the way, I already moved this function into the destructor since I'll invoke this function at the end of each unit test. Don't you feel much better?
FileBot::~FileBot()
{
    clearMemory(&*(fileList.begin()));
}

Invocation WsimportTool failed?!!

Hey there, I am making a new friend today. The meet this new friend in Eclipse Photon when I was creating a brand new web service project. Here is the story, when I first started the project, I only have the WSDL file ready in the source folder, and then I have the jaxws-maven-plugin in my pom file, like this:
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>jaxws-maven-plugin</artifactId>
    <version>2.5</version>
    <executions>
      <execution>
        <goals>
          <goal>wsimport</goal>
        </goals>
        <configuration>
          <keep>true</keep>
          <extension>true</extension>
          <sourceDestDir>${basedir}/src/main/java</sourceDestDir>
          <packageName>org.huahsin68.ws</packageName>
          <wsdlDirectory>${basedir}/src/main/resources</wsdlDirectory>
          <wsdlFiles>
            <wsdlFile>hello.wsdl</wsdlFile>
          </wsdlFiles>
        </configuration>
      </execution>
    </executions>
  </plugin>
Then this error immediately shown up right after the execution tag as shown below. What was that mean? I don’t even understand why the WsimportTool could fail?
Invocation of com.sun.tools.ws.wscompile.WsimportTool failed - check output (org.codehaus.mojo:jaxws-maven-plugin:2.5:wsimport:default:generate-sources)

org.apache.maven.plugin.MojoExecutionException: Invocation of com.sun.tools.ws.wscompile.WsimportTool failed - check output
 at org.codehaus.mojo.jaxws.AbstractJaxwsMojo.exec(AbstractJaxwsMojo.java:485)
 at org.codehaus.mojo.jaxws.WsImportMojo.processLocalWsdlFiles(WsImportMojo.java:339)
 at org.codehaus.mojo.jaxws.WsImportMojo.executeJaxws(WsImportMojo.java:292)
 at org.codehaus.mojo.jaxws.MainWsImportMojo.executeJaxws(MainWsImportMojo.java:54)
 at org.codehaus.mojo.jaxws.AbstractJaxwsMojo.execute(AbstractJaxwsMojo.java:386)
 at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
 at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
 at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:172)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:1)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1$1.call(MavenBuilder.java:115)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1.call(MavenBuilder.java:105)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod.execute(MavenBuilder.java:86)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:200)
 at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:795)
 at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:216)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:259)
 at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:312)
 at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:315)
 at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:367)
 at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:388)
 at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:142)
 at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:232)
 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:60)
Googling the around some said that this is a defect from the IDE; some even said that this could be caused by the JAVA version issue; some even said JDK 8 is not supported in Eclipse Mars version. Does it true? Not really. Firstly, I issued a command mvn clean and then follow by mvn generate-sources. The error is gone! What a weird day?!!

Sunday, June 24, 2018

Verifying the "files" are being loaded correctly

I'd made a function that its sole responsibility is to verify the files have successfully loaded into memory. Following code snippet shows my hard work:
std::list<string> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<string> mlist;

    BOOST_LOG_TRIVIAL(info) << "current root address : " << fileNode;

    if( fileNode == nullptr || fileNode->sibling.size() == 0 )
        return mlist;

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        FileNode *n = (&*it);

        if( n->getType() == 'd' ) {
            std::list<string> subList;
            subList = getLoadedFiles(n);

            mlist.insert(mlist.begin(), subList.begin(), subList.end());
        }
        else {
            string filePath = constructPathAddress(n);
            mlist.push_back(filePath);

            BOOST_LOG_TRIVIAL(info) << "filePath : " << filePath;
        }
    }

    return mlist;
}
I need this function in my unit test code to ensure constructParentPath() and constructChildPath() are doing their job correctly. Take the following test case for example:
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(true)))
{
    BOOST_TEST_MESSAGE("TC8 : one file is captured without sub folder");

    string testPathA = current_path().string() + string(1, path::preferred_separator) + "FolderA";
    createTestFile(testPathA, 1);

#if defined(WIN32)
    BOOST_TEST(testPathA == "D:\\workspaceqt\\backupUtil\\backupUtilUnitTest\\FolderA");
#else
    BOOST_TEST(testPathA == "/home/kokhoe/workspaceqt/debug/FolderA");
#endif

    FileBotUnderTest fb;
    FileNode *root = nullptr;
    if( (root = fb.constructParentPath(testPathA)) == nullptr ) {
        BOOST_TEST_MESSAGE("Initialization failed.");
        return;
    }

    string path = fb.getParentPathAddress();

#if defined(WIN32)
    BOOST_TEST(path == "D:\\workspaceqt\\backupUtil\\backupUtilUnitTest\\FolderA\\");
#else
    BOOST_TEST(path == "/home/kokhoe/workspaceqt/debug/FolderA/");
#endif

    fb.constructChildPath(root);

    // verify the file name was captured
    std::list<string> files = fb.getLoadedFiles(root);

    BOOST_TEST(files.size() == 1);

#if defined(WIN32)
    if( std::find(files.begin(), files.end(), "D:/workspaceqt/backupUtil/backupUtilUnitTest/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
#else
    if( std::find(files.begin(), files.end(), "/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
#endif

    destroyTestFile(testPathA);
}
This function is gets invoked right after the "files" are loaded into memory. There are not the physical file, it's just a shadow. To effectively process the files stored in the sub folder, it will invoke itself to dive into the sub folder bringing together a reference of the current path, until it has iterated through the end of the path. That is the reason why this function has a parameter of FileNode type. But there was a question come into my mind, why do I still need to pass the root (a reference value came from constructParentPath() in the unit test mention above) into this function since the FileBot class already got a reference to this value?
class FileBot
{
   ...
   ...

private:
    // *root should never be NULL
    FileNode *root = nullptr;       // store the a reference to root path

}
I know this was stupid. The initial objective is to create a function that could verify the "files" that has been loaded into memory are done correctly. I think this function is only used for the unit test purpose. I'll just leave it as it is.

Tuesday, June 19, 2018

Making a contract first web service with Spring Boot

I have been told that Spring Boot is a great tool to learn. What do I think? In order to get my hand dirty on that, I've chosen to use it to develop a web service project. Well, I'd spend almost an hour plus to launch a web service, able to accept a request and provide a response. It was really damn amazing.

Unlike the tutorial I found on the Internet, my job scope doesn't work in that way. It is the WSDL file was created first, a.k.a. contract-first, but the tutorial I found on the Internet are contract-last. Wait, there are still hope, I found following sample that could help out.

Sample 1
@EnableWs
@Configuration
public class WebServiceConfig extends WsConfigurerAdapter
{
   @Bean(name = "services")
   public Wsdl11Definition defaultWsdl11Definition() {
      SimpleWsdl11Definition wsdl11Definition = new SimpleWsdl11Definition();
      wsdl11Definition.setWsdl(new ClassPathResource("/schema/MyWsdl.wsdl")); //your wsdl location
      return wsdl11Definition;
   }
}
Sample 2
@EnableWs
@Configuration
public class WebServiceConfig extends WsConfigurerAdapter
{
   @Bean(name = "wsdlname")
   public DefaultWsdl11Definition defaultWsdl11Definition (XsdSchema cityRequestSchema) {
      DefaultWsdl11Definition wsdl11Definition = new DefaultWsdl11Definition();
      wsdl11Definition.setRequestSuffix("ByCountry");
      wsdl11Definition.setResponseSuffix("City");
      wsdl11Definition.setPortTypeName("Hotelport");
      wsdl11Definition.setLocationUri("/ProjectName");
      wsdl11Definition.setTargetNamespace("http://spring.io/guides/gs-producing-web-service");
      wsdl11Definition.setSchema(cityRequestSchema);
      return wsdl11Definition;
   }

   @Bean
   public XsdSchema cityRequestSchema() {
      return new SimpleXsdSchema(new ClassPathResource("CityRequest.xsd"));
   }
}
Wait, I'm looking at the DefaultWsdl11Definition class, which I felt kind of weird, it doesn't seem sound right to me. Is it going to create the WSDL specification version 1.1, but my version is 1.2. I don't think this will work. Wait, the story has not ended, after a few days of searching, I manage to find a tutorial which has done a very good job on this matter. There are 2 ways to do this:

First solution
Do it in Spring configuration and load this configuration in the main class.
<beans xmlns="http://www.springframework.org/schema/beans" 
 xmlns:context="http://www.springframework.org/schema/context" 
 xmlns:jaxws="http://cxf.apache.org/jaxws" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                     http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd
                     http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

   <context:component-scan base-package="org.huahsin68.core"/>

   <jaxws:endpoint 
      id="HelloService" 
      implementor="#helloResponseImpl"
      serviceName="s:HelloService"
      address="/hello"
      xmlns:s="http://huahsin68.org/helloService>
      <jaxws:features>
         < bean class="org.apache.cxf.ext.logging.LoggingFeature"/>
      </jaxws:features>
   </jaxws:endpoint> 
</beans>
@SpringBootApplication
@ImportResource({"classpath:beans.xml"})
public class Application {
   ...
   ...
}
Second solution
Do it in Java. Create an Endpoint bean inside the configuration bean.
@Configuration
public class WebServiceConfig {

   ...
   ...

   @Bean
   public Endpoint endpoint() {
      HelloService h = new HelloService();
      EndpointImpl endpoint = new EndpointImpl(springBus(), new InfoServiceImpl());

      endpoint.getFeatures().add(new LoggingFeature());
      endpoint.setWsdlLocation("classpath:hello.wsdl");
      endpoint.setServiceName(h.getServiceName());
      endpoint.publish("/hello");

      return endpoint;
   }
}

Reference:
Building soap web service with Apache CXF and Spring Boot

Friday, June 1, 2018

Maven unable to find valid certification path

Here is the most useful command that could help clean up the mess in Maven.
mvn –U clean compile
Unfortunately, that command is giving me a nasty error complaining that Maven was not able to find a valid certificate. WHAT THE HELL?
[ERROR] Plugin org.apache.maven.plugins:maven-compiler-plugin:3.7.0 or one of its dependencies could not be resolved: 
Failed to read artifact descriptor for org.apache.maven.plugins:maven-compiler-plugin:jar:3.7.0: Could not transfer artifact org.apache.maven.plugins:maven-plugins:pom:30 
from/to central (https://repo.maven.apache.org/maven2): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target -> [Help 1]
Will this error coming from the network firewall failure? If I read the details closely, this gives me a sense that my PC have not trusted the certificate for the site. In order to proceed the compilation, I must first trust the certificate, but how? I have no idea on dealing with Maven’s security issue during my past working experience. Spending hours on this issue and found this workaround from the Internet. First, follow the following steps to download the certificate and then import the certificate into a self-made keystore:
  1. Use a browser to go to https://repo.maven.apache.org/
  2. Click on lock icon and choose "View Certificate" 
  3. Go to the "Details"
  4. tab and choose "Save to File" Choose type "Base 64 X.509 (.CER)" and save it as mavenCert.cer 
  5. keytool -import -file mavenCert.cer -keystore mavenkeystore 
Once done, compile using following command:
mvn –U clean compile -Djavax.net.ssl.trustStore=mavenkeystore
From now on, whenever I need to download a new artifact from the maven repository or to do a clean build, I am required to use the command mention above instead of mvn –U clean compile.

Reference

Wednesday, May 16, 2018

Configure MinGW in Visual Studio Code

Just make some time for me to try out the tool - Visual Studio Code (VSC). It is a new toy from Microsoft. This writing was mainly for Windows platform targeted on MinGW compiler because I found out the setup in Windows is far more "complicated" than Linux. I think this is very rare for people living in the Windows world, but using MinGW as the primary compiler. Basically, there 3 configurations needed for my use case: c_cpp_properties.json, tasks.json, and launch.json.

c_cpp_properties.json

Linux was working fine without this file, but I am not sure why this file is needed for Windows. This file basically tells the tool how those declarations/definitions are resolved in the source file. Since I’m working on Windows, I will only focus on Win32 section. In my case, I need to tell the editor that I got extra MinGW header files located in other path. These paths are added into includePath. compilerPath tell the VSC where the compiler is located, unless the path has been added to the environment variable, then this field is optional. The default intelliSenseMode value was set to msvc-x64. It has to be changed to clang-x64 because this will throw a bunch of error on the std namespace declaration. The code snippet below shows the affected section:
{
    "configurations": [
        {
            "name": "Win32",
            "compilerPath": "C:/MinGW/bin/g++.exe",
            "includePath": [
                "C:/MinGW/include",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++/tr1",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++/mingw32",
                "C:/tool/gsoap-2.8.55/gsoap",
                "${workspaceFolder}"
            ],
            "defines": [
                "_DEBUG",
                "UNICODE",
                "_UNICODE"
            ],
            "intelliSenseMode": "clang-x64",
            "browse": {
                "path": [
                    "${workspaceFolder}"
                ],
                "limitSymbolsToIncludedHeaders": true,
                "databaseFilename": ""
            },
            "cStandard": "c11",
            "cppStandard": "c++17"
        }
    ],
   … 
   …
}

tasks.json 

This file is to tell how the source is going to build. Before the file is created, I’m using the following command to build the source.
g++ -o main.exe main.cpp soapC.cpp soapServiceProxy.cpp c:/tool/gsoap-2.8.55/gsoap/stdsoap2.cpp 
-I. -Ic:/tool/gsoap-2.8.55/gsoap 
-Lc:/MinGW/lib -Lc:/MinGW/lib/gcc/mingw32/6.3.0 
-lws2_32 
-std=c++11
In the configuration file, the value of args is reflecting the command. But sometimes when the error occurred, I’ll go back to the command line to do a few rounds of testing to ensure it is working before come to the final configuration. Sometimes the build may not reflect on the latest changes, just close and reopen the VSC will resolve the issue.
"tasks": [
        {
            "label": "build testlab",
            "type": "shell",
            "command": "g++",
            "args": [
                "-g", 
                "-o", "main.exe",
                "main.cpp soapC.cpp soapServiceProxy.cpp c:/tool/gsoap-2.8.55/gsoap/stdsoap2.cpp",
                "-Ic:/tool/gsoap-2.8.55/gsoap",
                "-I.",
                "-Lc:/MinGW/lib",
                "-Lc:/MinGW/lib/gcc/mingw32/6.3.0",
                "-lws2_32",
                "-std=c++11"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]

launch.json 

This file is to tell whether the program will start in debug mode or not. Basically, this file is ready to use when it was first generated, I just update the program field point to my program name, which is main.exe.
"configurations": [
    {
        "name": "(gdb) Launch",
        "type": "cppdbg",
        "request": "launch",
        "program": "${workspaceFolder}/main.exe",
        "args": [],
        "stopAtEntry": true,
        "cwd": "${workspaceFolder}",
        "environment": [],
        "externalConsole": true,
        "MIMode": "gdb",
        "miDebuggerPath": "C:/MinGW/bin/gdb.exe",
        "setupCommands": [
            {
                "description": "Enable pretty-printing for gdb",
                "text": "-enable-pretty-printing",
                "ignoreFailures": true
            }
        ]
    }
]

Saturday, May 5, 2018

Unit test code strategy on private method

During the initial development, the code was driven by the TDD concept. Thus, I have the following code published to the world temporarily. The 2 methods are the primary methods of this program; neither one of them missing will not work.
public:
    FileNode* constructParentPath(boost::filesystem::path inputPath);
    void constructChildPath(FileNode *currentRootNode);

    ...
    ...
This design was to make sure that I have enough test coverage for each method. After unit testing have been done, I have a plan to change the visibility of that 2 primary methods to private, and replace them with a more meaningful method. The new design would look like below:
public:
    FileNode* constructAbstractPath(boost::filesystem::path inputPath);


protected:
    FileNode* constructParentPath(boost::filesystem::path inputPath);
    void constructChildPath(FileNode *currentRootNode);


FileNode* FileBot::constructAbstractPath(boost::filesystem::path inputPath)
{
    constructParentPath(inputPath);
    constructChildPath(root);

    return root;
}
Ahhaa~ Feeling so nice with this method now. But the flip site of this change will spoil my existing unit test code. Thus, I think of a new solution that wouldn't make such a huge change while adapting the new design. I made a new class that mimics the existing method call as shown in below:
class FileBotUnderTest : public FileBot
{
public:
    FileBotUnderTest();

    FileNode* constructParentPath(boost::filesystem::path inputPath) {
        return FileBot::constructParentPath(inputPath);
    }

    void constructChildPath(FileNode *currentRootNode) {
        return FileBot::constructChildPath(currentRootNode);
    }
};
Changes in unit test code are very minimal, instead of interacting with FileBot, I'll just interact with FileBotUnderTest. Below are the work done for these changes:
BOOST_AUTO_TEST_CASE(TL_6, *boost::unit_test::precondition(skipTest(true)))
{
    BOOST_TEST_MESSAGE("TC6 : load 3 level parent path");
    string path = "";
    FileBotUnderTest fb;

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:\\workspaceqt\\ui1");
    path = "D:/workspaceqt/ui1";
#else
    BOOST_TEST_MESSAGE("Test path: /home/kokhoe/workspaceqt");
    path = "/home/kokhoe";
#endif

    if( fb.constructParentPath(path) != nullptr )
       path = fb.getParentPathAddress();

#if defined(WIN32)
    BOOST_TEST(path == "D:\\workspaceqt\\ui1\\");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home/kokhoe/");
#endif
}

Tuesday, April 17, 2018

Boost Intrusive Set lookup function doesn't accept search key as object?

Would it be easier if I change the intrusive list to intrusive set? Okay, my initial goal is to capture the files into the memory, which is an intrusive list for my case, and then search through the list for each input entry. The input entry might go up 1000 items in total. Now the problem here is I had never decide what algorithm to use for the searching part. Then I come across the intrusive set does provide the API for me to work with, code sample below is my experiment on this:
class FileNode3;
typedef boost::intrusive::set<FileNode3, compare<std::greater<FileNode3> >, constant_time_size<false> > BaseSet;


class FileNode3 : public set_base_hook<link_mode<safe_link> > {
public:
    boost::filesystem::path root;
    boost::filesystem::path name;
    char type; // f is file, d is directory

    BaseSet sibling;

    FileNode3() {}
    FileNode3(boost::filesystem::path name_) : name(name_) {}
    FileNode3(boost::filesystem::path name_, const char type_) : name(name_), type(type_) {}

public:
    void setName(boost::filesystem::path name) { this->name = name; }
    void setRoot(boost::filesystem::path root) { this->root = root; }
    void setType(char type) { this->type= type; }

    friend bool operator< (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) < 0;  }

    friend bool operator> (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) > 0;  }

    friend bool operator== (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) == 0;  }
};

BOOST_AUTO_TEST_CASE(TL_IntrusiveSet, *boost::unit_test::precondition(skipTest(false)))
{
    struct FileNodeEqual
    {
        bool operator()(const char *search, const FileNode3 &node) const
        {
            cout << "StrEqual arg1: " << search << " " << node.name.c_str() << endl;

            return strcmp(search, node.name.c_str()) == 0 ? false : true;
        }

        bool operator()(const FileNode3 &node, const char *search) const
        {
            cout << "StrEqual arg2: " << search << " " << node.name.c_str() << endl;

            return strcmp(node.name.c_str(), search) == 0 ? false : true;
        }
    };

    BaseSet s1;

    FileNode3 fn3;
    fn3.name = "fileB";
    fn3.type = 'f';

    FileNode3 fn1;
    fn1.name = "path_A";
    fn1.type = 'd';
    fn1.sibling.push_back(fn3);

    FileNode3 fn2;
    fn2.type = 'f';
    fn2.name = "fileA";

    s1.insert(fn1);
    s1.insert(fn2);

    FileNode3 f;
    f.name = "fileB";
    f.type = 'd';

    boost::intrusive::set<FileNode3>::iterator it = s1.find(f.name.c_str(), FileNodeEqual());

    FileNode3 *n = (&*it);

    if( it == s1.end() ) {
        cout << "record not found" << endl;
    }
    else {
        FileNode3 *n = (&*it);
        cout << "record found: " << n->name << endl;
    }

    if( fn1.sibling.size() > 0 ) {
        cout << "fn1 sibling has more child: " << fn1.sibling.size() << endl;

        boost::intrusive::set<FileNode3>::iterator it = fn1.sibling.find(fn3);

        if( it == s1.end() ) {
            cout << "record not found" << endl;
        }
        else {
            FileNode3 *n = (&*it);
            cout << "record found: " << n->name << endl;
        }
    }

    s1.remove_node(fn1);
    s1.remove_node(fn2);
    s1.remove_node(fn3);

    cout << "Test 1 end!" << endl;
}
But that is one problem with this code, I can only compare the name property of this object. How can I compare the rest of the properties of the object? Why don't I just pass down the whole object and then do the comparison? Then I modify the FileNodeEqual struct as follows, the search key has become an FileNode3 instead of char*:
struct FileNodeEqual
    {
        bool operator()(const FileNode3 &search, const FileNode3 &node) const
        {
            cout << "StrEqual arg1: " << search.name.c_str() << " " << node.name.c_str() << endl;

            return true;
        }

        bool operator()(const FileNode3 &node, const FileNode3 &search) const
        {
            cout << "StrEqual arg2: " << search.name.c_str() << " " << node.name.c_str() << endl;

            return true;
        }
    };
Then the compiler will complain following error to me:
error: 'bool testlab::TL_IntrusiveSet::test_method()::FileNodeEqual2::operator()(const testlab::FileNode3&, const testlab::FileNode3&) const' cannot be overloaded
         bool operator()(const FileNode3 &node, const FileNode3 &search) const
              ^~~~~~~~
error: with 'bool testlab::TL_IntrusiveSet::test_method()::FileNodeEqual2::operator()(const testlab::FileNode3&, const testlab::FileNode3&) const'
         bool operator()(const FileNode3 &search, const FileNode3 &node) const
              ^~~~~~~~
I have tested with other primitive data types are working fine. Not sure whether this is due to the performance penalty when calling those functions, thus it was design not allow to take any expensive object?

Wednesday, February 28, 2018

redesigning the constructChildPath();

inpuPath a former member of class FileBot. I decided to removed this member variable in the recent design for constructChildPath(). There are 5 test cases for this design specification - capturing files, 2 of the test case were failed, which is test case 4 and 5.
  1. one file is captured without sub folder.
  2. two files are captured without sub folder.
  3. one files are captured when there is a sub folder.
  4. one files are captured in directory under test and one file in sub folder.
  5. zero file are captured in directory under test and one file in sub folder.
The root cause for this failure was due to the misused of inputPath as reference point in constructChildPath().
void FileBot::constructChildPath()
{
    vector<path> pathList;
 
    // where am I now?
    BOOST_LOG_TRIVIAL(info) << "current path: " << inputPath << inputPath.filename() << endl;
 
    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    ...
    ...
}
For this purpose, I decide to remove this member variable to make constructChildPath() more robust in processing file directory. To do this, the first make over is to build a method to construct the path to feed directory_iterator().
string FileBot::constructPathAddress(FileNode *node)
{
    if( root == nullptr )
        return "";

    string filePath = "";

    if( node->getParentNode() != nullptr )
        filePath = constructPathAddress(node->getParentNode());

    return filePath + node->getName().string() + string(1, boost::filesystem::path::preferred_separator);
}
Unlike the previous version, the constructChildPath() has now been transformed into a recursive function. Take note on the inputPath is getting the outcome from constructPathAddress() and then feed into directory_iterator().
void FileBot::constructChildPath(FileNode *currentRootNode)
{
    vector<path> pathList;

    // construct path address
    string inputPath = constructPathAddress(currentRootNode);

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentRootNode);

            currentRootNode->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node);
        }
        else {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('f');
            node->setParentNode(currentRootNode);

            currentRootNode->sibling.push_back(*node);
        }
    }
}
Last is to reassign the task of constructor to validatePath(). I just felt that this task doesn't suitable to put into the constructor since it only use by constructParentPath().
path FileBot::validatePath(boost::filesystem::path filePath)
{
    // remove trailing file separator if found any
    char lastChar = filePath.generic_string().at(filePath.generic_string().length() - 1);
    if( lastChar == '/' )
        filePath = filePath.generic_string().substr(0, filePath.generic_string().length() - 1);

    return filePath;
}
Then in constructParentPath() just call this method to update the input value again.
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath)
{
    inputPath = validatePath(inputPath);

    // bail out if the path doesn't exists
    if( !exists(inputPath.generic_string()) )
        return nullptr;

    ...
    ...
}

Sunday, February 25, 2018

Grouping test cases with test suite

Sometimes I just want to skip those unit test cases that have already been pass and focus those test cases on my current development. And then re-run again the whole unit test case again when I have finished the task to ensure no flaw happened to the other unit test cases. Sometimes it is so annoying to run all unit test cases that are not relevant to the task I'm currently working on. I found out one trick that could allow me to control which unit test code should execute and which should stop.

The code snippet below is the ordinary declaration for my unit test code, this declaration form will be executed whenever the code were run.
BOOST_AUTO_TEST_CASE(TL_1)
{
   ...
}
Then later I added some spice to tell the framework whether the unit test case should be executed or not. Following code snippet is the trick I have done for this purpose.
BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::enable_if<false>())
{
   ...
}
In my case, this would be a great help. I don't need to wait for all unit test cases are finished in order to see the result. Some more there is only one or two test cases I can really focus on for the same tasks at the same time. Now, as my unit test code are expanding, I need more flexibility and the ability to group relevant test cases together in one source file, and then I can decide which test code to run. Here is the code snippet for the solution.
BOOST_AUTO_TEST_SUITE(testlab, *boost::unit_test::label("testlab"))

BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::enable_if<false>())
{
   ...
   ...
}

BOOST_AUTO_TEST_SUITE_END()
With this code, the test case are grouped under the test suite named testlab. When the code is run, all test cases under this suite will be execute. But there is one flaw in this code, the enable_if() is not working. The test will continue to execute even though it is false. Not only that, neither disabled() and enabled() are working. Luckily precondition() comes to a rescue. The code snippet below tells the framework to continue to execute the test case.
struct skipTest
{
    bool skip;
    skipTest(bool s) : skip(s){}

    boost::test_tools::assertion_result operator()(boost::unit_test::test_unit_id)
    {
        if( skip == false )
            return true;
        else
            return false;
    }
};

BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::precondition(skipTest(false)))
{
   ...
   ...
}
As of now, the test suite will help me to control which group of test case to execute and precondition() will allow me to choose which test case should skip whenever I want during the development.

Friday, February 16, 2018

Improving the piece in constructParentPath() and constructChildPath()

In the existing FileBot class design, I have an inputPath class member variable.
class FileBot
{
public:
   FileBot(boost::filesystem::path filePath);

   int constructParentPath();
   void constructChildPath();

   ...
   ...

private:
   boost::filesystem::path inputPath;
};
The purpose of this variable is to keep track of the starting point where the files should begin to search with. It is so important that nothing is missed during the initialization, otherwise nothing will get from the search. Thus I'm doing the initialization through the constructor.
FileBot::FileBot(boost::filesystem::path filePath)
{
    // remove trailing file separator if found any
    char lastChar = filePath.generic_string().at(filePath.generic_string().length() - 1);
    if( lastChar == '/' )
        inputPath = filePath.generic_string().substr(0, filePath.generic_string().length() - 1);
    else
        inputPath = filePath;
}
In the client code, the programmer must instantiate the class through the following way. Indirectly the variable will get initialized as well.
FileBot fb("/home");

// or

FileBot *fb = new FileBot("/home");
And then this variable was also incorporated in other methods such as loading path into memory. There are 2 scenarios on loading path into memory; the first scenario is to load the parent path into memory. Say for example when the given input value is /home/user, then only the particular home and user were constructed into memory. Any other files located under the /home directory will be ignored. This is the piece of this construct:
int FileBot::constructParentPath()
{
    // bail out if the path doesn't exists
    if( !exists(inputPath.generic_string()) )
        return 0;

    // construct the parent path first
    string path = inputPath.generic_string();
    FileNode *parent = NULL;

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(path, first_finder("/", is_equal()));
         it != string_split_iterator();
         ++it)
    {
        FileNode *node = new FileNode();
        node->setType('d');
        node->setName(copy_range<string>(*it));

        cout << "value inserted: [" << copy_range<string>(*it) << "]" << endl;

        if( parent == NULL ) {
            fileList.push_back(*node);
        }
        else
            parent->sibling.push_back(*node);

        parent = node;

        cout << "parent: " << parent;
        cout << " parent value: " << parent->getName();
        cout << " sibling size: " << parent->sibling.size() << endl;
    }

    // keep the current path in memory
    root = parent;

    cout << "***** debug : root node " << root << " : " << root->getName() << endl;

    return 1;
}

The second scenario would be loading the child path into memory. This time it will load all the files under the directory and expand to load the files in any other subdirectories (if any). Continuing from the previous example, /home/user, this section will load the files under the user directory, including the underlying sub directory. The construct for this is as follows:
void FileBot::constructChildPath()
{
    vector<path> pathList;

    // where am I now?
    BOOST_LOG_TRIVIAL(info) << "current path: " << inputPath << inputPath.filename() << endl;

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        // extract the file name from the path
        file = file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length());

        if( is_directory(file) ) {
            cout << file << " is a directory." << endl;

            FileNode *node = new FileNode();
            node->setName(inputPath.filename());
            node->setType('d');

            root->sibling.push_back(*node);
        }
        else {
            FileNode *node = new FileNode();
            node->setName(file);
            node->setType('f');

            root->sibling.push_back(*node);
        }
    }
}
To ensure my code is always implemented correctly, the following test case must return a successful result:
  • load 1 level parent path.
  • load 2 level parent path.
  • load 3 level parent path.
  • file separator appear at the end of a path.
  • test on one file is captured without sub folder.
  • test on two files are captured without sub folder.
  • test on one files are captured when there is a sub folder.