Tuesday, February 1, 2022

GLSL 3.10 doesn't support core profile

Just did an upgrade on my Fedora system, something seems not good after the upgrade. The shader program was not able to compile. It shows following error:
error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES
I have no clue what this message trying tell. Make a quick check on my program to see what is the current OpenGL version running on this system. And I got this:
Supported GL version: 3.1 Mesa 21.1.8
Supported GLEW version: 2.1.0
Check with OpenGL wiki page, the supported GLSL version in OpenGL 3.1 is 1.4. Looking at my shader source file, it shows this:
#version 330 core
this was the first thing I learn in GLSL in my first day, I have never try any other version before this. If this is not correct, what would be the correct one? I just make a try as follow:
#version 140 core
Then I got this error:
vertex shader compilation error: 0:1(10): error: illegal text following version number
Hmmm... A few hours later. I got this stackoverflow mention that the core and compatibility profile were introduce in OpenGL 3.2, thus the one installed in my system now doesn't support that one. That means core never exists in GLSL version 1.4, just as shown below:
#version 140

Thursday, June 4, 2020

Fedora 31 upgrade log

Upgrading Linux is always a challenging task. It will cause my heart beating fast, my head was spinning, and I was sweating. This round I was upgrading Fedora 30 to 31, there are few errors pops up during the upgrade. The first challenge is when the package gpgme(x86-64) required by python2-gpg couldn't be found.
$ sudo dnf system-upgrade download --releasever=31

...
...

Error: 
 Problem: package python2-gpg-1.13.1-7.fc30.x86_64 requires gpgme(x86-64) = 1.13.1-7.fc30, but none of the providers can be installed
  - gpgme-1.13.1-7.fc30.x86_64 does not belong to a distupgrade repository
  - problem with installed package python2-gpg-1.13.1-7.fc30.x86_64
(try to add '--skip-broken' to skip uninstallable packages)
I found a link mention that Python 2 has end of life, and thus the dependencies required by Python 2 waren't there anymore. Googling around couldn't find a solution on this, but I found a similar problem where the workaround is to remove the package.
$ sudo dnf remove python2-gpg-1.13.1-7.fc30.x86_64

...
...

Removed:
  annobin-8.71-4.fc30.x86_64                                                    
  createrepo_c-0.15.5-1.fc30.x86_64                                             
  createrepo_c-libs-0.15.5-1.fc30.x86_64                                        
  drpm-0.4.1-1.fc30.x86_64                                                      
  dwz-0.12-10.fc30.x86_64                                                       
  efi-srpm-macros-4-2.fc30.noarch                                               
  fpc-srpm-macros-1.2-1.fc30.noarch                                             
  ghc-srpm-macros-1.4.2-9.fc30.noarch                                           
  gnat-srpm-macros-4-9.fc30.noarch                                              
  go-srpm-macros-2-19.fc30.noarch                                               
  mach-1.0.4-9.fc30.x86_64                                                      
  nim-srpm-macros-2-1.fc30.noarch                                               
  ocaml-srpm-macros-5-5.fc30.noarch                                             
  openblas-srpm-macros-2-5.fc30.noarch                                          
  perl-srpm-macros-1-29.fc30.noarch                                             
  pyliblzma-0.5.3-25.fc30.x86_64                                                
  python-srpm-macros-3-47.fc30.noarch                                           
  python2-gpg-1.13.1-7.fc30.x86_64                                              
  python2-iniparse-0.4-33.fc30.noarch                                           
  python2-pycurl-7.43.0.2-6.fc30.x86_64                                         
  python2-pyxattr-0.6.1-1.fc30.x86_64                                           
  python2-rpm-4.14.2.1-5.fc30.x86_64                                            
  python2-urlgrabber-4.0.0-3.fc30.noarch                                        
  python3-urlgrabber-4.0.0-3.fc30.noarch                                        
  qt5-srpm-macros-5.12.5-1.fc30.noarch                                          
  redhat-rpm-config-132-1.fc30.noarch                                           
  rpm-build-4.14.2.1-5.fc30.x86_64                                              
  rust-srpm-macros-10-1.fc30.noarch                                             
  yum-3.4.3-522.fc30.noarch                                                     
  yum-metadata-parser-1.1.4-22.fc29.x86_64                                      
  zstd-1.4.4-1.fc30.x86_64                                                      

Complete!
A bunch of Python 2 related dependencies was removed. Now come to the next challenge, it shows that Cannot enable multiple streams for module 'gimp'. What the hell exactly of this? 
$ sudo dnf system-upgrade download --releasever=31

...
...

terminate called after throwing an instance of 'libdnf::ModulePackageContainer::EnableMultipleStreamsException'
  what():  Cannot enable multiple streams for module 'gimp'
Aborted
There is exactly the same error has been reported to Redhat Bugzilla, but that one is caused by VirtualBox. Then I found this guide which is quite straight forward to use.
$ sudo dnf module disable gimp
Last metadata expiration check: 0:29:49 ago on Wed 03 Jun 2020 09:05:06 AM +08.
Dependencies resolved.
======================================================================================================================
 Package                     Architecture               Version                     Repository                   Size
======================================================================================================================
Disabling modules:
 gimp                                                                                                                

Transaction Summary
======================================================================================================================

Is this ok [y/N]: y
Complete!
Now only I can proceed to upgrade the system. The next biggest challenge would be the driver. This always failed me. When I compile the driver with DKMS, it failed due to the compilation error shown in the make.log:
implicit declaration of function ‘ioremap_nocache’; did you mean ‘ioremap_cache’?
As I check in the NVIDIA driver source code, there are plenty of the function named ioremap_nocache. This is so unwise if I change it manually, throughout my professional career, programming does not work in that way. After a long search on the Internet, there is a patch for NVIDIA 390.132 driver. Use that patch driver instead of the original download from the NVIDIA website. Once done, verify with the following command and I got the driver installed successfully.
$ lsb_release -a
LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	Fedora
Description:	Fedora release 31 (Thirty One)
Release:	31
Codename:	ThirtyOne

Tuesday, January 7, 2020

Declaring a class type member variable in OO way

Wonder what is wrong with my code? Am I forgotten how could I declare and initialize static member of a class?
class Configuration
{
private:
   static b2Vec2 gravity(0.0f, -0.05f);
}
Should be. I am getting this error really scratching my head hard.
/home/kokhoe/workspacec/OpenGL2/header/Configuration.h:47:24: error: expected identifier before numeric constant
   47 |  static b2Vec2 gravity(0.0f, -0.05f);
      |                        ^~~~
No. Not really. It has nothing to do with static or not. Just that I have forgotten the way to declare a class type member variable in object oriented way. The correct way of declaring a class type should be like this:
class Configuration
{
private:
   static b2Vec2 gravity = b2Vec2(0.0f, -0.05f);
}
Somehow b2Vec2 is a struct not class. This has cause the compilation error:
/home/kokhoe/workspacec/OpenGL2/header/Configuration.h:44:16: error: in-class initialization of static data member ‘b2Vec2 Configuration::gravity’ of non-literal type
   44 |  static b2Vec2 gravity = b2Vec2(0.0f, -0.05f);
      |                ^~~~~~~
Since b2Vec2 is a struct, I remove the static keyword and it compile successfully.

Sunday, May 26, 2019

Rework on loadRemoteFS

I was infested by the novel The Song of Ice and Fire, I can't concentrate on my code now. One day when I was looking into the loadRemoteFS(), reading it for a few rounds, I see something wasn't right. The section in // continue to search the path under the section // route back to home path to start the search again was not execute if I read it correctly.
    for( ; it != fileList.end(); it++ ) {
        entry = (string)*it;

        LOGGER(lg, info) << "Processing entry: " << entry << " parent path: " << entry.parent_path()
             << " file name: " << entry.filename();

        LOGGER(lg, info) << "Current path: " << constructPathAddress(currentPosition);

        // navigate to new path if the entry wasn't same with the current position
        if( tmpParentMemory.compare(entry.parent_path().string()) != 0 ) {
            // validate local path before moving on to new path

            LOGGER(lg, info) << "Navigate to new path: " << entry.parent_path();

            string key = "";

            // search forward from current position
            if( tmpParentMemory.length() < entry.parent_path().string().length() ) {
                key = entry.parent_path().string().substr(tmpParentMemory.length(), entry.parent_path().string().length() - tmpParentMemory.length());

                LOGGER(lg, info) << "Search forward to: " << key;

                currentPosition = loadRemoteSubPath(currentPosition, key.substr(1, key.length()));
            }
            // route back to home path to start the search again
            else {
                key = entry.parent_path().string();

                // continue to search the path
                if( key.length() > (remoteHome->getName().string().length() + 2) ) {
                    key = key.substr(remoteHome->getName().string().length() + 2, remoteHome->getName().string().length() + 1);
                    currentPosition = loadRemoteSubPath(remoteHome, key);

                    LOGGER(lg, info) << "Searching for " << key << " from home path";
                }
                // we have arrive to home path
                else {
                    LOGGER(lg, info) << "Don't go elsewhere, the file is right under home path";

                    currentPosition = remoteHome;
                }
            }
        }

        // construct them just as in current level
        FileNode *newNode = new FileNode();
        newNode->setName(entry.filename());
        newNode->setType('f');
        newNode->setParentNode(currentPosition);

        currentPosition->sibling.push_back(*newNode);

        LOGGER(lg, info) << "Insert " << newNode->getName() << "[" << newNode << "] into the path: " << currentPosition;

        // update the new entry into memory
        tmpParentMemory = entry.parent_path().string();
    }
I was looking at the ceiling though of something for a while. I am not thinking how could I optimize the code, but thinking what would be the next of GoT instead... and then continue my novel reading. A few days later, I did some improvement to the code, the code now looks much cleaner than before.
    for( ; it != fileList.end(); it++ ) {
        entry = (string)*it;

        FileNode *currentPosition = analyseEntryNode(entry);

        // construct them just as in current level
        FileNode *newNode = new FileNode();
        newNode->setName(entry.filename());
        newNode->setType('f');
        newNode->setParentNode(currentPosition);

        currentPosition->sibling.push_back(*newNode);

        LOGGER(lg, info) << "Insert " << newNode->getName() << "[" << newNode << "] into the path: " << currentPosition;
    }
A damn lot of junk code has been removed. I am trying to let the code reading much easier. At here, I am doing some analysis to identify the position where should the new file node to be captured. If I want to know more how things done, then I further drill down to the details. I put them into a new method as shown below:
FileNode* FileBot::analyseEntryNode(path entry)
{
    string curLocation = Configuration::getInstance()->getRemotePath();
    FileNode *curPos = remoteHome;

    LOGGER(lg, info) << "Processing entry: " << entry << " parent path: " << entry.parent_path()
         << " file name: " << entry.filename();

    LOGGER(lg, info) << "Current path: " << constructPathAddress(curPos);

    // navigate to new path if the entry wasn't same with the current position
    if( curLocation.compare(entry.parent_path().string()) != 0 ) {
        // validate local path before moving on to new path

        LOGGER(lg, info) << "Navigate to new path: " << entry.parent_path();

        string key = "";

        // search forward from current position
        if( curLocation.length() < entry.parent_path().string().length() ) {
            key = entry.parent_path().string().substr(curLocation.length(), entry.parent_path().string().length() - curLocation.length());

            LOGGER(lg, info) << "Search forward to: " << key;

            curPos = lookupPath(curPos, key.substr(1, key.length()));
        }
    }

    return curPos;
}

Wednesday, February 20, 2019

New implementation on logger

I suppose to work on the logger but due to some unit test code has not been integrated with the PUGI XML, I worry that base code might have some flaw in it. I have to focus myself to complete this before moving on to the next task. It took me a few weeks to clean my mess. For now, it is time for me to clear my doubt on the logger. Before this, I am using a very simple method to show my log in the console output:
...

BOOST_LOG_TRIVIAL(info) << "my log message";

...
I don't have a proper file logger implement yet, I've been thinking to use log4cxx but then I give up since I begin it with Boost Log. From my experience on log4j, to implement a file logger in Boost Log, it would be a simple task to do provided I have followed the documentation. Otherwise, it's going to be so hard to do it. I found a working sample used for logging the file name and file number in the log in StackOverflow. This is what I need to do for my case, I define my logger in FileBot.h:
...

#define LOGGER(logger, sev) \
    BOOST_LOG_STREAM_WITH_PARAMS( \
        (logger), \
        (set_get_attrib("FILE", path_to_filename(__FILE__))) \
        (set_get_attrib("LINE", __LINE__)) \
        (::boost::log::keywords::severity = (boost::log::trivial::sev)) \
    )

...
The the bunch of utility function mention in the above #define in CPP file.
...

template<typename valuetype="">
ValueType set_get_attrib(const char* name, ValueType value)
{
    auto attr = logging::attribute_cast<logging::attributes::mutable_constant aluetype="">>(logging::core::get()->get_thread_attributes()[name]);
    attr.set(value);
    return attr.get();
}

std::string path_to_filename(std::string path)
{
    return path.substr(path.find_last_of("/\\")+1);
}

...
And then the initialize code in FileBot constructor:
...

FileBot::FileBot()
{
    logging::core::get()->add_thread_attribute("FILE", boost::log::attributes::mutable_constant<string>(""));
    logging::core::get()->add_thread_attribute("LINE", boost::log::attributes::mutable_constant<int>(0));

    logging::add_file_log
    (
       logging::keywords::file_name = "/home/kokhoe/workspaceqt/debug/sample_%N.log",
       logging::keywords::rotation_size = 10 * 1024 * 1024,
       logging::keywords::time_based_rotation = logging::sinks::file::rotation_at_time_point(0, 0, 0),
       logging::keywords::format = (
                expr::stream
                      << expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d %H:%M:%S")
                      << " [" << boost::log::trivial::severity << "] "
                      << '['   << expr::attr<std::string>("FILE")
                               << ':' << expr::attr<int>("LINE") << "] "
                      << expr::smessage
       )
    );

    logging::add_common_attributes();

...
The usage of the code is simply
...

LOGGER(lg, info) << "my log message";
...
Then the output will show this:

2019-02-19 22:03:26 [info] [FileBot.cpp:426] my log message

Monday, January 28, 2019

Integration of Configuration class and FileBot class

Since I have the Configuration class ready, is time to integrate it into my FileBot class. As for now, only loadRemoteFS() is requiring data from INDEX file. Thus, integration should be much easier. Before this, I am using a list for temporary mimic the data available from INDEX file. This is what I do in my unit test (testSearch.cpp):
/*****   FileBot.cpp   *****/

std::list<string> FileBot::loadRemoteFS(vector<string> fileList, bool showCaptureList) {
   
   for( vector<string>::iterator it = fileList.begin(); it != fileList.end(); it++ ) {
   }

   ...
   ...
}


/*****   testSearch.cpp   *****/

BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(false)))
{
    ...
    ...

    vector<string> keyList;
    keyList.push_back("/home/puiyee/workspaceqt/debug/FolderA");
    keyList.push_back("/home/puiyee/workspaceqt/debug/FolderA/subA/subB/file_3.txt");
    vector<string> found;

    fb.loadRemoteFS(keyList);

    ...
}
Now I have replaced this chunk of code with a more elegant piece. I am creating a real XML file to mimic the INDEX file in createDestinationConfig() during the test. And now the loadRemoteFS() is no longer taking any std::list as input parameter, it will digest the INDEX file, it knows what data to look for, shallow it, and then produce the output.
/*****   FileBot.cpp   *****/

FileNode* FileBot::loadRemoteFS()
{
   ...

   pugi::xpath_node_set fileList = Configuration::getInstance()->retrieveRemoteFiles("/backup/file");
   pugi::xpath_node_set::const_iterator it = fileList.begin();
   for( ; it != fileList.end(); it++ ) {
   }

   ...
   ...
}


/*****   testSearch.cpp   *****/

BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::precondition(skipTest(false)))
{
   ...
   ...

   std::vector<string> keyList;
   keyList.push_back("/home/puiyee/workspaceqt/debug/FolderA/subA/subB/file_3.txt");

   FileBotUnderTest fb;
   fb.createDestinationConfig("/home/puiyee/workspaceqt/debug/FolderA", keyList);
   fb.loadRemoteFS();

   ...
}
Don't confuse that the keyList mention in the test case above is required by the createDestinationConfig(). And also the first item of keyList, which indicate the root path of remote path is also not require anymore. Since it produces output, I need to verify the output to ensure consistency. I use this code at the end of unit test.
BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::precondition(skipTest(true)))
{
    ...
    ...

    pugi::xml_document doc;
    pugi::xml_parse_result result = doc.load_file("backup.xml");
    if( result ) {
        pugi::xpath_node_set files = doc.select_nodes("/backup/file");
        BOOST_TEST(files.size() == 0);

        files = doc.select_nodes("/backup/recover/dest");
        for( pugi::xpath_node_set::const_iterator it = files.begin(); it != files.end(); it++ ) {
            pugi::xml_node file = ((pugi::xpath_node)*it).node();
            string value = file.text().get();

            BOOST_TEST(value.compare("/home/puiyee/workspaceqt/debug/FolderA/sub/file_2.txt") == 0);
        }

        files = doc.select_nodes("/backup/recover/src");
        BOOST_TEST(files.size() == 0);
    }
In this unit test, I will load the backup.xml and then verify the /backup/recover/dest does created in following format.
 <backup>
    <recover>
       <dest></dest>
    </recover>
 </backup>
And same goes to /backup/recover/src.

Friday, January 25, 2019

Introducing new member - Configuration

Time flies, almost 2 months since my last update. I was working on a new class to handle the INDEX file. This class was given a name as Configuration, and its sole responsibility is to work together with INDEX file. The INDEX file consists of XML tag containing information about a file structure being scanned.

During that 2 months, I was struggling with Boost with handling the XML file, I have completed roughly 60% of my work only after I found out it is not easy to remove an XML tag that I don't need anymore. Many try and error still unable to work it out, then I start to look for alternate solutions and I found out Pugi XML able to remove and update easily. With that, I begin to switch my code to Pugi. I was lucky that around 30% of rework need to be done.

When I first design on this class, I try to think of a lazy way to accomplish a task. I try to avoid to call initialize() when the class is first initialize. This doesn't look smart. So I do it in the constructor. It is an old method, but effective. When this class is first born, it will look for the configuration setting for the source and remote path. Every time this class is loaded into memory, it will look for the remote path. If it's empty, then initialize it, otherwise it is a source path.
Configuration::Configuration()
{
    defaultPath = filesystem::current_path().string() + GENERIC_PATH_SEPARATOR + INDEX_FILENAME;

    // create new one if the index file doesn't exists
    if( !boost::filesystem::exists(INDEX_FILENAME) ) {
        auto declareNode = doc.append_child(pugi::node_declaration);
        declareNode.append_attribute("version") = "1.0";
        declareNode.append_attribute("encoding") = "UTF-8";

        pathLookup(filesystem::current_path().string());
    }
    // load the index file if it exists
    else {
        readConfigFile();

        destPath = doc.child("backup").child("configuration").child("destination_path").child_value();
        sourcePath = doc.child("backup").child("configuration").child("source_path").child_value();
    }
}

string Configuration::pathLookup(string inputPath)
{
    // assign the new destination path if the field doesn't exists
    if( destPath.size() == 0 && sourcePath.size() == 0 ) {
        destPath = inputPath;

        pugi::xml_node destPathNode = doc.append_child("backup").append_child("configuration").append_child("destination_path");
        destPathNode.append_child(pugi::node_pcdata).set_value(inputPath.c_str());

        updateConfiguration();

        return destPath;
    }
    // assign the new source path if the field doesn't exists
    else if( sourcePath.size() == 0 && destPath.size() != 0 ) {
        sourcePath = inputPath;

        pugi::xml_node configNode = doc.child("backup").child("configuration");
        configNode.append_child("source_path").append_child(pugi::node_pcdata).set_value(inputPath.c_str());

        updateConfiguration();

        return sourcePath;
    }
    else
        return "";
}
Next is the content construction. This class must be able to construct the XML tag from a given input. For example, if I pass in the input like this, recover.source, then it must be able to construct as shown below:
<recover>
   <source></source>
</recover>
And not something like this:
<recover.source></recover.source>
Well, that's about the design, but the code behind this logic isn't straight forward. One condition is to validate whether it is allowed to duplicate, another is to check whether the XML tag exists, if it doesn't, then create it.
pugi::xml_node Configuration::allocateNode(string key, bool duplicateKey)
{
    char_separator<char> sep(".");
    tokenizer<char_separator<char> > token(key, sep);
    pugi::xml_node node;

    BOOST_FOREACH( const string& nodeName, token ) {
        qDebug() << "processing node name: " << nodeName.c_str();

        // retrieve the root node for the first time
        if( node.empty() ) {
            // create a new node if the root node was not found
            if( doc.child(nodeName.c_str()) == nullptr )
                node = doc.append_child(nodeName.c_str());
            // retrieve the root node
            else
                node = doc.child(nodeName.c_str());
        }
        else {
            // test if the child node is there
            if( node.child(nodeName.c_str()) == nullptr )
                node = node.append_child(nodeName.c_str());
            // retrieve the node
            else
                node = node.child(nodeName.c_str());
        }
    }

    return node;
}


void Configuration::writeValue(string key, bool duplicateKey, string value)
{
    bool allowInsert = true;

    string xpath = key;
    // convert key to XPath
    std::replace(xpath.begin(), xpath.end(), '.', '/');
    xpath = "/" + xpath;
    qDebug() << "XML node path: " << xpath.c_str();

    // duplicate value is not allowed
    if( !duplicateKey ) {
        // overwrite the value without validation
        pugi::xpath_node node = doc.select_node(xpath.c_str());
        if( !node.node().empty() ) {
            qDebug() << node.node().name() << " : " << node.node().text().get();

            node.node().text().set(value.c_str());
        }
        else {
            pugi::xml_node tmp = allocateNode(key, duplicateKey);
            tmp.text().set(value.c_str());
        }
    }
    else {
        // walk throught each node to check any duplicate value
        pugi::xpath_node_set files = doc.select_nodes(xpath.c_str());
        for( pugi::xpath_node_set::const_iterator it = files.begin(); it != files.end(); ++it ) {
            pugi::xpath_node file = *it;
            string val = file.node().text().get();

            qDebug() << file.node().name() << " : " << file.node().text().get();

            if( value.compare(val) == 0 )
                allowInsert = false;
        }

        // no duplicate value, proceed to insert the value
        if( allowInsert ) {
            // bail if this is equal to first node
            if( key.find_last_of(".") == -1 )
                return;

            string parentNode;
            parentNode = key.substr(0, key.find_last_of("."));

            if( parentNode.compare(key) == 0 )
                return;

            // is the XML node missing? Make a new one if it went missing
            pugi::xml_node node = allocateNode(parentNode, duplicateKey);

            string nodeName = key.substr(key.find_last_of(".") + 1, key.length());
            node = node.append_child(nodeName.c_str());
            node.append_child(pugi::node_pcdata).set_value(value.c_str());
        }
    }

    updateConfiguration();
}
Last but not least, this class is also able to remove an XML tag. The nodePath will tell which part of the XML tag will be removed, the removal condition must contain the value mention in the nodeValue.
void Configuration::removeNode(string nodePath, string nodeValue)
{
    // convert key to XPath
    std::replace(nodePath.begin(), nodePath.end(), '.', '/');
    nodePath = "/" + nodePath;
    qDebug() << "XML node path: " << nodePath.c_str();

    pugi::xml_node node;
    pugi::xpath_node_set nodes = doc.select_nodes(nodePath.c_str());
    for( pugi::xpath_node_set::const_iterator it = nodes.begin(); it != nodes.end(); ++it ) {
        pugi::xpath_node file = *it;
        string val = file.node().text().get();

        if( nodeValue.compare(val) == 0 ) {
            node = file.node();
            break;
        }
    }

    node.parent().remove_child(node);
    updateConfiguration();
}

Tuesday, November 6, 2018

Found a new usage of getLoadFile()

Initially I was thinking to sunset the getLoadFile() as it's just another dump function for debugging purpose, but now I am going to need it in the search process. The original function will accept a FileNode object and then recursively loop through each path until the last object has reached, then convert each object into the URL string.
std::list<string> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<string> mlist;

    if( fileNode == nullptr || fileNode->sibling.size() == 0 )
        return mlist;

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        FileNode *n = (&*it);

        if( n->getType() == 'd' ) {
            std::list<string> subList;
            subList = getLoadedFiles(n);

            mlist.insert(mlist.begin(), subList.begin(), subList.end());
        }
        else {
            string filePath = constructPathAddress(n);
            mlist.push_back(filePath);
        }
    }

    return mlist;
}
This was originally used in path construction process to verify the path has correctly loaded into memory by verifying the URL returned. For now, I need this to retrieve all the FileNode objects that park under a particular FileNode object rather than the URL string. Thus, I am converting the list storage to store the FileNode object instead of string:
std::list<FileNode*> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<FileNode*> mlist;

    ...

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        ...

        if( n->getType() == 'd' ) {
           ...
           ...
        }
        else {
            mlist.push_back(n);
        }
    }

    return mlist;
}
On the unit testing site, originally I have the following code to verify my desired URL was loaded into the memory. The std::find is a very useful function to lookup a value in the list.
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(false)))
{
    ...

    // verify the file name was captured
    std::list<string> files = fb.getLoadedFiles(root);

    if( std::find(files.begin(), files.end(), "/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
}
Due to the change to object type, the std::find is unusable anymore. But I found a workaround on this, to lookup a value in the object list, that is to create a new functor for this purpose. The URL is now constructed and perform matching inside functor.
struct FileNodeFinder
{
    FileNodeFinder(const std::string &s) : s_(s) {}

    bool operator() (const FileNode* obj)
    {
        FileBotUnderTest fb;

        return fb.constructPathAddress(obj).compare(s_) == 0 ? true : false ;
    }

private:
    const std::string s_;
};
Thus, the new way of doing work is like this:
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(false)))
{
    ...

    // verify the file name was captured
    std::list<FileNode*>::iterator it = find_if(files.begin(),
                                                files.end(),
                                                FileNodeFinder("/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt"));
    FileNode *n = *it;

    if( it == files.end() )
        BOOST_FAIL("nothing were found");
}

Thursday, November 1, 2018

Pretty bad design on the return of constructParentPath()

Not long ago, there was some major change on constructParentPath() in order to maintain 2 different types of request, one for local home path and another one for the remote home path. The FileListType is used to determine whether this function is generated in local home path or remote home path before it get returned. See the code snippet below:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    FileNode *parent = NULL;

    ...
    ...

    for( string_split_iterator it = make_split_iterator(path, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        ...
        ...

        if( parent == NULL )
            if( listType == local )
                localList.push_back(*node);
            else
                remoteList.push_back(*node);
        else {
            parent->sibling.push_back(*node);
            node->setParentNode(parent);
        }

        ...
        ...
    }

    ...

    if( listType == local ) {
        localHome = parent;
        return localHome;
    }
    else {
        remoteHome = parent;
        return remoteHome;
    }
}
Notice the parent is assigned back to the respective home variable, I was thinking this code might need to rework if there is another third option is coming in. Pretty bad, right?!! Thus, I just feel the return code at the end is redundant. To prevent that from happening in future, I might as well just return the parent and let the caller to decide where the value should assign.
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    FileNode *parent = NULL;

    ...
    ...

    return parent;
}
As for this change, there are 2 callers on this function. One is loadLocalFS() and another one is loadRemoteFS(). Below are the changes made to adopt this new change:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    localHome = constructParentPath(inputPath);
    constructChildPath(localHome);

    ...
}


FileNode* FileBot::loadRemoteFS(vector<string> fileList)
{
    ...

    // load up the abstract path for first entry
    remoteHome = constructParentPath(fileList[0], remote);

    ...
    ...
}

Thursday, October 4, 2018

Message Queue in Liberty Profile ​

I didn't know that Liberty Profile has come with an internal embedded messaging server? This sound interesting to me, I should to give it a try now. For this to work, it required the full profile version of Liberty Profile. When I first start, I didn’t realize the one I'm using now is a web profile version, end up I'm not able to configure the messaging features in the server. I got the Liberty full profile from here. According to the tutorial found on the web, the configuration on the server.xml is pretty straightforward. Here is the work done in PTP configuration:

The <jmsactivationspec> tag is the very crucial part of connecting the embedded messaging engine to the MDB deployed in the server. The documentation mentions here has detail explanation on how the things should work. Without this tag or miss configure the id will give a friendly error reminding you that there are some problem with the tag. Below is the sample error in my case, notice the name of the id has mentioned in the error:
[WARNING ] CNTR4015W: The message endpoint for the MessageBean message-driven bean cannot be activated because the jms2-0.0.1-SNAPSHOT/MessageBean activation specification is not available. The message endpoint will not receive messages until the activation specification becomes available.
Code snippet below is a simple MDB app for the sake of this testing purpose:


Connect to MDB without using JNDI 

Now the server is ready to accept the incoming messages. The sample below is a standalone Java program that will establish a connection to Liberty Profile, it doesn’t use the JNDI to send the message to MDB. This is the hardest part as not much information available on the web. Eventually, I have identified 3 libraries which are needed in order to build this program, there are available in the WAS installation path located at AppServer/runtimes​. Here are the 3 libraries:
  1. com.ibm.ws.ejb.thinclient_8.5.0.jar
  2. com.ibm.ws.sib.client.thin.jms_8.5.0.jar
  3. com.ibm.ws.orb_8.5.0.jar
Looking at the code above, I have no idea what the setBusName() on line 16 trying to do? It will just throw an error as seen below if I remove that line:
Caused by: com.ibm.websphere.sib.exception.SIIncorrectCallException: CWSIT0003E: No busName property was found in the connection properties.
  at com.ibm.ws.sib.trm.client.ClientAttachProperties.< init>(ClientAttachProperties.java:109)
  at com.ibm.ws.sib.trm.client.TrmSICoreConnectionFactoryImpl.createConnection(TrmSICoreConnectionFactoryImpl.java:295)
  at com.ibm.ws.sib.trm.client.TrmSICoreConnectionFactoryImpl.createConnection(TrmSICoreConnectionFactoryImpl.java:222)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createCoreConnection(JmsJcaConnectionFactoryImpl.java:711)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createCoreConnection(JmsJcaConnectionFactoryImpl.java:647)
  at com.ibm.ws.sib.api.jmsra.impl.JmsJcaConnectionFactoryImpl.createConnection(JmsJcaConnectionFactoryImpl.java:376)
  at com.ibm.ws.sib.api.jms.impl.JmsManagedConnectionFactoryImpl.createConnection(JmsManagedConnectionFactoryImpl.java:162)
  ... 3 more

Connect to MDB with JNDI

This is much easier to implement as compare to the previous client app just because it uses JNDI. It doesn’t require any additional library from WAS server. Long story short, here is the code:

Tuesday, October 2, 2018

Resolving conflict between nVidia and mesa when upgrading Fedora 25

What a busy week! I was doing a major upgrade to the Fedora system and I was almost getting myself killed in the action. During the upgrade process, I was stopped by this nasty error:
$ sudo dnf system-upgrade download --releasever=25
...
...
...
Error: Transaction check error:
file /usr/lib64/libGLX_indirect.so.0 from install of mesa-libGL-17.0.5-3.fc25.x86_64 conflicts with file from package nvidia-driver-libs-2:378.13-3.fc24.x86_64
file /usr/lib/libGLX_indirect.so.0 from install of mesa-libGL-17.0.5-3.fc25.i686 conflicts with file from package nvidia-driver-libs-2:378.13-3.fc24.i686

Error Summary 
-------------
I spend almost one week to tackle this error, from day till night researching on the root cause. To keep the story short, 2 things need to do. First step is to uninstall the driver, I follow the instructions from this link which is dedicated for Fedora. After this step, there are some configuration files was left over in the xorg.config.d directory and some other rpm packages were still not yet remove. This is the sample output for my case:
$ rpm –qa | grep nvidia
nvidia-driver-libs-378.13-3.fc24.x86_64
nvidia-driver-libs-378.13-3.fc24.i686
nvidia-settings-378.13-1.fc24.x86_64
nvidia-libXNVCtrl-378.13-1.fc24.x86_64
dkms-nvidia-378.13-2.fc24.x86_64
nvidia-driver-378.13-3.fc24.x86_64
Then the next step would be this:

$ sudo dnf remove nvidia-kmod

Never try to uninstall individual rpm package manually, as the command above is the professional way to do it. Clean the whole chunk in one shot. Only after this command, then only the system upgrade can continue. During the last weekend, I have successfully upgraded from Fedora 24 all the way up to Fedora 27 in one shot.

Sunday, September 23, 2018

Memory leaks again when clearing memory

Arhhh! Memory leaks again?!! How many times I have been working on this shit? It has been reviewed and rework for many, many times. Remember, there are once a major changes to the code, most probably I miss out to run the unit test on Windows then now all test cases were failed in clearMemory(). For now I am reworking on the logic for clearing the memory, I hope this is the last and final.

Friday, September 21, 2018

How Boost Property Tree works in object loop?

I am in the midst of working on the extraction of file listing captured in memory into a text file. The task was simple, I will just write each file path into a line and then follow by another path in the next line. But I felt this method does not look good, I was thinking about future expansion, there would be a possibility of rework. And I hate rework. And thus, I change my direction to keep them in XML, which I felt much more organize. Besides storing the file listing, I'll also store the date captured on this listing, the destination path and the source path.

For this purpose, I'm using Boost Property Tree. According to the tutorial on Boost Property Tree, the code was very easy to handle. For that, I separate the piece away from the core object and put them into a brand new object, where the object will take the sole responsibility in handling the reading and writing of XML content. And for the sake of the whole execution, the object will have only one instance in the memory. And then I give a name for this object called Configuration.

Brilliant! When I test out the code, the content was successfully transferred into the designate file as shown in the following.

Unfortunately, in another test case, I am not able to read it back from the file, there are funny characters shown in the output on line 9 in the following code. There is no problem with the file being generated. I have no clue how this could happen. I am totally blind now.

Many hours were spent to debug this shit, and I found the root cause at last. It was really shocking on my finding. I just rework at line 8 in the above code and make it like the code shown below at line 10 and 12.

From this experience, I wonder how this Boost Property Tree is actually works in object loop?

Friday, August 31, 2018

Git local repository concept is really a confusion

Now only I got to know that there are 2 "types" of the local repository in the Git world. The first one is used to compare it against the local repository, where the command for this is git diff ; the second one is used to compare it against the remote repository, where the command for this is git diff --cached. One thing that really confuses me is there are 2 stages in local repository, one is called Changes not staged or commit, whereas the other one is called Changes to be committed

Confuse huh? Every time when I have done something, and I need to know the changes happen to my local PC, I'll use the command git status. The following will shows up:
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

 modified:   backupUtilUnitTest/backupUtilUnitTest.pro
 modified:   core/BackupHeader.h
 modified:   core/FileBot.cpp
 modified:   core/FileBot.h
 modified:   core/FileHelper.cpp
 modified:   core/FileHelper.h
 modified:   core/FileNode.h
 new file:   unittest/FileBotUnderTest.cpp
 new file:   unittest/FileBotUnderTest.h
 modified:   unittest/testloadfiles.cpp
 new file:   unittest/testsearch.cpp

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

 modified:   backupUtilUnitTest/backupUtilUnitTest.pro.user
 modified:   unittest/testlab.cpp

Untracked files:
  (use "git add <file>..." to include in what will be committed)

 backupUtil.zip
 backupUtilUnitTest/backupUtilUnitTest.pro.user.RX2720

Take closer attention here, when I have changes to my work in my local computer, only git diff is useful here. If I use the command git diff --cached will show nothing. This experiment shows that I am comparing the changes with the work locate in the local PC with the local repository locate at the level named Changes not staged for commit. When I issue the command git add , the file will be moved from the level of Changes not staged for commit to a new level named Changes to be committed. Nothing will show up if the command git diff is used. At this stage, git diff --cached can only be used, and this will only compare the file at the level named Changes to be committed to the remote repository, not the one in local PC. If anything has been changed to the file, the changes will not reflect with this command. But if I use the command git diff , this will show on the changes I have made to the file in local PC.

Monday, August 27, 2018

constructChildPath() need to be clean

Currently the constructChildPath() is taking 2 arguments, the first argument is a pointer to a FileNode type and the second argument is a string.
void FileBot::constructChildPath(FileNode *currentPath, string inputPath)
{
   ...
   ...
}
The purpose of having these 2 arguments is to allow the method to identify where should I scan the path and where should I start parking those paths in the list. I have been thinking for so long to revamp this method so that it doesn't take any argument for its process. I'm just trying to make it as clean as possible.

Here is my thought on the finding. In this method, the string argument, is the crucial part of this method because it tells me where should I start scanning the path. If it doesn't contain any value, then nothing would be scanned, the return list would be empty. A screenshot of the piece is as follows:
void FileBot::constructChildPath(FileNode *currentPath, string inputPath)
{
    vector<path> pathList;

    // do not proceed for root path
    if( inputPath == "" )
        return;

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentPath);

            currentPath->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node, inputPath + string(1, boost::filesystem::path::preferred_separator) + node->getName().string());
        }

    ...
    ...
}
As mention in the code above, this is a recursive function, it will recursively dive into itself searching for more files when it sees a directory. Hence, the second argument of this method, inputPath, is constantly extending the directory underneath the current path. This method is only accessible through following method and it is highly dependent on the constructParentPath() because it needs a very important element for child path construction:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    if( valiateRootPath(inputPath.generic_string()) == false ) {
        BOOST_LOG_TRIVIAL(info) << "Failed to validate input path";
        return nullptr;
    }

    constructParentPath(inputPath);
    constructChildPath(searchPath, inputPath.string());

    return searchPath;
}
Notice there is a junk variable called searchPath. I called it junk because it serves for temporary purpose. This variable was used to store the home path, which was generated in constructParentPath() as mention above, and then this variable will pass down to the constructChildPath(). This is what I mention the code is highly depends on others. Duhhh~

I'm restudy the code to see what I could improve about these junk code. As I first start, I remove the inputPath argument because I felt it was a redundant. It can generate on the spot from the currentPath, below is the upgrade version of this change:
void FileBot::constructChildPath(FileNode *currentPath)
{
    vector<path> pathList;

    // for the first time, retrieve the root path
    // bail otherwise
    if( currentPath == nullptr ) {
        return;
    }

    // construct path address
    string inputPath = constructPathAddress(currentPath);

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentPath);

            currentPath->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node);
        }

    ...
    ...
}
Since the string argument has been removed, the caller doesn't require to pass down the inputPath anymore and also the searchPath is no longer there. The caller has been trimmed down as below:
FileNode* FileBot::loadLocalFS(boost::filesystem::path inputPath)
{
    constructChildPath(constructParentPath(inputPath));

    return localHome;
}
Since searchPath has already taken away, this will have a major impact on constructParentPath() as well, thus I have the following logic append to the end of the method:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath, FileListType listType)
{
    ...

    if( listType == local ) {
        localHome = parent;
        return localHome;
    }
    else {
        remoteHome = parent;
        return remoteHome;
    }

//    return searchPath;
}

Monday, July 16, 2018

A minor mistake from make_split_iterator

Recently, I found out there are some minor mistake lie underneath the following function:
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath)
{
    ...
    ...

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(path, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
       FileNode *node = new FileNode();
       node->setType('d');
       node->setName(copy_range<string>(*it));

       BOOST_LOG_TRIVIAL(info) << "value inserted: [" << copy_range<string>(*it) << "]";

       if( parent == NULL )
           fileList.push_back(*node);
       else {
           parent->sibling.push_back(*node);
           node->setParentNode(parent);
       }

       parent = node;
       ...
       ...
    }
}
This mistake would not cause any major crash, it just doesn't look "pretty" in terms of behavioral design. Before I go into the details, let's consider following 2 possible inputs:
  1. /home/user
  2. C:/home/user 
The output for entry 2 will generate a nice and beautiful link-list as shown in the image below:
But entry 1 would become like this, the first node in the link-list doesn't contain any name.
 
Why this could happen? I though the code will automatically handle the first token when it is empty? And I also realized this only happened to Linux due to Linux file system structure design behave differently from Windows. To prove my assumption, I run a series of test on make_split_iterator(), and the result shows the method does return an empty string. Another test on boost::split() also having this issue. The only one success is boost::tokenizer().

BOOST_AUTO_TEST_CASE(TL_split_words, *boost::unit_test::precondition(skipTest(false)) )
{
    string win = "D:/home/user";
    string lin = "/home/user";

    vector<string> words;
    string str, firstElement;

    cout << "***** version 1 *****" << endl;
    cout << "Windows platform: ";

    str = win;
    boost::split(words, str, boost::is_any_of(GENERIC_PATH_SEPARATOR), boost::token_compress_on);

    for(string w : words) {
        cout << "[" << w << "]";
    }
    cout << endl;

    firstElement = words.at(0);

    BOOST_TEST(words.size() == 4);
    BOOST_TEST(firstElement.compare("D:") == 0);

    words.clear();
    str = lin;
    boost::split(words, str, boost::is_any_of(GENERIC_PATH_SEPARATOR), boost::token_compress_on);

    cout << "Linux platform: ";
    for(string w : words) {
        cout << "[" << w << "]";
    }
    cout << endl;

    firstElement = words.at(0);

    BOOST_TEST(words.size() == 4);
    BOOST_TEST(firstElement.compare("") == 0);

    cout << "***** version 2 *****" << endl;
    cout << "Windows platform: ";

    str = win;

    typedef split_iterator< string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(str, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        cout << "[" << copy_range<string>(*it) << "]";
    }
    cout << endl;

    cout << "Linux platform: ";
    str = lin;

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(str, first_finder(GENERIC_PATH_SEPARATOR, is_equal()));
         it != string_split_iterator();
         ++it)
    {
        cout << "[" << copy_range<string>(*it) << "]";
    }
    cout << endl;

    cout << "***** version 3 *****" << endl;
    cout << "Windows platform: ";

    str = win;

    boost::char_separator<char> sep{GENERIC_PATH_SEPARATOR.c_str()};
    boost::tokenizer<boost::char_separator<char> > token{str, sep};

    for( const auto &t : token ) {
        cout << "[" << t << "]";
    }
    cout << endl;

    cout << "Linux platform: ";
    str = lin;
    token = {str, sep};

    for( const auto &t : token ) {
        cout << "[" << t << "]";
    }
    cout << endl;
}
The output of this test would be like this:
***** version 1 *****
Windows platform: [D:][home][user]
Linux platform: [][home][user]
***** version 2 *****
Windows platform: [D:][home][user]
Linux platform: [][home][user]
***** version 3 *****
Windows platform: [D:][home][user]
Linux platform: [home][user]
I admit that I have overlooked the code, it is just too detail which something I have missed. Although the test doesn't fail, but the consequent impact would cause the link-list contain a node without a name.

Friday, July 6, 2018

Rebuild gSOAP from the source

I'm making a web service in C++. Never try it before, just want to find out how hard could it be? Unlike JAVA, the web service library in C++ isn't that popular, however, I still manage to get one. It is called gSOAP, I download it from sourceForge. And the experts are putting a thumbs up on it too.

For the first time, I am doing it on the Windows platform, it was working just fine. When I move on to the Linux machine, I got the following error pop up when I'm compiling my source:
In file included from soapH.h:16:0,
                 from HuahsinServiceBindingPort.nsmap:2,
                 from main.cpp:3:
soapStub.h:24:3: error: #error "GSOAP VERSION 20828 MISMATCH IN GENERATED CODE VERSUS LIBRARY CODE: PLEASE REINSTALL PACKAGE"
 # error "GSOAP VERSION 20828 MISMATCH IN GENERATED CODE VERSUS LIBRARY CODE: PLEASE REINSTALL PACKAGE"
   ^~~~~
What's happening just now? The error seems trying to remind me that the gSoap version isn't compatible with something else. This never happened in Windows. Wouldn't it be the cause of gSOAP I downloaded is for Windows only? Could it be some other thing else? Flashing back my memory what I did wrong, I downloaded the gSOAP from sourceForge, and then at the same time, I downloaded another version from Fedora repository. Then I generate the stub objects using the following command with Fedora's version, like this:
wsdl2h -o hello.h http://localhost:8080/services/hello?wsdl -t/usr/share/gsoap/WS/typemap.dat
soapcpp2 -jCL -I/usr/share/gsoap/import hello.h
And then I compile the source code with the stdsoap2.cpp from sourceForge's version.
g++ -g -o main.exe main.cpp soapC.cpp soapHuahsinServiceBindingPortProxy.cpp 
/home/kokhoe/tool/gsoap-2.8.68/gsoap/stdsoap2.cpp 
-I/home/kokhoe/tool/gsoap-2.8.68/gsoap 
-I. -std=c++11
Is this the reason why the compilation failed? Am I correct? I wasn't sure whether my assumption makes sense. As I read on the documentation, it does mention this:
This error is caused by mixing old with new gSOAP versions of the generated code and gSOAP libraries. Make sure to include the latest stdsoap2.h and link with the latest libgsoapXX.a libraries (or use the stdsoap2.c or stdsoap2.cpp source code).
I think this has given a clear sign on my mistake. I knew that I am going to make a fresh build from source, but still trying hard to find a lazy way, without making any hassle work to rebuild from the source. I am making a deep breath... thinking... and then keep on searching... for quite some time... and I found a step by step manual guide that could help me from building the source until the installation. It is accompanied with the source I downloaded. Thus, I made a bold decision to rebuild everything, including gSOAP.

During the build process, there are many libraries were missing, among those errors I have, following error tells no clue on what was missing. The compiler just stops like that with the given command like this:
gcc -DWITH_YACC -DWITH_FLEX -DSOAPCPP2_IMPORT_PATH="\"/usr/local/share/gsoap/import\"" 
-DLINUX -g -O2 -o soapcpp2 soapcpp2-soapcpp2_yacc.o soapcpp2-soapcpp2_lex.o soapcpp2-symbol2.o soapcpp2-error2.o 
soapcpp2-init2.o soapcpp2-soapcpp2.o -ly  
/usr/bin/ld: cannot find -ly
It gave me a hard time to identify the relationship between the -DWITH_YACC, -DWITH_FLEX, and -ly. There are the Bison and Flex libraries actually. Thank god. The rebuild process continues at last and my source was built with no error.

Friday, June 29, 2018

Some flaw happened in clearing the memory

There is some amendment being done in clearMemory() for a very long time ago. I can't recall what I did last time, the code seemed likely been done in a rush and there are not test well. This is the piece I mention in this post:
void FileBot::clearMemory(FileNode *fileNode)
{
    bool lastNode = false;

    // this case will happened only when the root is supply
    if( fileList.size() == 0 || (fileList.begin())->sibling.empty() ) {
        BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << endl;
        return;
    }

    // if this is the only node in the container
    // clean up the container and then bail
    FileNode *firstNode = &*(fileList.begin());

    // test on first node is empty
    if( !firstNode->sibling.empty() && firstNode->sibling.size() == 0 ) {
        BOOST_LOG_TRIVIAL(info) << "This is the only node. Root size [" << fileList.size() << "]";
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
        return;
    }

    // if this is not the only node in the container
    // clean up the container recursively
    if( fileNode == NULL ) {
       BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << fileNode;

       fileNode = &*(fileList.begin());
       fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
       lastNode = true;
    }
    else {
        BOOST_LOG_TRIVIAL(info) << "current node has sibling: " << fileNode << " : " << fileNode->getName() << " sibling : " << fileNode->sibling.size();

        if( fileNode->sibling.size() > 0 ) {
            for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); it++ ) {
                clearMemory(&*(it));
            }
        }
        else {
            cout << fileNode << " has no more sibling." << endl;
            return;
        }

        BOOST_LOG_TRIVIAL(info) << "cleaning sibling memory for this node: " << fileNode << " node name: " << fileNode->getName();
        fileNode->sibling.erase_and_dispose(fileNode->sibling.begin(), fileNode->sibling.end(), DisposeFileNode());
    }

    if( lastNode ) {
        BOOST_LOG_TRIVIAL(info) << "Reacing the root. Root size [" << fileList.size() << "]";
        fileList.erase_and_dispose(fileList.begin(), fileList.end(), DisposeFileNode());
    }
}
In the past few days I sat down just to review my test case, I found out some of the code was not covered by the test cases. Those uncover pieces are 16-20, 24-30, and 49-52. I was shocked when I see those uncovered pieces. I wonder whether those pieces are unused code since none of the test cases cover them up.

I study the code carefully, test run for few times and finally I got a fantastic conclusion. There are 2 sections are required for this function to carry on its job; first section would be the validation, this will validate whether the container, as well as, the FileNode's sibling contain any elements in it? This is the one:
    // this case will happened only when the root is supply
    if( fileList.size() == 0 || (fileList.begin())->sibling.empty() ) {
        BOOST_LOG_TRIVIAL(info) << "current node has no sibling: " << endl;
        return;
    }
Whereas the second section would be the core logic focus on cleaning the container, as well as, the sibling of each FileNode. Here is the one:
    BOOST_LOG_TRIVIAL(info) << "current node has sibling: " << fileNode << " : " << fileNode->getName() << " sibling : " << fileNode->sibling.size();

    if( fileNode->sibling.size() > 0 ) {
        for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); it++ ) {
            clearMemory(&*(it));
        }
    }
    else {
         cout << fileNode << " has no more sibling." << endl;
         return;
    }

    BOOST_LOG_TRIVIAL(info) << "cleaning sibling memory for this node: " << fileNode << " node name: " << fileNode->getName();
    fileNode->sibling.erase_and_dispose(fileNode->sibling.begin(), fileNode->sibling.end(), DisposeFileNode());
Now the code is much more readable. Don't you agree? Oh! by the way, I already moved this function into the destructor since I'll invoke this function at the end of each unit test. Don't you feel much better?
FileBot::~FileBot()
{
    clearMemory(&*(fileList.begin()));
}

Invocation WsimportTool failed?!!

Hey there, I am making a new friend today. The meet this new friend in Eclipse Photon when I was creating a brand new web service project. Here is the story, when I first started the project, I only have the WSDL file ready in the source folder, and then I have the jaxws-maven-plugin in my pom file, like this:
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>jaxws-maven-plugin</artifactId>
    <version>2.5</version>
    <executions>
      <execution>
        <goals>
          <goal>wsimport</goal>
        </goals>
        <configuration>
          <keep>true</keep>
          <extension>true</extension>
          <sourceDestDir>${basedir}/src/main/java</sourceDestDir>
          <packageName>org.huahsin68.ws</packageName>
          <wsdlDirectory>${basedir}/src/main/resources</wsdlDirectory>
          <wsdlFiles>
            <wsdlFile>hello.wsdl</wsdlFile>
          </wsdlFiles>
        </configuration>
      </execution>
    </executions>
  </plugin>
Then this error immediately shown up right after the execution tag as shown below. What was that mean? I don’t even understand why the WsimportTool could fail?
Invocation of com.sun.tools.ws.wscompile.WsimportTool failed - check output (org.codehaus.mojo:jaxws-maven-plugin:2.5:wsimport:default:generate-sources)

org.apache.maven.plugin.MojoExecutionException: Invocation of com.sun.tools.ws.wscompile.WsimportTool failed - check output
 at org.codehaus.mojo.jaxws.AbstractJaxwsMojo.exec(AbstractJaxwsMojo.java:485)
 at org.codehaus.mojo.jaxws.WsImportMojo.processLocalWsdlFiles(WsImportMojo.java:339)
 at org.codehaus.mojo.jaxws.WsImportMojo.executeJaxws(WsImportMojo.java:292)
 at org.codehaus.mojo.jaxws.MainWsImportMojo.executeJaxws(MainWsImportMojo.java:54)
 at org.codehaus.mojo.jaxws.AbstractJaxwsMojo.execute(AbstractJaxwsMojo.java:386)
 at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
 at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
 at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
 at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:172)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:1)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1$1.call(MavenBuilder.java:115)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1.call(MavenBuilder.java:105)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
 at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod.execute(MavenBuilder.java:86)
 at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:200)
 at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:795)
 at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:216)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:259)
 at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:312)
 at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
 at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:315)
 at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:367)
 at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:388)
 at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:142)
 at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:232)
 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:60)
Googling the around some said that this is a defect from the IDE; some even said that this could be caused by the JAVA version issue; some even said JDK 8 is not supported in Eclipse Mars version. Does it true? Not really. Firstly, I issued a command mvn clean and then follow by mvn generate-sources. The error is gone! What a weird day?!!

Sunday, June 24, 2018

Verifying the "files" are being loaded correctly

I'd made a function that its sole responsibility is to verify the files have successfully loaded into memory. Following code snippet shows my hard work:
std::list<string> FileBot::getLoadedFiles(FileNode *fileNode)
{
    std::list<string> mlist;

    BOOST_LOG_TRIVIAL(info) << "current root address : " << fileNode;

    if( fileNode == nullptr || fileNode->sibling.size() == 0 )
        return mlist;

    for( FileNodeListType::iterator it = fileNode->sibling.begin(); it != fileNode->sibling.end(); ++it )
    {
        FileNode *n = (&*it);

        if( n->getType() == 'd' ) {
            std::list<string> subList;
            subList = getLoadedFiles(n);

            mlist.insert(mlist.begin(), subList.begin(), subList.end());
        }
        else {
            string filePath = constructPathAddress(n);
            mlist.push_back(filePath);

            BOOST_LOG_TRIVIAL(info) << "filePath : " << filePath;
        }
    }

    return mlist;
}
I need this function in my unit test code to ensure constructParentPath() and constructChildPath() are doing their job correctly. Take the following test case for example:
BOOST_AUTO_TEST_CASE(TL_5, *boost::unit_test::precondition(skipTest(true)))
{
    BOOST_TEST_MESSAGE("TC8 : one file is captured without sub folder");

    string testPathA = current_path().string() + string(1, path::preferred_separator) + "FolderA";
    createTestFile(testPathA, 1);

#if defined(WIN32)
    BOOST_TEST(testPathA == "D:\\workspaceqt\\backupUtil\\backupUtilUnitTest\\FolderA");
#else
    BOOST_TEST(testPathA == "/home/kokhoe/workspaceqt/debug/FolderA");
#endif

    FileBotUnderTest fb;
    FileNode *root = nullptr;
    if( (root = fb.constructParentPath(testPathA)) == nullptr ) {
        BOOST_TEST_MESSAGE("Initialization failed.");
        return;
    }

    string path = fb.getParentPathAddress();

#if defined(WIN32)
    BOOST_TEST(path == "D:\\workspaceqt\\backupUtil\\backupUtilUnitTest\\FolderA\\");
#else
    BOOST_TEST(path == "/home/kokhoe/workspaceqt/debug/FolderA/");
#endif

    fb.constructChildPath(root);

    // verify the file name was captured
    std::list<string> files = fb.getLoadedFiles(root);

    BOOST_TEST(files.size() == 1);

#if defined(WIN32)
    if( std::find(files.begin(), files.end(), "D:/workspaceqt/backupUtil/backupUtilUnitTest/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
#else
    if( std::find(files.begin(), files.end(), "/home/kokhoe/workspaceqt/debug/FolderA/file_1.txt") == files.end() )
        BOOST_TEST(false);
#endif

    destroyTestFile(testPathA);
}
This function is gets invoked right after the "files" are loaded into memory. There are not the physical file, it's just a shadow. To effectively process the files stored in the sub folder, it will invoke itself to dive into the sub folder bringing together a reference of the current path, until it has iterated through the end of the path. That is the reason why this function has a parameter of FileNode type. But there was a question come into my mind, why do I still need to pass the root (a reference value came from constructParentPath() in the unit test mention above) into this function since the FileBot class already got a reference to this value?
class FileBot
{
   ...
   ...

private:
    // *root should never be NULL
    FileNode *root = nullptr;       // store the a reference to root path

}
I know this was stupid. The initial objective is to create a function that could verify the "files" that has been loaded into memory are done correctly. I think this function is only used for the unit test purpose. I'll just leave it as it is.