Friday, June 1, 2018

Maven unable to find valid certification path

Here is the most useful command that could help clean up the mess in Maven.
mvn –U clean compile
Unfortunately, that command is giving me a nasty error complaining that Maven was not able to find a valid certificate. WHAT THE HELL?
[ERROR] Plugin org.apache.maven.plugins:maven-compiler-plugin:3.7.0 or one of its dependencies could not be resolved: 
Failed to read artifact descriptor for org.apache.maven.plugins:maven-compiler-plugin:jar:3.7.0: Could not transfer artifact org.apache.maven.plugins:maven-plugins:pom:30 
from/to central (https://repo.maven.apache.org/maven2): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target -> [Help 1]
Will this error coming from the network firewall failure? If I read the details closely, this gives me a sense that my PC have not trusted the certificate for the site. In order to proceed the compilation, I must first trust the certificate, but how? I have no idea on dealing with Maven’s security issue during my past working experience. Spending hours on this issue and found this workaround from the Internet. First, follow the following steps to download the certificate and then import the certificate into a self-made keystore:
  1. Use a browser to go to https://repo.maven.apache.org/
  2. Click on lock icon and choose "View Certificate" 
  3. Go to the "Details"
  4. tab and choose "Save to File" Choose type "Base 64 X.509 (.CER)" and save it as mavenCert.cer 
  5. keytool -import -file mavenCert.cer -keystore mavenkeystore 
Once done, compile using following command:
mvn –U clean compile -Djavax.net.ssl.trustStore=mavenkeystore
From now on, whenever I need to download a new artifact from the maven repository or to do a clean build, I am required to use the command mention above instead of mvn –U clean compile.

Reference

Wednesday, May 16, 2018

Configure MinGW in Visual Studio Code

Just make some time for me to try out the tool - Visual Studio Code (VSC). It is a new toy from Microsoft. This writing was mainly for Windows platform targeted on MinGW compiler because I found out the setup in Windows is far more "complicated" than Linux. I think this is very rare for people living in the Windows world, but using MinGW as the primary compiler. Basically, there 3 configurations needed for my use case: c_cpp_properties.json, tasks.json, and launch.json.

c_cpp_properties.json

Linux was working fine without this file, but I am not sure why this file is needed for Windows. This file basically tells the tool how those declarations/definitions are resolved in the source file. Since I’m working on Windows, I will only focus on Win32 section. In my case, I need to tell the editor that I got extra MinGW header files located in other path. These paths are added into includePath. compilerPath tell the VSC where the compiler is located, unless the path has been added to the environment variable, then this field is optional. The default intelliSenseMode value was set to msvc-x64. It has to be changed to clang-x64 because this will throw a bunch of error on the std namespace declaration. The code snippet below shows the affected section:
{
    "configurations": [
        {
            "name": "Win32",
            "compilerPath": "C:/MinGW/bin/g++.exe",
            "includePath": [
                "C:/MinGW/include",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++/tr1",
                "C:/MinGW/lib/gcc/mingw32/6.3.0/include/c++/mingw32",
                "C:/tool/gsoap-2.8.55/gsoap",
                "${workspaceFolder}"
            ],
            "defines": [
                "_DEBUG",
                "UNICODE",
                "_UNICODE"
            ],
            "intelliSenseMode": "clang-x64",
            "browse": {
                "path": [
                    "${workspaceFolder}"
                ],
                "limitSymbolsToIncludedHeaders": true,
                "databaseFilename": ""
            },
            "cStandard": "c11",
            "cppStandard": "c++17"
        }
    ],
   … 
   …
}

tasks.json 

This file is to tell how the source is going to build. Before the file is created, I’m using the following command to build the source.
g++ -o main.exe main.cpp soapC.cpp soapServiceProxy.cpp c:/tool/gsoap-2.8.55/gsoap/stdsoap2.cpp 
-I. -Ic:/tool/gsoap-2.8.55/gsoap 
-Lc:/MinGW/lib -Lc:/MinGW/lib/gcc/mingw32/6.3.0 
-lws2_32 
-std=c++11
In the configuration file, the value of args is reflecting the command. But sometimes when the error occurred, I’ll go back to the command line to do a few rounds of testing to ensure it is working before come to the final configuration. Sometimes the build may not reflect on the latest changes, just close and reopen the VSC will resolve the issue.
"tasks": [
        {
            "label": "build testlab",
            "type": "shell",
            "command": "g++",
            "args": [
                "-g", 
                "-o", "main.exe",
                "main.cpp soapC.cpp soapServiceProxy.cpp c:/tool/gsoap-2.8.55/gsoap/stdsoap2.cpp",
                "-Ic:/tool/gsoap-2.8.55/gsoap",
                "-I.",
                "-Lc:/MinGW/lib",
                "-Lc:/MinGW/lib/gcc/mingw32/6.3.0",
                "-lws2_32",
                "-std=c++11"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]

launch.json 

This file is to tell whether the program will start in debug mode or not. Basically, this file is ready to use when it was first generated, I just update the program field point to my program name, which is main.exe.
"configurations": [
    {
        "name": "(gdb) Launch",
        "type": "cppdbg",
        "request": "launch",
        "program": "${workspaceFolder}/main.exe",
        "args": [],
        "stopAtEntry": true,
        "cwd": "${workspaceFolder}",
        "environment": [],
        "externalConsole": true,
        "MIMode": "gdb",
        "miDebuggerPath": "C:/MinGW/bin/gdb.exe",
        "setupCommands": [
            {
                "description": "Enable pretty-printing for gdb",
                "text": "-enable-pretty-printing",
                "ignoreFailures": true
            }
        ]
    }
]

Saturday, May 5, 2018

Unit test code strategy on private method

During the initial development, the code was driven by the TDD concept. Thus, I have the following code published to the world temporarily. The 2 methods are the primary methods of this program; neither one of them missing will not work.
public:
    FileNode* constructParentPath(boost::filesystem::path inputPath);
    void constructChildPath(FileNode *currentRootNode);

    ...
    ...
This design was to make sure that I have enough test coverage for each method. After unit testing have been done, I have a plan to change the visibility of that 2 primary methods to private, and replace them with a more meaningful method. The new design would look like below:
public:
    FileNode* constructAbstractPath(boost::filesystem::path inputPath);


protected:
    FileNode* constructParentPath(boost::filesystem::path inputPath);
    void constructChildPath(FileNode *currentRootNode);


FileNode* FileBot::constructAbstractPath(boost::filesystem::path inputPath)
{
    constructParentPath(inputPath);
    constructChildPath(root);

    return root;
}
Ahhaa~ Feeling so nice with this method now. But the flip site of this change will spoil my existing unit test code. Thus, I think of a new solution that wouldn't make such a huge change while adapting the new design. I made a new class that mimics the existing method call as shown in below:
class FileBotUnderTest : public FileBot
{
public:
    FileBotUnderTest();

    FileNode* constructParentPath(boost::filesystem::path inputPath) {
        return FileBot::constructParentPath(inputPath);
    }

    void constructChildPath(FileNode *currentRootNode) {
        return FileBot::constructChildPath(currentRootNode);
    }
};
Changes in unit test code are very minimal, instead of interacting with FileBot, I'll just interact with FileBotUnderTest. Below are the work done for these changes:
BOOST_AUTO_TEST_CASE(TL_6, *boost::unit_test::precondition(skipTest(true)))
{
    BOOST_TEST_MESSAGE("TC6 : load 3 level parent path");
    string path = "";
    FileBotUnderTest fb;

#if defined(WIN32)
    BOOST_TEST_MESSAGE("Test path: D:\\workspaceqt\\ui1");
    path = "D:/workspaceqt/ui1";
#else
    BOOST_TEST_MESSAGE("Test path: /home/kokhoe/workspaceqt");
    path = "/home/kokhoe";
#endif

    if( fb.constructParentPath(path) != nullptr )
       path = fb.getParentPathAddress();

#if defined(WIN32)
    BOOST_TEST(path == "D:\\workspaceqt\\ui1\\");
    fb.clearMemory();
#else
    BOOST_TEST(path == "/home/kokhoe/");
#endif
}

Tuesday, April 17, 2018

Boost Intrusive Set lookup function doesn't accept search key as object?

Would it be easier if I change the intrusive list to intrusive set? Okay, my initial goal is to capture the files into the memory, which is an intrusive list for my case, and then search through the list for each input entry. The input entry might go up 1000 items in total. Now the problem here is I had never decide what algorithm to use for the searching part. Then I come across the intrusive set does provide the API for me to work with, code sample below is my experiment on this:
class FileNode3;
typedef boost::intrusive::set<FileNode3, compare<std::greater<FileNode3> >, constant_time_size<false> > BaseSet;


class FileNode3 : public set_base_hook<link_mode<safe_link> > {
public:
    boost::filesystem::path root;
    boost::filesystem::path name;
    char type; // f is file, d is directory

    BaseSet sibling;

    FileNode3() {}
    FileNode3(boost::filesystem::path name_) : name(name_) {}
    FileNode3(boost::filesystem::path name_, const char type_) : name(name_), type(type_) {}

public:
    void setName(boost::filesystem::path name) { this->name = name; }
    void setRoot(boost::filesystem::path root) { this->root = root; }
    void setType(char type) { this->type= type; }

    friend bool operator< (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) < 0;  }

    friend bool operator> (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) > 0;  }

    friend bool operator== (const FileNode3 &a, const FileNode3 &b)
        {  return a.name.compare(b.name) == 0;  }
};

BOOST_AUTO_TEST_CASE(TL_IntrusiveSet, *boost::unit_test::precondition(skipTest(false)))
{
    struct FileNodeEqual
    {
        bool operator()(const char *search, const FileNode3 &node) const
        {
            cout << "StrEqual arg1: " << search << " " << node.name.c_str() << endl;

            return strcmp(search, node.name.c_str()) == 0 ? false : true;
        }

        bool operator()(const FileNode3 &node, const char *search) const
        {
            cout << "StrEqual arg2: " << search << " " << node.name.c_str() << endl;

            return strcmp(node.name.c_str(), search) == 0 ? false : true;
        }
    };

    BaseSet s1;

    FileNode3 fn3;
    fn3.name = "fileB";
    fn3.type = 'f';

    FileNode3 fn1;
    fn1.name = "path_A";
    fn1.type = 'd';
    fn1.sibling.push_back(fn3);

    FileNode3 fn2;
    fn2.type = 'f';
    fn2.name = "fileA";

    s1.insert(fn1);
    s1.insert(fn2);

    FileNode3 f;
    f.name = "fileB";
    f.type = 'd';

    boost::intrusive::set<FileNode3>::iterator it = s1.find(f.name.c_str(), FileNodeEqual());

    FileNode3 *n = (&*it);

    if( it == s1.end() ) {
        cout << "record not found" << endl;
    }
    else {
        FileNode3 *n = (&*it);
        cout << "record found: " << n->name << endl;
    }

    if( fn1.sibling.size() > 0 ) {
        cout << "fn1 sibling has more child: " << fn1.sibling.size() << endl;

        boost::intrusive::set<FileNode3>::iterator it = fn1.sibling.find(fn3);

        if( it == s1.end() ) {
            cout << "record not found" << endl;
        }
        else {
            FileNode3 *n = (&*it);
            cout << "record found: " << n->name << endl;
        }
    }

    s1.remove_node(fn1);
    s1.remove_node(fn2);
    s1.remove_node(fn3);

    cout << "Test 1 end!" << endl;
}
But that is one problem with this code, I can only compare the name property of this object. How can I compare the rest of the properties of the object? Why don't I just pass down the whole object and then do the comparison? Then I modify the FileNodeEqual struct as follows, the search key has become an FileNode3 instead of char*:
struct FileNodeEqual
    {
        bool operator()(const FileNode3 &search, const FileNode3 &node) const
        {
            cout << "StrEqual arg1: " << search.name.c_str() << " " << node.name.c_str() << endl;

            return true;
        }

        bool operator()(const FileNode3 &node, const FileNode3 &search) const
        {
            cout << "StrEqual arg2: " << search.name.c_str() << " " << node.name.c_str() << endl;

            return true;
        }
    };
Then the compiler will complain following error to me:
error: 'bool testlab::TL_IntrusiveSet::test_method()::FileNodeEqual2::operator()(const testlab::FileNode3&, const testlab::FileNode3&) const' cannot be overloaded
         bool operator()(const FileNode3 &node, const FileNode3 &search) const
              ^~~~~~~~
error: with 'bool testlab::TL_IntrusiveSet::test_method()::FileNodeEqual2::operator()(const testlab::FileNode3&, const testlab::FileNode3&) const'
         bool operator()(const FileNode3 &search, const FileNode3 &node) const
              ^~~~~~~~
I have tested with other primitive data types are working fine. Not sure whether this is due to the performance penalty when calling those functions, thus it was design not allow to take any expensive object?

Wednesday, February 28, 2018

redesigning the constructChildPath();

inpuPath a former member of class FileBot. I decided to removed this member variable in the recent design for constructChildPath(). There are 5 test cases for this design specification - capturing files, 2 of the test case were failed, which is test case 4 and 5.
  1. one file is captured without sub folder.
  2. two files are captured without sub folder.
  3. one files are captured when there is a sub folder.
  4. one files are captured in directory under test and one file in sub folder.
  5. zero file are captured in directory under test and one file in sub folder.
The root cause for this failure was due to the misused of inputPath as reference point in constructChildPath().
void FileBot::constructChildPath()
{
    vector<path> pathList;
 
    // where am I now?
    BOOST_LOG_TRIVIAL(info) << "current path: " << inputPath << inputPath.filename() << endl;
 
    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    ...
    ...
}
For this purpose, I decide to remove this member variable to make constructChildPath() more robust in processing file directory. To do this, the first make over is to build a method to construct the path to feed directory_iterator().
string FileBot::constructPathAddress(FileNode *node)
{
    if( root == nullptr )
        return "";

    string filePath = "";

    if( node->getParentNode() != nullptr )
        filePath = constructPathAddress(node->getParentNode());

    return filePath + node->getName().string() + string(1, boost::filesystem::path::preferred_separator);
}
Unlike the previous version, the constructChildPath() was transform into a recursive function. Take note on the inputPath is getting the outcome from constructPathAddress() and then feed into directory_iterator().
void FileBot::constructChildPath(FileNode *currentRootNode)
{
    vector<path> pathList;

    // construct path address
    string inputPath = constructPathAddress(currentRootNode);

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        if( is_directory(file) ) {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('d');
            node->setParentNode(currentRootNode);

            currentRootNode->sibling.push_back(*node);

            BOOST_LOG_TRIVIAL(info) << "subFolderName : " << node->getName().string() << " : " << node;

            constructChildPath(node);
        }
        else {
            FileNode *node = new FileNode();
            node->setName(file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length()));
            node->setType('f');
            node->setParentNode(currentRootNode);

            currentRootNode->sibling.push_back(*node);
        }
    }
}
Last is to reassign the task of constructor to validatePath(). I just felt that this task doesn't suitable to put into the constructor since it only use by constructParentPath().
path FileBot::validatePath(boost::filesystem::path filePath)
{
    // remove trailing file separator if found any
    char lastChar = filePath.generic_string().at(filePath.generic_string().length() - 1);
    if( lastChar == '/' )
        filePath = filePath.generic_string().substr(0, filePath.generic_string().length() - 1);

    return filePath;
}
Then in constructParentPath() just call this method to update the input value again.
FileNode* FileBot::constructParentPath(boost::filesystem::path inputPath)
{
    inputPath = validatePath(inputPath);

    // bail out if the path doesn't exists
    if( !exists(inputPath.generic_string()) )
        return nullptr;

    ...
    ...
}

Sunday, February 25, 2018

Grouping test cases with test suite

Sometimes I just want to skip those unit test cases that have already been pass and focus those test cases on my current development. And then re-run again the whole unit test case again when I have finished the task to ensure no flaw happened to the other unit test cases. Sometimes it is so annoying to run all unit test cases that are not relevant to the task I'm currently working on. I found out one trick that could allow me to control which unit test code should execute and which should stop.

The code snippet below is the ordinary declaration for my unit test code, this declaration form will be executed whenever the code were run.
BOOST_AUTO_TEST_CASE(TL_1)
{
   ...
}
Then later I added some spice to tell the framework whether the unit test case should be executed or not. Following code snippet is the trick I have done for this purpose.
BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::enable_if<false>())
{
   ...
}
In my case, this would be a great help. I don't need to wait for all unit test cases are finished in order to see the result. Some more there is only one or two test cases I can really focus on for the same tasks at the same time. Now, as my unit test code are expanding, I need more flexibility and the ability to group relevant test cases together in one source file, and then I can decide which test code to run. Here is the code snippet for the solution.
BOOST_AUTO_TEST_SUITE(testlab, *boost::unit_test::label("testlab"))

BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::enable_if<false>())
{
   ...
   ...
}

BOOST_AUTO_TEST_SUITE_END()
With this code, the test case are grouped under the test suite named testlab. When the code is run, all test cases under this suite will be execute. But there is one flaw in this code, the enable_if() is not working. The test will continue to execute even though it is false. Not only that, neither disabled() and enabled() are working. Luckily precondition() comes to a rescue. The code snippet below tells the framework to continue to execute the test case.
struct skipTest
{
    bool skip;
    skipTest(bool s) : skip(s){}

    boost::test_tools::assertion_result operator()(boost::unit_test::test_unit_id)
    {
        if( skip == false )
            return true;
        else
            return false;
    }
};

BOOST_AUTO_TEST_CASE(TL_1, *boost::unit_test::precondition(skipTest(false)))
{
   ...
   ...
}
As of now, the test suite will help me to control which group of test case to execute and precondition() will allow me to choose which test case should skip whenever I want during the development.

Friday, February 16, 2018

Improving the piece in constructParentPath() and constructChildPath()

In the existing FileBot class design, I have an inputPath class member variable.
class FileBot
{
public:
   FileBot(boost::filesystem::path filePath);

   int constructParentPath();
   void constructChildPath();

   ...
   ...

private:
   boost::filesystem::path inputPath;
};
The purpose of this variable is to keep track of the starting point where the files should begin to search with. It is so important that nothing is missed during the initialization, otherwise nothing will get from the search. Thus I'm doing the initialization through the constructor.
FileBot::FileBot(boost::filesystem::path filePath)
{
    // remove trailing file separator if found any
    char lastChar = filePath.generic_string().at(filePath.generic_string().length() - 1);
    if( lastChar == '/' )
        inputPath = filePath.generic_string().substr(0, filePath.generic_string().length() - 1);
    else
        inputPath = filePath;
}
In the client code, the programmer must instantiate the class through the following way. Indirectly the variable will get initialized as well.
FileBot fb("/home");

// or

FileBot *fb = new FileBot("/home");
And then this variable was also incorporated in other methods such as loading path into memory. There are 2 scenarios on loading path into memory; the first scenario is to load the parent path into memory. Say for example when the given input value is /home/user, then only the particular home and user were constructed into memory. Any other files located under the /home directory will be ignored. This is the piece of this construct:
int FileBot::constructParentPath()
{
    // bail out if the path doesn't exists
    if( !exists(inputPath.generic_string()) )
        return 0;

    // construct the parent path first
    string path = inputPath.generic_string();
    FileNode *parent = NULL;

    typedef split_iterator<string::iterator> string_split_iterator;
    for( string_split_iterator it = make_split_iterator(path, first_finder("/", is_equal()));
         it != string_split_iterator();
         ++it)
    {
        FileNode *node = new FileNode();
        node->setType('d');
        node->setName(copy_range<string>(*it));

        cout << "value inserted: [" << copy_range<string>(*it) << "]" << endl;

        if( parent == NULL ) {
            fileList.push_back(*node);
        }
        else
            parent->sibling.push_back(*node);

        parent = node;

        cout << "parent: " << parent;
        cout << " parent value: " << parent->getName();
        cout << " sibling size: " << parent->sibling.size() << endl;
    }

    // keep the current path in memory
    root = parent;

    cout << "***** debug : root node " << root << " : " << root->getName() << endl;

    return 1;
}

The second scenario would be loading the child path into memory. This time it will load all the files under the directory and expand to load the files in any other subdirectories (if any). Continuing from the previous example, /home/user, this section will load the files under the user directory, including the underlying sub directory. The construct for this is as follows:
void FileBot::constructChildPath()
{
    vector<path> pathList;

    // where am I now?
    BOOST_LOG_TRIVIAL(info) << "current path: " << inputPath << inputPath.filename() << endl;

    // capture the paths/files list under the source
    copy(filesystem::directory_iterator(inputPath), filesystem::directory_iterator(), back_inserter(pathList));

    std::sort(pathList.begin(), pathList.end());

    // scan through path and those files sit in that path
    // scan for files in sub directory if there is any sub directory in that path
    for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
        string file = FileHelper::convertPathToString((*it));

        // extract the file name from the path
        file = file.substr(file.find_last_of(boost::filesystem::path::preferred_separator) + 1, file.length());

        if( is_directory(file) ) {
            cout << file << " is a directory." << endl;

            FileNode *node = new FileNode();
            node->setName(inputPath.filename());
            node->setType('d');

            root->sibling.push_back(*node);
        }
        else {
            FileNode *node = new FileNode();
            node->setName(file);
            node->setType('f');

            root->sibling.push_back(*node);
        }
    }
}
To ensure my code is always implemented correctly, the following test case must return a successful result:
  • load 1 level parent path.
  • load 2 level parent path.
  • load 3 level parent path.
  • file separator appear at the end of a path.
  • test on one file is captured without sub folder.
  • test on two files are captured without sub folder.
  • test on one files are captured when there is a sub folder.