Saturday, January 21, 2017

replace multiple character in a string

OK, I have done with unit test work. And I'm ready to plug into the real world. When I run my backup program, my index file wasn't shown. What is wrong?

A silly mistake was spotted in my code. Suppose I want to replace a string containing a slash symbol with underscore symbol. Following code did the job very well:
    int seperatorPos = filename.find("/");
    if(seperatorPos > 0) {
        filename.replace(seperatorPos, 1, "_");
    }
I was making a wrong assumption that as long as I found a slash symbol, seperatorPos is greater than 1. The slash symbol will then replace with an underscore symbol. But if I have more than 1 slash, the code will fail. To correct my mistake, this is the solution:
    int seperatorPos = filename.find("/");
    while( seperatorPos != string::npos ) {
        filename.replace(seperatorPos, 1, "_");
        seperatorPos = filename.find("/");
    }
The while loop will keep searching for slash symbol till the end of the string and then replace them with underscore symbol.

Wednesday, December 28, 2016

Unit test my code

Have I ever forgotten to unit test my work? I'm a big fan of Test Driven Development, and I'm so addicted to adopting this methodology in each of my development. Implementing unit test in C++ is my first time. Spending some time doing a search on C++ unit test framework, surprisingly, there are many of them and Boost is one of them. Since I've already implemented Boost in this project, I'll just go with it.

For each use case I have developed, a unit test will run through it to ensure I'm doing the right work. This is my practice. In order to run on a clean environment, in the beginning of every test case, I'll recreate the test files for each test case. To begin, I have this:
class PrepareFile
{
public:
    PrepareFile() {
        create_directory("path_A");
        create_directory("path_B");

        BOOST_TEST_MESSAGE("Creating test folder");
    }
    ~PrepareFile() {
        remove_all("path_A");
        remove_all("path_B");

        BOOST_TEST_MESSAGE("Removing test folder");
    }
};

/*  Test case: Cap_T1
 *  Test scenario: path_A has one file
 *  Test result: one file is captured in index file
 */
BOOST_FIXTURE_TEST_CASE(Cap_T1, PrepareFile)
{
    ...
}

/*  Test case: Cap_T2
 *  Test scenario: path_A has one file, path_B has one file.
 *  Test result: files in both paths has captured in index file.
 */
BOOST_FIXTURE_TEST_CASE(Cap_T2, PrepareFile)
{
    ...
}
I tied up each test case to a class for the purpose of initialization work before a test run and clean up the test result after the test is completed. I mention that I want a clean test, thus the constructor will be invoked to create a clean the path when a test start to run and the destructor will remove the path when a test has completed. So far I have 11 test cases that cover all the possible use case happen to capture the files.

Saturday, December 24, 2016

Generate WSDL file with wsgen

Today I'm running a Web Service experiment again. But I just feel that I'm making a tutorial right now. No way, this is just a finding on my silly mistake though. Assuming I have the following code setup:
package org.huahsin.main.bo;

...

@WebService(targetNamespace = "http://main.huahsin.org/")
public interface IPersonService {

 @WebMethod
 public String getPerson(String name);
}


package org.huahsin.main;

...

@WebService(serviceName = "PersonService", 
   endpointInterface = "org.huahsin.main.bo.IPersonService",
   portName = "PersonServicePort")
public class PersonService implements IPersonService {

 @Override
 public String getPerson(String name) {
  return "Hello World " + name;
 }

}
Take good care of the endpointInterface, I mess up this name with the implementation class name, that suppose belong to an Interface. Next is to fire up the WSDL through HTTP. Previously I was using this trick to make a WSDL file:
public class PersonClient {

 public static void main(String[] args) {
  Endpoint.publish("http://localhost:8080/ws", new PersonService());
 }
}
And then hit the URL http://localhost:8080/ws?wsdl in the browser, copy the content in the WSDL file. Actually wsgen has already covered the WSDL file generation with -wsdl as shown following command:

wsgen -keep -wsdl -cp ./WebContent/WEB-INF/classes org.huahsin.main.PersonService

There will be some minor modification on the REPLACE_WITH_ACTUAL_URL after WSDL file is generated:
  <service name="PersonService">
    <port binding="tns:PersonServicePortBinding" name="PersonServicePort">
      <soap:address location="REPLACE_WITH_ACTUAL_URL"/>
    </port>
  </service>
Just replace any URL as you like, make sure the IP and port are accessible, it is very useful to determine what are the service available currently deployed.

* Do take note that when I deploy this code into WebSphere Liberty Profile (v8.5.5.8 as of this writing) with CXF runtime bundle with my code, it causes conflict error. According to the documentation, I should not enable jaxws-2.2 feature.
If you application provides its own copy of CXF JAR files as the application libraries, for example, in the WEB-INF/lib directory of a web application, you cannot enable the jaxws-2.2 feature in the server.xml file.

Thursday, December 22, 2016

New use case identified.

Things start to become more and more complicate when following diagram pop up. I think the key element during the capturing process is the index file.
Figure 1: Files are captured into index file
Here are the requirements during the capturing process:
  • First scan will be done on the destination location.
  • Next scan will be done on the origin location.
  • If the file shown in origin location, but missing in a destination location, put a plus sign to indicate this file need to be added in. During the sync process, the file will be copied to the destination.
  • If the file was missing in origin location, but shown in the destination location, put a minus sign to indicate the file need to be removed. During the sync process, the file will be removed from the destination.
The files are captured in index file, but not a destination. Why? This is due to the reason the destination might not available at that time. This design surf 2 purpose, while synchronize files:
  1. to let the program know there are some files has been removed and thus the copy in destination must remove as well. 
  2. prevent the destination copy removed accidentally even when the files in original source weren't there. (Side note: the same path might contain different files in different PC?)
Since the index file holding the key information about what need to be synced and what need not to sync. There would be a huge change in my code. To cope for this change, I'm going to making a big renovation on captureFiles().

Working on this never easy and I feel a little bit lazy. It required me to work around the logic. First thing first, I'll load up the content of index files into memory - fileContent.
void FilePath::captureExistingFile()
{
    std::ifstream indexFile(indexFilePath, ios_base::in);
    string line = "";
    while( std::getline(indexFile, line) ) {
        fileContent.insert(line);
    }
    indexFile.close();
}
Then I invoke this method from constructor.
class FilePath
{
...
...

private:
    set<string> fileContent;
    path rootPath;
    std::ofstream indexFile;
};

FilePath::FilePath(path thePath)
    : rootPath(thePath), searchDone(false)
{
    ...

    // capture the index file content into memory
    indexFilePath = rootPath + string(1, path::preferred_separator) + ((seperatorPos > 0) ? filename : converterX.to_bytes(thePath.wstring())) + ".i";

    if( exists(indexFilePath) ) {
        captureExistingFile();
        bFileExist = true;
    }

    indexFile.open(indexFilePath, ios_base::in | ios_base::out | ios_base::app);
}
And then I'm using bMinus flag and bPlus flag to work around the logic:
void FilePath::captureFiles(path searchPath)
{
    bool bPlus = false, bMinus = false;
    vector<path> pathList;

    if (rootPath.size() == 0) {
        rootPath = searchPath;
    }

    if( exists(searchPath) ) {
        // capture the paths/files list under the source

        ...

        // scan through path and those files sit in that path
        // scan for files in sub directory if there is any sub directory in that path
        for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {

            ...

            file = file.substr(thePath.length() + 1, file.length());

            // remove the files when index file content is larger than actual path
            if( fileContent.size() > pathList.size() - 1 ) {
                set<string>::iterator result = std::find(std::begin(fileContent), std::end(fileContent), file);
                if( result != std::end(fileContent) ) {
                    fileContent.erase(file);
                    bMinus = true;
                }
            }
            // new file has arrive when index file does exists
            // this block wouldn't execute if the path has never been scan before
            else if( fileContent.size() != 0 ){
                set<string>::iterator result = std::find(std::begin(fileContent), std::end(fileContent), file);
                if( result == std::end(fileContent) ) {
                    bPlus = true;
                    fileContent.insert(file);
                }
                else {
                    continue;
                }
            }

            ...

            // it is a file
            else {
                fileSet.insert(file);

                string filename(file.begin(), file.end());

                if( bPlus ) {
                    bPlus = false;
                    indexFile << "+" << filename << endl;
                }
                else if( !bMinus ) {
                    bMinus = false;
                    indexFile << filename << endl;
                }
            }
        }
    }

    indexFile.close();

    // there are some files has been remove
    // replace the content of index file
    if( bFileExist && bMinus ) {
        indexFile.open(indexFilePath, ios_base::out | ios_base::trunc);
        for( set<string>::iterator it=fileSet.begin(); it != fileSet.end(); it++ ) {
            indexFile << *it << endl;
        }

        for( set<string>::iterator it=fileContent.begin(); it != fileContent.end(); it++ ) {
            indexFile << "-" << *it << endl;
        }
    }

}
Tada! The magic works.

Sunday, December 18, 2016

The searching mechanism is ready

Let's make the nonsense short. I start off with the experiment on file searching mechanism with boost::filesystem. The library just works so perfectly in my test lab. An object named FilePath was spawn that responsible for file searching in the path. This object represents one path at a time.
class FilePath
{
public:
    FilePath();
    FilePath(path thePath);

    void captureFiles(path searchPath);
    set<wstring> getFileSet() { return fileSet; }


private:
    bool checkPathExists(path workingPath);


private:

    set<wstring> fileSet;
    path rootPath;
    bool searchDone = false;
    void clearFileSet() { fileSet.clear(); }
};
In this object, captureFiles() is the most important method and the primary function of this object. It searches for files, and capture them into memory.
void FilePath::captureFiles(path searchPath)
{
    vector<path> pathList;

    if( rootPath.wstring().length() == 0 )
        rootPath = searchPath;

    if( exists(searchPath) ) {
        // capture the paths/files list under the source
        copy(directory_iterator(searchPath), directory_iterator(), back_inserter(pathList));

        // scan through each path/file to categorize them either path or file
        for(vector<path>::const_iterator it(pathList.begin()); it != pathList.end(); ++it) {
            wstring file = (*it).wstring();
            wstring thePath = rootPath.wstring();
            file = file.substr(thePath.length() + 1, file.length());

            // it is a directory
            if( is_directory(status(*it)) ) {
                captureFiles(*it);
            }
            // it is a file
            else {
                fileSet.insert(file);
            }
        }
    }

    searchDone = true;
}
Wait! did I forgot to initialize where the path supposes to search? I'd make it in the constructor, set the rootPath when the object is initialized. This would represent the particular path for the object:
FilePath::FilePath(path thePath)
    :rootPath(thePath), searchDone(false)
{
}
Well, the basic mechanism has been set up. It's time for more serious things.

Sunday, December 11, 2016

Going deep to the backup plan

To start off the backup plan, I will be using Qt and Boost. Before this, I'm a fan of MFC, everything I did is for Windows (only). But we are now not living alone, thus Qt would be my best support on UI work for cross platform environment, this is greatly reduced my effort in maintaining separate code for different platform. Boost contains many flavor type of algorithm that could ease up my development affort. And of course C++ would definitely become the primary language for this project.

I start off this project with unit test. I am a true fan of Test Driven Development methodology. I've been using this methodology for years, and it proves the lesser defects report from the tester. I also believe that a quality product comes from quality code. A quality code depends on how well you are designing the code. And unit test would be the quality controller on my code. Unit tests can also be my code documentation for other developers, a shortcut to the traditional way of preparing the documentation and then kept in somewhere and nobody will going to read it at the end.

Monday, December 5, 2016

I need a backup plan

In the year of 2016, I was planning to write more code about game development but struggling where could I get start. Well, I start off making the fundamental stuff and end up making a tutorial on game development quick start guide. No! This isn't my objective. I want to make a real person of myself while I'm coding on a game. Somehow in one day, my hard disk failed to boot up and all my hard work effort are gone. The most unlucky thing was I don't have a "real" backup plan, thus restoring my files has turn into nightmare. I spend few months doing the recovery until now, there are still missing parts.

In the past few months while doing my recovery job and also trying my best to recover the code, I started to realize on the disaster recovery plan. I put this at the top and highest priority in my working queue. And I have come up a draft plan for the design. Since my backup files are scattered around in different computers and the external medium, I would need a tool to consolidate all my backup files in one place. And also house keeping the backup files is a must to maintain storage capacity.

The high level design would be something like this: as for now, the backup files are currently scattered around in different computer. There will be relocated to a central storage and I'm pretty sure there must be the same files locate in others PC as well, thus I need to have the central storage setup.

Figure 1: When all PCs come online

After that all files on other computers will be synced up together. Well, that's about the plan. If I want to automate the process, how complicate will it be?