Category: Technology

Read Ctxt from stream in using HElib

Assume you need to read Ctxt from a stream (file, socket, or whatever) in using HElib, the general pattern is like this:

Ctxt ctxt(*pubKey);
while (fs >> ctxt)
{
    std::cout << ++count << std::endl;
}

But this will trigger following error (If you use one version of HElib to encrypt Ctxt, while use another version of HElib to process Ctxt, this error may occur too.):

Searching for cc='[' (ascii 91), found c='▒' (ascii -1)

The root cause of this issue is about handling EOF. To solve this issue, there are 2 methods:

(1)

Ctxt ctxt(*pubKey);
while (!fs.eof())
{
    fs >> ctxt >> std::ws;
    if (fs.fail())
    {
        break;
    }
}

fs >> ctxt >> std::ws; will stop when the EOF is met. Since fs.fail() won’t return true when eofbit is set (please refer here), fs.eof() will becometrue to terminate while-loop.

(2) know how many Ctxt will be read in advance. For example:

Ctxt ctxt(*pubKey);
for (int i = 0; i < 10; i++)
{
    fs >> ctxt;
}

Invoke profile function in Nsight

When using Nsight to develop CUDA program, you can use profile function to profile the program:

1

You can also toggle the C/C++ and profile view in the right corner:

2

BTW, if you only want to profile a part of the program (not the whole), you can usecudaProfilerStart() and cudaProfilerStop to surround the code, then untick “Start execution with profiling enabled” in “Profile Configuration“:

Process large data in external memory

This week, I implemented a small program which handles a data set. The volume of data set is so big that it can’t be stored in main memory.

I first tried to use stxxl. stxxl is an awesome library which mimics STL and processes data in external memory. But it has many limitations, such as data type should be plain old data type. Since my type doesn’t provide default constructor, stxxl can’t satisfy my need (please refer this discussion). I also make attempts on other workaruonds, but all failed.

Finally, I used a simple method: Open a file, serialize the data set into file, and treat the file like the main memory. Although it is not the most efficient approach, the program is vey clear, and not prone to bugs. So I decide to use it as a demo, and improve it gradually.

Update: Split large file into smaller ones, and use multiple threads to handle them is a good idea.

 

Be careful of file sequence in linking process

Check following A.h:

# cat A.h
#pragma once

#include <iostream>
#include <vector>

class A
{
public:
        std::vector<int> v;
        A()
        {
                v.push_back(1);
                std::cout << "Enter A's constructor...\n";
        }
        int getFirstElem()
        {
                v.push_back(2);
                std::cout << "Enter A's getFirstElem...\n";
                return v[0];
        }
        ~A()
        {
                std::cout << "Enter A's destructor...\n";
        }
};

int func();

And A.cpp:

# cat A.cpp
#include "A.h"

static A a;

int func()
{
        return a.getFirstElem();
}

The A.cpp just define a A‘s static instance, and a func() returns first element in a‘s internal vector.

Check another file which utilizes A.h and A.cpp:

# cat hello.cpp
#include <iostream>
#include "A.h"

static int gP = func();

int main()
{
    std::cout << gP << std::endl;
    return 0;
}

Compile them:

# clang++ -c hello.cpp
# clang++ -c A.cpp

Link hello.o first and execute the program:

# clang++ hello.o A.o
# ./a.out
Enter A's getFirstElem...
Enter A's constructor...
2
Enter A's destructor...

Then link A.o first and execute the program:

# clang++ A.o hello.o
# ./a.out
Enter A's constructor...
Enter A's getFirstElem...
1
Enter A's destructor...

The results are different. In first case, when call a‘s getFirstElem() function, its constructor is not even called. Please pay attention to it!