What is a good software engineer job?

After working as a professional software engineer for 10 years, I want to share what I think is a good software job here.

(1) The opportunity of trying different stuff.
You have the chance to touch the different technology, not necessarily the newest. E.g., after working as an application engineer for one or two years, you can make your hands dirty on kernel. Be bored of using C++, it is time to have a taste of rust or Go. All in all, keep freshment is an important factor to motivate you.

(2) Contribute to FOSS.
No one will deny that both the tech giants and start-ups depend on FOSS (Free/Open Source Software) greatly now (Even Microsoft “loves” Linux). If your employer encourage you to contribute back to the project rather than only exploit it, it is really beneficial for community, yourself and company. “A stone kill three birds!”

(3) Sponsor you attend tech conferences and training.
Attending tech conference can expand your horizon, know more new people, and be aware of what other companies are doing and concentrating on in your area. If you can give a talk on some famous conferences, it will earn reputation for both you and your company. Besides this, life-long studying is important for software engineer, so it will be fantastic if your boss is willing to pay money for your enrichment.

(4) Encourage innovation and knowledge sharing.
It will be a great favor if you have ~10% work time for your own project. Actually, the company can select and cultivate seed from these projects, and it may bring potential success. The classical example is gmail. On top of that, Knowledge sharing is a good section in team meeting in which you can learn skills from your colleagues.

Besides the aspects mentioned above, I think there are some other factors. For example, salary, welfare, promotion, etc. Since these are not software job exclusive, and a little complicated among different industries, companies and even countries, I won’t elaborate it here.

Read Ctxt from stream in using HElib

Assume you need to read Ctxt from a stream (file, socket, or whatever) in using HElib, the general pattern is like this:

Ctxt ctxt(*pubKey);
while (fs >> ctxt)
{
    std::cout << ++count << std::endl;
}

But this will trigger following error (If you use one version of HElib to encrypt Ctxt, while use another version of HElib to process Ctxt, this error may occur too.):

Searching for cc='[' (ascii 91), found c='▒' (ascii -1)

The root cause of this issue is about handling EOF. To solve this issue, there are 2 methods:

(1)

Ctxt ctxt(*pubKey);
while (!fs.eof())
{
    fs >> ctxt >> std::ws;
    if (fs.fail())
    {
        break;
    }
}

fs >> ctxt >> std::ws; will stop when the EOF is met. Since fs.fail() won’t return true when eofbit is set (please refer here), fs.eof() will becometrue to terminate while-loop.

(2) know how many Ctxt will be read in advance. For example:

Ctxt ctxt(*pubKey);
for (int i = 0; i < 10; i++)
{
    fs >> ctxt;
}

Process large data in external memory

This week, I implemented a small program which handles a data set. The volume of data set is so big that it can’t be stored in main memory.

I first tried to use stxxlstxxl is an awesome library which mimics STL and processes data in external memory. But it has many limitations, such as data type should be plain old data type. Since my type doesn’t provide default constructor, stxxl can’t satisfy my need (please refer this discussion). I also make attempts on other workaruonds, but all failed.

Finally, I used a simple method: Open a file, serialize the data set into file, and treat the file like the main memory. Although it is not the most efficient approach, the program is vey clear, and not prone to bugs. So I decide to use it as a demo, and improve it gradually.

Update: Split large file into smaller ones, and use multiple threads to handle them is a good idea.