Be careful of thread stack size

Today, my colleague came across a thread stack overflow core dump:

Capture

From above diagram, we can see only this function’s stack will occupy ~7 MiB space. Check the stack size configuration on system:

$ ulimit -S -s
8192

just 8 MiB. Double the stack size:

$ ulimit -S -s 16384

The program won’t crash.

Reference:
General: How do I change my default limits for stack size, core file size, etc.?.

 

Reflection on my work in 2017

As 2017 draws to a close, it is time to recap what I did in this year. The “work” here is divided into 2 parts: daily and part-time.

Daily job:

(1) Cryptography and security.
I joined in current company in December 2016, so 2017 is literally my fresh year. Our team focuses on cryptography, and this area has a high requirement on maths, such as Number Theory, Linear Algebra, etc, so it is a really challenge for me since I have left campus for nearly 10 years, and my past work experience involved much on engineering. Honestly, I have made some progress in maths compared to one year ago, but there is still a large distance to the expert in cryptography field. If you want to dive into this discipline, this tutorial is a good rudimentary material.
In this December, I went to London to attend Black Hat meeting (The trip report is here). The security is definitely becoming more and more important, and one of my deep feeling is many sub-fields of security don’t require much maths background like cryptography, but they require you are versed in the whole computer system. Maybe security becomes next big business opportunity.

(2) High performance computing.
Since cryptography introduces a lot of computation, reducing time is an important task.

a) GPU programming.
GPU is heavily used in HPC area now, so I learned CUDA/GPU programming and implemented some algorithms in GPU. One by-product isĀ lscuda, a command mimics lscpu on Linux.

b) OpenMP.
Besides using GPU, harnessing OpenMP to paralleling code in CPU is my another task in the past year.

c) C++.
Because most Open Source code in cryptography is written in C++, one big harvest is I got the chance to refresh and use C++ in my daily life, and it really improved my coding skill. E.g., a generic log implemented in C++.

d) Performance tuning.
Finding the hot-spot of program is an eternal topic in tuning high performance computing. I dived into perf/Flamegraph tools shipped on Linux, and these weapons indeed helped me to find the culprits.

(3) Others.
I not only utilize the Open Source work, but also contribute to them, such as FHEW, sql-parser, and so on.

Part-time projects:

(1) Rust programming language.
Every year I will try to get my hands dirty on one new programming language. After watching this video, I decided to learn Rust this year. STREAM and RustTCPFramework are 2 small exercises.

(2) eBPF.
When using perf, I knew eBPF is a new powerful tool on Linux, so I also spent a lot of time in bcc project. The by-product of this procedure is using Python.

(3) OpenBSD.
The OpenBSD is famous in security area. Besides submitting patch for it, I also wrote some articles introducing this OS. BTW, lscpuand umalloc are 2 projects I created for BSDs and Unix.

(4) Recommended books:
The following are books I read this year and think they are worthy for recommendation:

a) Code: The Hidden Language of Computer Hardware and Software.
This book introduces the basic composition of the computer. For implementing some cryptography algorithms, I need to build the circuit logic. This book gives a good reference.

b) Multicore Application Programming: for Windows, Linux, and Oracle Solaris (Developer’s Library).
This book was published in 2010, but it is still a comprehensive handbook for learning parallel programming.

c) Grokking Algorithms: An illustrated guide for programmers and other curious people.
This book just covers the basic knowledge of algorithm, not too deep. It is suitable for reviewing algorithm before interview.

For the next year, now I plan to do the following tasks:
(1) Continue to learn security and cryptography;
(2) Study one functional programming language.

Let me keep going!

Specify compiler when using CUDA

I try to build a project which uses CUDA. The CUDA on my machine is the newest 9.1. The following error is reported:

In file included from /opt/cuda/include/host_config.h:50:0,
                 from /opt/cuda/include/cuda_runtime.h:78,
                 from <command-line>:0:
/opt/cuda/include/crt/host_config.h:121:2: error: #error -- unsupported GNU version! gcc versions later than 6 are not supported!
 #error -- unsupported GNU version! gcc versions later than 6 are not supported!
  ^~~~~

Check /opt/cuda/include/crt/host_config.h:

#if __GNUC__ > 6

#error -- unsupported GNU version! gcc versions later than 6 are not supported!

#endif /* __GNUC__ > 6 */

The solution is to specify the gcc-6 compiler during running cmake command:

# cmake -DCMAKE_C_COMPILER=gcc-6 -DCMAKE_CXX_COMPILER=g++-6 ..

The reference is here.

Fix locale configuration issue on Arch Linux

On a new installed Arch Linux server, I come across the following problem:

# locale -a
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_COLLATE to default locale: No such file or directory
C
POSIX
# locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Especially when using man command:

# man wait
man: can't set the locale; make sure $LC_* and $LANG are correct

After referring this post, I fix the issue. My /etc/locale.conf is like this:

# cat /etc/locale.conf
LANG=en_US.UTF-8

But en_US.UTF-8 is commented out in /etc/locale.gen:

#en_US.UTF-8 UTF-8

uncomment it, then run locale-gen command, all become normal.