Forgetting “-pthread” option may give you a big surprise!

Today, I wrote a small pthread program to do some testing:

#include <pthread.h>

int main(void)
        pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
        pthread_cond_t cv = PTHREAD_COND_INITIALIZER;

        pthread_cond_wait(&cv, &mutex);
        return 0;

Build and test it on OpenBSD-current (version is 6.4):

# cc cv_test.c -o cv_test
# ./cv_test

The program will block there and it is my expected result. Switch to Arch Linux (kernel version is 4.18.9):

# cc cv_test.c -o cv_test
# ./cv_test

The program will exit immediately. I doubt it is “spurious awake” firstly, but can’t get a convincing explanation. Using ldd to check program. On OpenBSD:

# ldd cv_test
        Start            End              Type  Open Ref GrpRef Name
        000000d4c3a00000 000000d4c3c02000 exe   1    0   0      cv_test
        000000d6e6007000 000000d6e62f6000 rlib  0    1   0      /usr/lib/
        000000d6db100000 000000d6db100000 0    1   0      /usr/libexec/

On Arch Linux:

# ldd cv_test (0x00007ffde91c6000) => /usr/lib/ (0x00007f3e3169b000)
        /lib64/ => /usr/lib64/ (0x00007f3e3187a000)

Nothing special. After seeking help on stackoverflow, the answer is I need adding -pthread option:

# cc -pthread cv_test.c -o cv_test
# ./cv_test

This time it worked perfectly. Checking linked library:

# ldd cv_test (0x00007fff48be8000) => /usr/lib/ (0x00007fa46f84c000) => /usr/lib/ (0x00007fa46f688000)
        /lib64/ => /usr/lib64/ (0x00007fa46f888000)

Why doesn’t Linux give me a link error which prompts I need link libpthread? It seems not make sense.

First taste of MPI

Different with OpenMP which focuses on multiple threads in one process, MPI defines how multiple processes can collaborate with each other. In this post, I use Open MPI on Arch Linux to do a simple test.

The “Hello World” program is from here, build and run it on one node, not a cluster containing many nodes:

$ mpirun mpi_hello_world
Hello world from processor tesla-p100, rank 16 out of 52 processors
Hello world from processor tesla-p100, rank 34 out of 52 processors
Hello world from processor tesla-p100, rank 35 out of 52 processors

Check the CPU information:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              104
On-line CPU(s) list: 0-103
Thread(s) per core:  2
Core(s) per socket:  26
Socket(s):           2
NUMA node(s):        2

Although there are 2 physical CPUs in the system, the mpirun only utilizes 1 CPU. Modify the program to output process ID:

#include <unistd.h>
printf("Hello world from process %d, processor %s, rank %d out of %d processors\n",
         getpid(), processor_name, world_rank, world_size);

This time you can make sure different processes are spawned:

$ mpirun mpi_hello_world
Hello world from process 52528, processor tesla-p100, rank 21 out of 52 processors
Hello world from process 52557, processor tesla-p100, rank 31 out of 52 processors
Hello world from process 52597, processor tesla-p100, rank 43 out of 52 processors

P.S., if you run mpirun as root, please add --allow-run-as-root option:

# mpirun --allow-run-as-root mpi_hello_world


Build SPDZ-2 on Arch Linux

To build SPDZ-2 on Arch Linux, besides installing necessary packages (mpir, libsodium, etc.), You also need to do following steps:

(1) Add following line in CONFIG.mine:


Otherwise, you will meet errors when executing Scripts/

# Scripts/
terminate called after throwing an instance of 'std::runtime_error'
  what():  You are trying to use insecure benchmarking functionality for preprocessing.
You can activate this at compile time by adding -DINSECURE to the compiler options.
Make sure to run make clean as well.
Scripts/ line 33: 10355 Aborted                 (core dumped) $SPDZROOT/Fake-Offline.x ${players} -lgp ${bits} -lg2 ${g} --default ${default}
dd: failed to open 'Player-Data/Private-Input-0': No such file or directory
dd: failed to open 'Player-Data/Private-Input-1': No such file or directory

(2) Execute make command;

(3) Run Scripts/;

(4) SPDZ-2 requires python2, but the default python is python3 on Arch Linux. So you need to install python2 manually:

# pacman -S python2

Then modify

#!/usr/bin/env python2

Otherwise, you will encounter following errors when running ./ tutorial:

# ./ tutorial
Traceback (most recent call last):
  File "./", line 19, in <module>
    import Compiler
  File "/root/SPDZ-2/Compiler/", line 3, in <module>
    import compilerLib, program, instructions, types, library, floatingpoint
ModuleNotFoundError: No module named 'compilerLib'

Execute ./ tutorial.

Now you can play the example:

# ./Server.x 2 5000 &
# Scripts/ tutorial


Configure Boost.log in CMakeLists.txt on Arch Linux

Today, I tried to build Boost.Log on Arch Linux, but encountered following errors:

undefined reference to `boost::log::v2s_mt_posix::aux::stream_provider<char>::allocate_compound(boost::log::v2s_mt_posix::record&)'
undefined reference to `boost::log::v2s_mt_posix::aux::stream_provider<char>::release_compound(boost::log::v2s_mt_posix::aux::stream_provider<char>::stream_compound*)'
undefined reference to `boost::log::v2s_mt_posix::sources::aux::get_severity_level()'
undefined reference to `boost::log::v2s_mt_posix::aux::unhandled_exception_count()'
undefined reference to `boost::log::v2s_mt_posix::core::open_record(boost::log::v2s_mt_posix::attribute_set const&)'
undefined reference to `boost::log::v2s_mt_posix::core::push_record_move(boost::log::v2s_mt_posix::record&)'
undefined reference to `boost::log::v2s_mt_posix::trivial::logger::get()'
undefined reference to `boost::log::v2s_mt_posix::core::get_logging_enabled() const'
undefined reference to `boost::log::v2s_mt_posix::record_view::public_data::destroy(boost::log::v2s_mt_posix::record_view::public_data const*)'

After some investigation, I find at least on Arch Linux, some modules need to be specified explicitly (E.g., log) while some not (E.g., graph). Besides, I also need claim that I want static library:


Then use Boost_LIBRARIES in linking executable file:



Porting CUDA program from ArchLinux to Ubuntu 16.04

Today I ported a CUDA project from Arch Linux to Ubuntu 16.04, and this post records the pitfalls.

(0) Update cmake to newest version (follow this);

(1) Specify nvcc path in CMakeLists.txt:

SET(CMAKE_CUDA_COMPILER /usr/local/cuda-9.0/bin/nvcc)

otherwise, following error may generate:

No CMAKE_CUDA_COMPILER could be found.

(2) Since Ubuntu 16.04‘s default compiler is still gcc-5, install gcc-6 first, then pass gcc-6 as default compiler for nvcc:

SET(CMAKE_CUDA_FLAGS "-std=c++11 -ccbin gcc-6")

(3) Execute cmake command:


Not Locating CUDA Compiler;
CMake: How to pass mode dependent compile flags to nvcc in visual studio environment;
Tensorflow crashes on build on Ubuntu 16.04 when building for skylake (avx512).