Tag: Linux

Downgrade boost on Ubuntu 16.04

In the past 2 days, I was tortured by boost. The default boostversion on Ubuntu 16.04 is 1.58, but I met following compile errors:

......
/usr/include/boost/multi_index/detail/bucket_array.hpp: In static member function ‘static std::size_t boost::multi_index::detail::bucket_array_base<_>::size_index(std::size_t)’:
/usr/include/boost/multi_index/detail/bucket_array.hpp:84:62: error: invalid use of non-lvalue array
     const std::size_t *bound=std::lower_bound(sizes,sizes+sizes_length,n);
                                                              ^~~~~~~~~~~~
/usr/include/boost/multi_index/detail/bucket_array.hpp:85:25: error: invalid use of non-lvalue array
     if(bound==sizes+sizes_length)--bound;
                         ^~~~~~~~~~~~
/usr/include/boost/multi_index/detail/bucket_array.hpp:86:22: error: invalid use of non-lvalue array
     return bound-sizes;
......

I downgraded boost to 1.55; and downloaded and built it:

$ ./bootstrap.sh --prefix=/usr/local  
$ sudo ./b2 -a -q install

This time I found the default gcc-5 could not compile successfully. So I followed this post to install gcc-6, and modified/home/nan/boost_1_55_0/tools/build/v2/user-config.jam file to use gcc-6 to compile boost:

......
# Configure specific gcc version, giving alternative name to use.
using gcc : 6 : g++-6 ;
......

Then my project can be compiled successfully. Check gcc search header file path and library path:

$ echo | gcc-6 -E -Wp,-v -
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/6/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/6/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
......

$ ldconfig -v 2>/dev/null | grep -v ^$'\t'
/usr/local/cuda-9.0/targets/x86_64-linux/lib:
/usr/lib/x86_64-linux-gnu/libfakeroot:
/usr/local/lib:
/lib/x86_64-linux-gnu:
/usr/lib/x86_64-linux-gnu:
/usr/lib/nvidia-384:
/usr/lib32/nvidia-384:
/lib32:
/usr/lib32:
/lib:
/usr/lib:
/usr/lib/nvidia-384/tls: (hwcap: 0x8000000000000000)
/usr/lib32/nvidia-384/tls: (hwcap: 0x8000000000000000)

You will find the fresh installed boost 1.55 (in /usr/local directory) always be found before default boost 1.58 (header files are in /usr/include/boostand libraries in /usr/lib/x86_64-linux-gnu).

Update keyring first if your Arch Linux is old enough

My Arch Linux is not updated for nearly 3 months. When running pacman -Syu, it prompts following errors:

$ sudo pacman -Syu
......
error: python-dnspython: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-dnspython-1.16.0-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: python-distlib: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-distlib-0.2.8-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: python-pytoml: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-pytoml-0.1.20-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: python2-distlib: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python2-distlib-0.2.8-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: python2-dnspython: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python2-dnspython-1.16.0-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: python2-pytoml: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python2-pytoml-0.1.20-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: qbittorrent: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/qbittorrent-4.1.5-1-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.
......

The solution is updating archlinux-keyring first:

$ sudo pacman -S archlinux-keyring

Then all goes well!

Conditional variable takeaways

Conditional variable is a common concept in both user-space and kernel-space programming. IMHO, it is a little complicated synchronization mechanism. Recently, I came across Measuring context switching and memory overheads for Linux threads, and this article provides an example which I think is a good tutorial about how to understand and use conditional variable.

The parent thread code is like following:

  /* parent thread */
  pthread_mutex_lock(&si.mutex);
  pthread_t childt;
  pthread_create(&childt, NULL, threadfunc, (void*)&si);

  // Each iteration of this loop will switch context from the parent to the
  // child and back - two context switches. The child signals first.
  ......
  for (int i = 0; i < NUM_ITERATIONS; ++i) {
    pthread_cond_wait(&si.cv, &si.mutex);
    pthread_cond_signal(&si.cv);
  }
  pthread_mutex_unlock(&si.mutex);

And this is the child thread code:

// The child thread signals first
  pthread_mutex_lock(&si->mutex);
  pthread_cond_signal(&si->cv);
  for (int i = 0; i < NUM_ITERATIONS; ++i) {
    pthread_cond_wait(&si->cv, &si->mutex);
    pthread_cond_signal(&si->cv);
  }
  pthread_mutex_unlock(&si->mutex);

(1) The first takeaway is pthread_cond_signal() must be called after pthread_cond_wait() in timing sequence; otherwise the signal won’t be received.

Check preceding code, before launching child thread:

    ......
    pthread_t childt;
    pthread_create(&childt, NULL, threadfunc, (void*)&si);
    ......

The parent thread must hold mutex first:

    ......
    pthread_mutex_lock(&si.mutex);
    ......

Then in the first iteration of loop, release the mutex and wait for notification:

    ......
    pthread_cond_wait(&si.cv, &si.mutex);
    ......

This can guarantee when child thread sends signal, the parent thread is already in the wait queue:

  ......
  pthread_mutex_lock(&si->mutex);
  pthread_cond_signal(&si->cv);
  ......

(2) The other thing we should remember is before and after calling pthread_cond_wait(), the current thread must hold the mutex. I.e., before callingpthread_cond_wait(), the current thread get the mutex, then in pthread_cond_wait(), put the current thread in the wait queue and release the mutexatomically. Once another thread signals current thread, it will reacquire mutex and return from pthread_cond_wait().

Forgetting “-pthread” option may give you a big surprise!

Today, I wrote a small pthread program to do some testing:

#include <pthread.h>

int main(void)
{
        pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
        pthread_cond_t cv = PTHREAD_COND_INITIALIZER;

        pthread_mutex_lock(&mutex);
        pthread_cond_wait(&cv, &mutex);
        return 0;
}

Build and test it on OpenBSD-current (version is 6.4):

# cc cv_test.c -o cv_test
# ./cv_test

The program will block there and it is my expected result. Switch to Arch Linux (kernel version is 4.18.9):

# cc cv_test.c -o cv_test
# ./cv_test
#

The program will exit immediately. I doubt it is “spurious awake” firstly, but can’t get a convincing explanation. Using ldd to check program. On OpenBSD:

# ldd cv_test
cv_test:
        Start            End              Type  Open Ref GrpRef Name
        000000d4c3a00000 000000d4c3c02000 exe   1    0   0      cv_test
        000000d6e6007000 000000d6e62f6000 rlib  0    1   0      /usr/lib/libc.so.92.5
        000000d6db100000 000000d6db100000 ld.so 0    1   0      /usr/libexec/ld.so

On Arch Linux:

# ldd cv_test
        linux-vdso.so.1 (0x00007ffde91c6000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f3e3169b000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f3e3187a000)

Nothing special. After seeking help on stackoverflow, the answer is I need adding -pthread option:

# cc -pthread cv_test.c -o cv_test
# ./cv_test

This time it worked perfectly. Checking linked library:

# ldd cv_test
        linux-vdso.so.1 (0x00007fff48be8000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fa46f84c000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fa46f688000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fa46f888000)

Why doesn’t Linux give me a link error which prompts I need link libpthread? It seems not make sense.

First taste of MPI

Different with OpenMP which focuses on multiple threads in one process, MPI defines how multiple processes can collaborate with each other. In this post, I use Open MPI on Arch Linux to do a simple test.

The “Hello World” program is from here, build and run it on one node, not a cluster containing many nodes:

$ mpirun mpi_hello_world
Hello world from processor tesla-p100, rank 16 out of 52 processors
Hello world from processor tesla-p100, rank 34 out of 52 processors
Hello world from processor tesla-p100, rank 35 out of 52 processors
......

Check the CPU information:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              104
On-line CPU(s) list: 0-103
Thread(s) per core:  2
Core(s) per socket:  26
Socket(s):           2
NUMA node(s):        2
......

Although there are 2 physical CPUs in the system, the mpirun only utilizes 1 CPU. Modify the program to output process ID:

......
#include <unistd.h>
......
printf("Hello world from process %d, processor %s, rank %d out of %d processors\n",
         getpid(), processor_name, world_rank, world_size);
......

This time you can make sure different processes are spawned:

$ mpirun mpi_hello_world
Hello world from process 52528, processor tesla-p100, rank 21 out of 52 processors
Hello world from process 52557, processor tesla-p100, rank 31 out of 52 processors
Hello world from process 52597, processor tesla-p100, rank 43 out of 52 processors
......

P.S., if you run mpirun as root, please add --allow-run-as-root option:

# mpirun --allow-run-as-root mpi_hello_world