First taste of MPI

Different with OpenMP which focuses on multiple threads in one process, MPI defines how multiple processes can collaborate with each other. In this post, I use Open MPI on Arch Linux to do a simple test.

The “Hello World” program is from here, build and run it on one node, not a cluster containing many nodes:

$ mpirun mpi_hello_world
Hello world from processor tesla-p100, rank 16 out of 52 processors
Hello world from processor tesla-p100, rank 34 out of 52 processors
Hello world from processor tesla-p100, rank 35 out of 52 processors
......

Check the CPU information:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              104
On-line CPU(s) list: 0-103
Thread(s) per core:  2
Core(s) per socket:  26
Socket(s):           2
NUMA node(s):        2
......

Although there are 2 physical CPUs in the system, the mpirun only utilizes 1 CPU. Modify the program to output process ID:

......
#include <unistd.h>
......
printf("Hello world from process %d, processor %s, rank %d out of %d processors\n",
         getpid(), processor_name, world_rank, world_size);
......

This time you can make sure different processes are spawned:

$ mpirun mpi_hello_world
Hello world from process 52528, processor tesla-p100, rank 21 out of 52 processors
Hello world from process 52557, processor tesla-p100, rank 31 out of 52 processors
Hello world from process 52597, processor tesla-p100, rank 43 out of 52 processors
......

P.S., if you run mpirun as root, please add --allow-run-as-root option:

# mpirun --allow-run-as-root mpi_hello_world

 

Build SPDZ-2 on Arch Linux

To build SPDZ-2 on Arch Linux, besides installing necessary packages (mpir, libsodium, etc.), You also need to do following steps:

(1) Add following line in CONFIG.mine:

MY_CFLAGS = -DINSECURE

Otherwise, you will meet errors when executing Scripts/setup-online.sh:

# Scripts/setup-online.sh
terminate called after throwing an instance of 'std::runtime_error'
  what():  You are trying to use insecure benchmarking functionality for preprocessing.
You can activate this at compile time by adding -DINSECURE to the compiler options.
Make sure to run make clean as well.
Scripts/setup-online.sh: line 33: 10355 Aborted                 (core dumped) $SPDZROOT/Fake-Offline.x ${players} -lgp ${bits} -lg2 ${g} --default ${default}
dd: failed to open 'Player-Data/Private-Input-0': No such file or directory
dd: failed to open 'Player-Data/Private-Input-1': No such file or directory

(2) Execute make command;

(3) Run Scripts/setup-online.sh;

(4) SPDZ-2 requires python2, but the default python is python3 on Arch Linux. So you need to install python2 manually:

# pacman -S python2

Then modify compile.py:

#!/usr/bin/env python2

Otherwise, you will encounter following errors when running ./compile.py tutorial:

# ./compile.py tutorial
Traceback (most recent call last):
  File "./compile.py", line 19, in <module>
    import Compiler
  File "/root/SPDZ-2/Compiler/__init__.py", line 3, in <module>
    import compilerLib, program, instructions, types, library, floatingpoint
ModuleNotFoundError: No module named 'compilerLib'

Execute ./compile.py tutorial.

Now you can play the example:

# ./Server.x 2 5000 &
# Scripts/run-online.sh tutorial

 

Configure Boost.log in CMakeLists.txt on Arch Linux

Today, I tried to build Boost.Log on Arch Linux, but encountered following errors:

......
undefined reference to `boost::log::v2s_mt_posix::aux::stream_provider<char>::allocate_compound(boost::log::v2s_mt_posix::record&)'
undefined reference to `boost::log::v2s_mt_posix::aux::stream_provider<char>::release_compound(boost::log::v2s_mt_posix::aux::stream_provider<char>::stream_compound*)'
undefined reference to `boost::log::v2s_mt_posix::sources::aux::get_severity_level()'
undefined reference to `boost::log::v2s_mt_posix::aux::unhandled_exception_count()'
undefined reference to `boost::log::v2s_mt_posix::core::open_record(boost::log::v2s_mt_posix::attribute_set const&)'
undefined reference to `boost::log::v2s_mt_posix::core::push_record_move(boost::log::v2s_mt_posix::record&)'
undefined reference to `boost::log::v2s_mt_posix::trivial::logger::get()'
undefined reference to `boost::log::v2s_mt_posix::core::get_logging_enabled() const'
undefined reference to `boost::log::v2s_mt_posix::record_view::public_data::destroy(boost::log::v2s_mt_posix::record_view::public_data const*)'
......

After some investigation, I find at least on Arch Linux, some modules need to be specified explicitly (E.g., log) while some not (E.g., graph). Besides, I also need claim that I want static library:

SET(Boost_USE_STATIC_LIBS ON)
FIND_PACKAGE(Boost REQUIRED COMPONENTS log)

Then use Boost_LIBRARIES in linking executable file:

TARGET_LINK_LIBRARIES(... ${Boost_LIBRARIES})   

Reference:
stackoverflow.

Porting CUDA program from ArchLinux to Ubuntu 16.04

Today I ported a CUDA project from Arch Linux to Ubuntu 16.04, and this post records the pitfalls.

(0) Update cmake to newest version (follow this);

(1) Specify nvcc path in CMakeLists.txt:

SET(CMAKE_CUDA_COMPILER /usr/local/cuda-9.0/bin/nvcc)

otherwise, following error may generate:

......
No CMAKE_CUDA_COMPILER could be found.
......

(2) Since Ubuntu 16.04‘s default compiler is still gcc-5, install gcc-6 first, then pass gcc-6 as default compiler for nvcc:

......
SET(CMAKE_CXX_STANDARD 11)
SET(CMAKE_CUDA_FLAGS "-std=c++11 -ccbin gcc-6")
......

(3) Execute cmake command:

cmake -DCMAKE_C_COMPILER=gcc-6 -DCMAKE_CXX_COMPILER=g++-6 -DCMAKE_BUILD_TYPE=Release ..

References:
Not Locating CUDA Compiler;
CMake: How to pass mode dependent compile flags to nvcc in visual studio environment;
Tensorflow crashes on build on Ubuntu 16.04 when building for skylake (avx512).

 

Fix “error: file owned by …” issue on Arch Linux

I tried to install caffe on my Arch Linux, but it prompted following errors:

$ yaourt -S caffe
......
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/BasicThreadPool.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/FFT.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/FacVec.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/GF2.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/GF2E.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/GF2EX.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/GF2EXFactoring.h'
error: file owned by 'ntl' and 'ntl-threading': 'usr/include/NTL/GF2X.h'\
......

The reason is I installed NTL through pacman before. So the solution is uninstalling NTL first:

$ sudo pacman -R ntl

Then installing caffe successfully:

$ yaourt -S caffe
......
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
No database errors have been found!

Reference:
pacman a file owned by two packages.