First taster of Standard C++ Thread Library on OpenBSD

Today I tried Standard C++ Thread Library on OpenBSD, since it requires the compiler to support C++11 standard, and the default c++ only support C++98 (please refer here), so I need to switch to clang++. The program is just a classic “Hello World”:

#include <thread>
#include <iostream>

void hello()
{
    std::cout << "Hello World!\n";
}

int main(void)
{
    std::thread t(hello);
    t.join();
    return 0;
}

Built and run it:

# clang++ -std=c++11 hello.cpp
root:/root/Project# ./a.out
terminate called after throwing an instance of 'std::system_error'
  what():  Enable multithreading to use std::thread: Operation not permitted
Abort trap (core dumped)

Whoops! The program crashed. After reading this post, adding -pthread during compilation fixed this issue:

# clang++ -pthread -std=c++11 hello.cpp
# ./a.out
Hello World!

The anatomy of ldd program on OpenBSD

In the past week, I read the ldd source code on OpenBSD to get a better understanding of how it works. And this post should also be a reference for other*NIX OSs.

The ELF file is divided into 4 categories: relocatable, executable, shared, and core. Only the executable and shared object files may have dynamic object dependencies, so the ldd only check these 2 kinds of ELF file:

(1) Executable.

ldd leverages the LD_TRACE_LOADED_OBJECTS environment variable in fact, and the code is as following:

if (setenv("LD_TRACE_LOADED_OBJECTS", "true", 1) < 0)
    err(1, "setenv(LD_TRACE_LOADED_OBJECTS)");

When LD_TRACE_LOADED_OBJECTS is set to 1 or true, running executable file will show shared objects needed instead of running it, so you even not needldd to check executable file. See the following outputs:

# /usr/bin/ldd
usage: ldd program ...
# LD_TRACE_LOADED_OBJECTS=1 /usr/bin/ldd
        Start            End              Type Open Ref GrpRef Name
        00000b6ac6e00000 00000b6ac7003000 exe  1    0   0      /usr/bin/ldd
        00000b6dbc96c000 00000b6dbcc38000 rlib 0    1   0      /usr/lib/libc.so.89.3
        00000b6d6ad00000 00000b6d6ad00000 rtld 0    1   0      /usr/libexec/ld.so  

(2) Shared object.

The code to print dependencies of shared object is as following:

if (ehdr.e_type == ET_DYN && !interp) {
    if (realpath(name, buf) == NULL) {
        printf("realpath(%s): %s", name,
            strerror(errno));
        fflush(stdout);
        _exit(1);
    }
    dlhandle = dlopen(buf, RTLD_TRACE);
    if (dlhandle == NULL) {
        printf("%s\n", dlerror());
        fflush(stdout);
        _exit(1);
    }
    _exit(0);
}

Why the condition of checking a ELF file is shared object or not is like this:

if (ehdr.e_type == ET_DYN && !interp) {
    ......
}

That’s because the file type of position-independent executable (PIE) is the same as shared object, but normally PIE contains a interpreter program header since it needs dynamic linker to load it while shared object lacks (refer this article). So the above condition will filter PIE file.

The dlopen(buf, RTLD_TRACE) is used to print dynamic object information. And the actual code is like this:

if (_dl_traceld) {
    _dl_show_objects();
    _dl_unload_shlib(object);
    _dl_exit(0);
}

In fact, you can also implement a simple application which outputs dynamic object information for shared object yourself:

#include <dlfcn.h>

int main(int argc, char **argv)
{
    dlopen(argv[1], RTLD_TRACE);
    return 0;
}

Compile and use it to analyze /usr/lib/libssl.so.43.2:

# cc lddshared.c
# ./a.out /usr/lib/libssl.so.43.2
    Start            End              Type Open Ref GrpRef Name
    000010e2df1c5000 000010e2df41a000 dlib 1    0   0      /usr/lib/libssl.so.43.2
    000010e311e3f000 000010e312209000 rlib 0    1   0      /usr/lib/libcrypto.so.41.1

The same as using ldd directly:

# ldd /usr/lib/libssl.so.43.2
/usr/lib/libssl.so.43.2:
    Start            End              Type Open Ref GrpRef Name
    00001d9ffef08000 00001d9fff15d000 dlib 1    0   0      /usr/lib/libssl.so.43.2
    00001d9ff1431000 00001d9ff17fb000 rlib 0    1   0      /usr/lib/libcrypto.so.41.1

Through the studying of ldd source code, I also get many by-products: such as knowledge of ELF file, linking and loading, etc. So diving into code is a really good method to learn *NIX deeper!

Install googletest on OpenBSD

To use googletest on OpenBSD, I need to install CMake firstly:

# pkg_add cmake

Then Download googletest and create build directory:

# git clone https://github.com/google/googletest.git
# cd googletest
# mkdir build
# cd build

If use default gcc compiler, the compilation will generate following errors:

# make
[  2%] Building CXX object googlemock/CMakeFiles/gmock.dir/__/googletest/src/gtest-all.cc.o
cc1plus: warnings being treated as errors
In file included from /usr/include/g++/tr1/tuple:159,
                 from /root/Project/googletest/googletest/include/gtest/internal/gtest-port.h:746,
                 from /root/Project/googletest/googletest/include/gtest/internal/gtest-internal.h:40,
                 from /root/Project/googletest/googletest/include/gtest/gtest.h:58,
                 from /root/Project/googletest/googletest/src/gtest-all.cc:39:
/usr/include/g++/tr1/functional: In constructor 'std::tr1::_Mem_fn<_Res _Class::*>::_Mem_fn(_Res _Class::*)':
/usr/include/g++/tr1/functional:470: warning: declaration of '__pm' shadows a member of 'this'
In file included from /usr/include/g++/tr1/tuple:159,
                 from /root/Project/googletest/googletest/include/gtest/internal/gtest-port.h:746,
                 from /root/Project/googletest/googletest/include/gtest/internal/gtest-internal.h:40,
                 from /root/Project/googletest/googletest/include/gtest/gtest.h:58,
                 from /root/Project/googletest/googletest/src/gtest-all.cc:39:
/usr/include/g++/tr1/functional: In constructor 'std::tr1::_Simple_type_wrapper<_Tp>::_Simple_type_wrapper(_Tp)':
/usr/include/g++/tr1/functional:804: warning: declaration of '__value' shadows a member of 'this'
*** Error 1 in . (googlemock/CMakeFiles/gmock.dir/build.make:63 'googlemock/CMakeFiles/gmock.dir/__/googletest/src/gtest-all.cc.o': cd /root...)
*** Error 1 in . (CMakeFiles/Makefile2:90 'googlemock/CMakeFiles/gmock.dir/all')
*** Error 1 in /root/Project/googletest/build (Makefile:139 'all')

So I switch to clang, and all build is OK:

# CC=clang cmake ..
# make
# make install

The header file and library search path of cc on OpenBSD

On my OpenBSD 6.1/amd64 system, the default header file search path of cc compiler is only /usr/include:

# echo | cc -E -Wp,-v -
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"

The default library search path is only /usr/lib:

# cc -Xlinker --verbose  2>/dev/null | grep SEARCH | sed 's/SEARCH_DIR("=\?\([^"]\+\)"); */\1\n/g'  | grep -vE '^$'
SEARCH_DIR("/usr/lib");

So if you want to include other header file or link libraries in other directory, you need to specify it explicitly. For example:

# cc -I/usr/local/include -L/usr/local/lib ...

Reference:
Finding out what the GCC include path is;
How to print the ld(linker) search path.

lscpu for OpenBSD/FreeBSD

There is a neat command, lscpu, which is very handy to display CPU information on GNU/Linux OS:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           2
......

But unfortunately, the BSD OSs lack this command, maybe one reason is lscpu relies heavily on /proc file system which BSD don’t provide, :-). TakeOpenBSD as an example, if I want to know CPU information, dmesg should be one choice:

# dmesg | grep -i cpu
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz, 2527.35 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF,SENSOR
cpu0: 3MB 64b/line 8-way L2 cache
cpu0: apic clock running at 266MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2.1.3, IBE
......

But the output makes me feeling messy, not very clear. As for dmidecode, it used to be another option, but now can’t work out-of-box because it will access /dev/mem which for security reason, OpenBSD doesn’t allow by default (You can refer this discussion):

# ./dmidecode
# dmidecode 3.1
Scanning /dev/mem for entry point.
/dev/mem: Operation not permitted

Based on above situation, I want a specified command for showing CPU information for my BSD box. So in the past 2 weeks, I developed a lscpu program for OpenBSD/FreeBSD, or more accurately, OpenBSD/FreeBSD on x86 architecture since I only have some Intel processors at hand. The application getsCPU metrics from 2 sources:

(1) sysctl functions.
The BSD OSs provide sysctl interface which I can use to get general CPU particulars, such as how many CPUs the system contains, the byte-order of CPU, etc.

(2) CPUID instruction. For x86 architecture, CPUID instruction can obtain very detail information of CPU. This coding work is a little tedious and error-prone, not only because I need to reference both Intel and AMD specifications since these 2 vendors have minor distinctions, but also I need to parse the bits of register values.

The code is here, and if you run OpenBSD/FreeBSD on x86 processors, please try it. It will be better you can give some feedback or report the issues, and I appreciate it very much. In the future if I have other CPUs resource, such as ARM or SPARC64, maybe I will enrich this small program.