Tag: benchmark

I want to use google/benchmark on OpenBSD, but find it support many platforms whereas lacks OpenBSD (the code is here):

#if defined(__CYGWIN__)
  #define BENCHMARK_OS_CYGWIN 1
#elif defined(_WIN32)
  #define BENCHMARK_OS_WINDOWS 1
#elif defined(__APPLE__)
  #define BENCHMARK_OS_APPLE 1
  #include "TargetConditionals.h"
  #if defined(TARGET_OS_MAC)
    #define BENCHMARK_OS_MACOSX 1
    #if defined(TARGET_OS_IPHONE)
      #define BENCHMARK_OS_IOS 1
    #endif
  #endif
#elif defined(__FreeBSD__)
  #define BENCHMARK_OS_FREEBSD 1
#elif defined(__NetBSD__)
  #define BENCHMARK_OS_NETBSD 1
#elif defined(__linux__)
  #define BENCHMARK_OS_LINUX 1
#elif defined(__native_client__)
  #define BENCHMARK_OS_NACL 1
#elif defined(EMSCRIPTEN)
  #define BENCHMARK_OS_EMSCRIPTEN 1
#elif defined(__rtems__)
  #define BENCHMARK_OS_RTEMS 1
#elif defined(__Fuchsia__)
#define BENCHMARK_OS_FUCHSIA 1
#elif defined (__SVR4) && defined (__sun)
#define BENCHMARK_OS_SOLARIS 1
#endif

Although it can be built successfully on OpenBSD, but “make test” reports some failures:

# make test

......

91% tests passed, 5 tests failed out of 54

Total Test time (real) =  40.18 sec

The following tests FAILED:
          1 - benchmark (Child aborted)
         38 - options_benchmarks (Child aborted)
         39 - basic_benchmark (Child aborted)
         43 - fixture_test (Child aborted)
         47 - reporter_output_test (Child aborted)
Errors while running CTest
*** Error 8 in /root/Project/benchmark/build (Makefile:130 'test': /usr/local/bin/ctest --force-new-ctest-process --exclude-regex "CMake.Fil...)

Check the following simple test file:

# cat test.cc
#include <benchmark/benchmark.h>

static void BM_StringCreation(benchmark::State& state) {
  for (auto _ : state)
    std::string empty_string;
}
// Register the function as a benchmark
BENCHMARK(BM_StringCreation);

// Define another benchmark
static void BM_StringCopy(benchmark::State& state) {
  std::string x = "hello";
  for (auto _ : state)
    std::string copy(x);
}
BENCHMARK(BM_StringCopy);

BENCHMARK_MAIN();

Compile and run it:

# c++ -I/usr/local/include -L/usr/local/lib -std=c++11 test.cc -o test -lbenchmark
root:/root/Project# ./test
failed to open /proc/cpuinfo
2018-05-02 17:14:11
Running ./test
Run on (-1 X 2545.25 MHz CPU )
***WARNING*** Library was built as DEBUG. Timings may be affected.
---------------------------------------------------------
Benchmark                  Time           CPU Iterations
---------------------------------------------------------
BM_StringCreation         40 ns         40 ns   17597275
BM_StringCopy             13 ns         13 ns   53511385

“failed to open /proc/cpuinfo“? “-1 X 2545.25 MHz CPU“? Messy output, so I decided to port it on OpenBSD:

(1) The first thing to do is defining BENCHMARK_OS_OPENBSDin src/internal_macros.h:

......
#elif defined(__OpenBSD__)
  #define BENCHMARK_OS_OPENBSD 1
......

(2) The second task should fill the value of CPUInfo‘s members:

struct CPUInfo {
  ......
  int num_cpus;
  double cycles_per_second;
  std::vector<CacheInfo> caches;
  bool scaling_enabled;
  ......
};

Check CPUInfo‘s constructor:

CPUInfo::CPUInfo()
    : num_cpus(GetNumCPUs()),
      cycles_per_second(GetCPUCyclesPerSecond()),
      caches(GetCacheSizes()),
      scaling_enabled(CpuScalingEnabled(num_cpus)) {}

I know I need to implement GetNumCPUs(), GetCPUCyclesPerSecond(), etc. For FreeBSD and NetBSD, benchmark uses sysctlbyname function:

......
if (sysctlbyname(Name.c_str(), nullptr, &CurBuffSize, nullptr, 0) == -1)
    return ValueUnion();
......

Unfortunately, OpenBSD doesn’t support sysctlbyname, so I use sysctl to get CPU’s number and speed:

 if ((Name == "hw.ncpu") || (Name == "hw.cpuspeed")){
    ValueUnion buff(sizeof(int));
    ......
    if (sysctl(mib, 2, buff.data(), &buff.Size, nullptr, 0) == -1) {
      return ValueUnion();
    }
    return buff;
  }

For cache information, the OpenBSD can’t provide ready-made information, and I think it is not worthy to use other work-around method to get it. E.g., use CPUID instruction on X86 architectures (If you really want to know it, lscpu will give you a hand on X86 platform) . Regarding to whether CPU support scaling or not, I can’t find any help about OpenBSD, so just leave it here.

The whole patch is here. Not sure whether google likes to merge it or not (Update: it is already merged), But at least all test cases can pass on OpenBSD now:

# make test

......

100% tests passed, 0 tests failed out of 54

Total Test time (real) =  17.54 sec

And the test program also outputs normal log:

# ./test
2018-05-02 14:49:03
Running ./a.out
Run on (2 X 2534 MHz CPU s)
***WARNING*** Library was built as DEBUG. Timings may be affected.
---------------------------------------------------------
Benchmark                  Time           CPU Iterations
---------------------------------------------------------
BM_StringCreation         42 ns         42 ns   16761725
BM_StringCopy             13 ns         13 ns   51990267

P.S., if you want to use google/benchmark on OpenBSD, you can consider importing my patch. 🙂