Nan Xiao's Blog

A system software / performance engineer's home

Month: July 2017 (Page 1 of 2)

Parse BPF_ARRAY macro in bcc

BPF_ARRAY is a very common macro used in bcc scripts. To analyze it, I put all BPF_ARRAY related macros in an example file:

// Changes to the macro require changes in BFrontendAction classes
#define BPF_F_TABLE(_table_type, _key_type, _leaf_type, _name, _max_entries, _flags) \
struct _name##_table_t { \
  _key_type key; \
  _leaf_type leaf; \
  _leaf_type * (*lookup) (_key_type *); \
  _leaf_type * (*lookup_or_init) (_key_type *, _leaf_type *); \
  int (*update) (_key_type *, _leaf_type *); \
  int (*insert) (_key_type *, _leaf_type *); \
  int (*delete) (_key_type *); \
  void (*call) (void *, int index); \
  void (*increment) (_key_type); \
  int (*get_stackid) (void *, u64); \
  _leaf_type data[_max_entries]; \
  int flags; \
}; \
__attribute__((section("maps/" _table_type))) \
struct _name##_table_t _name = { .flags = (_flags) }

#define BPF_TABLE(_table_type, _key_type, _leaf_type, _name, _max_entries) \
BPF_F_TABLE(_table_type, _key_type, _leaf_type, _name, _max_entries, 0);

#define BPF_ARRAY1(_name) \
  BPF_TABLE("array", int, u64, _name, 10240)
#define BPF_ARRAY2(_name, _leaf_type) \
  BPF_TABLE("array", int, _leaf_type, _name, 10240)
#define BPF_ARRAY3(_name, _leaf_type, _size) \
  BPF_TABLE("array", int, _leaf_type, _name, _size)

// helper for default-variable macro function
#define BPF_ARRAYX(_1, _2, _3, NAME, ...) NAME

// Define an array function, some arguments optional
// BPF_ARRAY(name, leaf_type=u64, size=10240)
#define BPF_ARRAY(...) \

enum stat_types {
  S_COUNT = 1,

void main(void)
  BPF_ARRAY(stats, u64, S_MAXSTAT + 1); 

Use gcc -E to preprocess it (I have formatted code to make it clear):

# 1 "test.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "test.c"
# 38 "test.c"
enum stat_types {
    S_COUNT = 1,

void main(void)
    struct stats_table_t { 
      int key; 
      u64 leaf; 
      u64 * (*lookup) (int *); 
      u64 * (*lookup_or_init) (int *, u64 *); 
      int (*update) (int *, u64 *); 
      int (*insert) (int *, u64 *); 
      int (*delete) (int *);
      void (*call) (void *, int index); 
      void (*increment) (int); 
      int (*get_stackid) (void *, u64); 
      u64 data[S_MAXSTAT + 1]; 
      int flags; 
    __attribute__((section("maps/" "array"))) struct stats_table_t stats = { .flags = (0) };;

Let’s analyze how the final result is got:


BPF_ARRAY(stats, u64, S_MAXSTAT + 1)    

will be expanded to (__VA_ARGS__ will be replaced by all arguments):

BPF_ARRAYX(stats, u64, S_MAXSTAT + 1, BPF_ARRAY3, BPF_ARRAY2, BPF_ARRAY1)(stats, u64, S_MAXSTAT + 1)

(2) According to:

// helper for default-variable macro function
#define BPF_ARRAYX(_1, _2, _3, NAME, ...) NAME

BPF_ARRAYX(stats, u64, S_MAXSTAT + 1, BPF_ARRAY3, BPF_ARRAY2, BPF_ARRAY1)(stats, u64, S_MAXSTAT + 1) will be replaced by following code:

BPF_ARRAY3(stats, u64, S_MAXSTAT + 1)

We can compared it to BPF_ARRAY1 (both _leaf_type and size have default values) and BPF_ARRAY2 (only size have default value.).

(3) Convert BPF_ARRAY3 to final BPF_F_TABLE is more straightforward, so I won’t drill that down.

Use the same method you can analyze other macros, such as BPF_HASH. Hope this small post can give you a tip!

Install package from Arch Linux user repository

To install package from Arch Linux user repository, you should use a common user account, not a root. Take setting up trace-cmd as an example:

$ git clone
$ cd trace-cmd
$ makepkg -si

gcc’s enable “–enable-default-pie” option make you stuck at “relocation R_X86_64_32S against …” error

Recently, after I upgrade gcc on my Arch Linux, I find it has enabled “--enable-default-pie” option by default:

$ gcc -v
Using built-in specs.
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl= --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --disable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.1.1 20170630 (GCC)

One consequence of this enhancement is you should rebuild the static libraries which you projects depend on, otherwise you may counter totally confused link errors like this:

relocation R_X86_64_32S against `.text’ can not be used when making a shared object; recompile with -fPIC

A caveat you must pay attention to is if your static library has assembly code object which is not position independent, you must specify “-no-pie” option during link stage of generating final executable binary. This issue let me spend half day to debug, so it is a really good habit to check critical packages’ change log, such as your compiler.


You should publish your contributions of Open Source, even it’s not required

Although many Open Source projects don’t require you publish the modifications of them, I still propose you should make the changes open. The reasons are as following:

(1) For other people: the out-of-box Open Source projects save your time and money, and you get benefit from them. You should not only take, but also need to give! Only if everyone shares his code, document, or whatever related, the Open Source projects can grow up healthy.

(2) For yourself: You release your code can actually make you “richer”. With more and more people use your code, more and more bugs will be found, and associated patches would also be submitted simultaneously. In fact, you can leverage the whole community to help test your code and it saves your bucks which could be used to employ QA/testing engineers. The other invisible gain is the contributions can boost both you and your employer’s reputation, and it will definitely be a big fortune in your future life!

Based on the above points, don’t be hesitate to share your changes. You just need to know give more, gain more!

The subtlety of building ANTLR C runtime library

If you want to use ANTLR C runtime library, you should pay attention that it will be compiled to 32-bit library by default even in 64-bit platform:

# ./configure
# make
make  all-am
make[1]: Entering directory '/root/Project/libantlr3c-3.4'
/bin/sh ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -Iinclude    -m32  -O2  -Wall -MT antlr3baserecognizer.lo -MD -MP -MF .deps/antlr3baserecognizer.Tpo -c -o antlr3baserecognizer.lo `test -f 'src/antlr3baserecognizer.c' || echo './'`src/antlr3baserecognizer.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -Iinclude -m32 -O2 -Wall -MT antlr3baserecognizer.lo -MD -MP -MF .deps/antlr3baserecognizer.Tpo -c src/antlr3baserecognizer.c  -fPIC -DPIC -o .libs/antlr3baserecognizer.o
In file included from /usr/include/features.h:434:0,
                 from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/stdio.h:28,
                 from include/antlr3defs.h:248,
                 from include/antlr3baserecognizer.h:39,
                 from src/antlr3baserecognizer.c:9:
/usr/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
 # include <gnu/stubs-32.h>
compilation terminated.
make[1]: *** [Makefile:449: antlr3baserecognizer.lo] Error 1
make[1]: Leaving directory '/root/Project/libantlr3c-3.4'
make: *** [Makefile:308: all] Error 2

In order to build 64-bit library, you should specify --enable-64bit option during configure stage:

./configure --enable-64bit

Then it will be generated to 64-bit library.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén