Nan Xiao's Blog

Be aware of blocking IO APIs

Yesterday, I did performance analysis for one project. This is the output of mpstat command for old version:

And this is the CPU utilisation for new version:

For new version, the iowait ratio is remarkably high. After Checking the code, I found the original serialisation was just a fflush, but now for some reasons, it was replaced by fdatasync which is a blocking API and only returns when the data is transferred to the storage device. Therefore the thread which invokes fdatasync will be stuck there and can’t process any other message. So we must pay attention to use blocking IO APIs, sometimes they may bring you results which you don’t want.

Exit main thread and keep other threads running

In C programming, if using return in main function, the whole process will terminate. To only let main thread gone, and keep other threads live, you can use thrd_exit in main function. Check following code:

#include <stdio.h>
#include <threads.h>
#include <unistd.h>

int
print_thread(void *s)
{
    thrd_detach(thrd_current());
    for (size_t i = 0; i < 5; i++)
    {
        sleep(1);
        printf("i=%zu\n", i);
    }
    thrd_exit(0);
}

int
main(void)
{
    thrd_t tid;
    if (thrd_success != thrd_create(&tid, print_thread, NULL)) {
        fprintf(stderr, "Create thread error\n");
        return 1;
    }
    thrd_exit(0);
}

Run it:

$ ./main
i=0
i=1
i=2
i=3
i=4

You can see even main thread exited, the other thread still worked.

P.S., the code can be downloaded here.

Add timestamp for pcap file’s name

I wrote a post about splitting large pcap file into small ones before. After that, you should add timestamp for pcap‘s file name, and it will be easy for you to find related pcap files to process.

Assume there is a folder which includes all pcap files generated by following tcpdump command:

tcpdump -r input.pcap -w ./pcap/frag -C 1000

It will generate ./pcap/frag, ./pcap/frag1, …, etc. You can use following script to add timestamp for every file:

#!/bin/sh

directory=./pcap
cd "$directory" || exit 1

for old_file_name in *
do
    timestamp=$(tshark -nr "${old_file_name}" -T fields -e frame.time_epoch -c 1)
    new_file_name="${old_file_name}.${timestamp}.pcap"
    mv "${old_file_name}" "${new_file_name}"
done

The file’s name will be fragxx.1542222065.974954000.pcap now.

P.S., the script can be downloaded here.

The c99 program on Void Linux

Today I found there is a c99 program in Void Linux:

$ c99
cc: fatal error: no input files
compilation terminated.
$ which c99
/usr/bin/c99

Check what it is:

$ file /usr/bin/c99
/usr/bin/c99: POSIX shell script, ASCII text executable
$ cat /usr/bin/c99
#!/bin/sh
exec /usr/bin/cc -std=c99 "$@"

Um, just a shell script which invokes /usr/bin/cc. So check cc program:

$ ll /usr/bin/cc
lrwxrwxrwx 1 root root 3 Jun  9 05:32 /usr/bin/cc -> gcc

Oh, a link to gcc.

AddressSanitizer’s no_sanitize_address attribute

AddressSanitizer has a no_sanitize_address attribute which can be used to turn off instrumentation. Check following code:

$ cat ../main.c
#include <stdlib.h>
#include <sanitizer/asan_interface.h>

#define ARRAY_SIZE 4

int *array;

void
foo()
{
    array[0] = 1;
}

int
main()
{
    array = malloc(sizeof(int) * ARRAY_SIZE);
    ASAN_POISON_MEMORY_REGION(array, sizeof(int) * ARRAY_SIZE);
    foo();
}

Build and run this program:

$ ./main
=================================================================
==1558==ERROR: AddressSanitizer: use-after-poison on address 0x602000000010 at pc 0x55839733c1b7 bp 0x7ffc102fb320 sp 0x7ffc102fb310
WRITE of size 4 at 0x602000000010 thread T0
    #0 0x55839733c1b6 in foo (/home/nan/code-for-my-blog/2020/05/asan_no_sanitize_address/build/main+0x11b6)
    #1 0x55839733c1f2 in main (/home/nan/code-for-my-blog/2020/05/asan_no_sanitize_address/build/main+0x11f2)
    #2 0x7fe4d79d9dea in __libc_start_main ../csu/libc-start.c:308
    #3 0x55839733c0b9 in _start (/home/nan/code-for-my-blog/2020/05/asan_no_sanitize_address/build/main+0x10b9)

0x602000000010 is located 0 bytes inside of 16-byte region [0x602000000010,0x602000000020)
allocated by thread T0 here:
    #0 0x7fe4d7c824c8 in __interceptor_malloc (/usr/lib/libasan.so.5+0x10c4c8)
    #1 0x55839733c1cd in main (/home/nan/code-for-my-blog/2020/05/asan_no_sanitize_address/build/main+0x11cd)
    #2 0x7fe4d79d9dea in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: use-after-poison (/home/nan/code-for-my-blog/2020/05/asan_no_sanitize_address/build/main+0x11b6) in foo
Shadow bytes around the buggy address:
  0x0c047fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c047fff8000: fa fa[f7]f7 fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1558==ABORTING

Unsurprisingly, AddressSanitizer reports error because the array is poisoned but foo() is trying to modify its first element. Add __attribute__((no_sanitize_address)) for foo():

__attribute__((no_sanitize_address))
void
foo()
{
    array[0] = 1;
}

Run the program again:

$ ./main
$

This time program exits normally.

P.S., the code can be downloaded here.