Tag: multi-thread

The caveat of thread name length in glibc

Recently, our team met an interesting bug: the process is configured to spawn 16 threads, but only spawns 10 threads in reality. The thread code is like this:

static void *
stat_consumer_thread_run(void *data)
{
    stat_consumer_thread_t *thread = data;
    char thread_name[64];
    snprintf(thread_name, sizeof(thread_name), "stat.consumer.%d",
        thread->id);
    int rc = pthread_setname_np(pthread_self(), thread_name);
    if (rc != 0) {
        return NULL;
    }

    ......
    return NULL;
}

After checking pthread_setname_np manual, we found:

The thread name is a meaningful C language string, whose length is restricted to 16 characters, including the terminating null byte (’\0’).

So thread name is restricted to 16 characters, “stat.consumer.0” ~ “stat.consumer.9” are set successfully, but “stat.consumer.10” ~ “stat.consumer.15” are not, and the corresponding threads are failed to run.

Exit main thread and keep other threads running

In C programming, if using return in main function, the whole process will terminate. To only let main thread gone, and keep other threads live, you can use thrd_exit in main function. Check following code:

#include <stdio.h>
#include <threads.h>
#include <unistd.h>

int
print_thread(void *s)
{
    thrd_detach(thrd_current());
    for (size_t i = 0; i < 5; i++)
    {
        sleep(1);
        printf("i=%zu\n", i);
    }
    thrd_exit(0);
}

int
main(void)
{
    thrd_t tid;
    if (thrd_success != thrd_create(&tid, print_thread, NULL)) {
        fprintf(stderr, "Create thread error\n");
        return 1;
    }
    thrd_exit(0);
}

Run it:

$ ./main
i=0
i=1
i=2
i=3
i=4

You can see even main thread exited, the other thread still worked.

P.S., the code can be downloaded here.

First taste of C thread APIs

Since C11, C provides standard thread APIs like what C++ does. It means technically, you should use C‘s standard thread APIs to do multi-thread stuff, not pthread APIs. Below is a simple example:

#include <threads.h>
#include <stdio.h>

int print_thread(void* s)
{
    printf("%s\n", (char*)s);
    thrd_exit(0);
}
int main()
{
    thrd_t tid;
    if (thrd_success != thrd_create(&tid, print_thread, "Hello world"))
    {
        fprintf(stderr, "Create thread error\n");
        return 1;
    }
    thrd_join(tid, NULL);
    return 0;
}

Check thrd_create implementation in glibc:

#include "thrd_priv.h"

int
thrd_create (thrd_t *thr, thrd_start_t func, void *arg)
{
  _Static_assert (sizeof (thr) == sizeof (pthread_t),
          "sizeof (thr) != sizeof (pthread_t)");

  int err_code = __pthread_create_2_1 (thr, ATTR_C11_THREAD,
                       (void* (*) (void*))func, arg);
  return thrd_err_map (err_code);
}

You can see thrd_create just encapsulates __pthread_create_2_1, so you can guess in glibc, the standard C thread APIs are just wrappers of pthreadimplementation. It means you need to link pthread library during compiling. Otherwise you will meet following errors:

main.c:(.text+0x1e): undefined reference to `thrd_exit'
......
main.c:(.text+0x4f): undefined reference to `thrd_create'
/usr/bin/ld: main.c:(.text+0x60): undefined reference to `thrd_join'

P.S., the full code is here.

Explore Dragonfly BSD thread model

From Dragonfly BSD‘s official document:

A user process contains one or more LWP (Light Weight Process) entities. Each entity represents a user thread under that process.

I want to explore the thread model myself. So I write a simple program which launches 2 threads in main function, and print process ID, LWP ID and thread id from C++‘s get_id function:

void output_thread_id(const std::string& prefix)
{
    std::stringstream ss;
    ss <<"Process ID is " << getpid() <<
        ", lwp ID is " << lwp_gettid() <<
        ", " << prefix << std::this_thread::get_id() <<'\n';
    std::cout << ss.str();
}

Build and run it:

# ./spawn_threads
Process ID is 16394, lwp ID is 1, main thread ID is 0x8007d00c0
Process ID is 16394, lwp ID is 2, sub thread ID is 0x8007d0240
Process ID is 16394, lwp ID is 3, sub thread ID is 0x8007d03c0

All 3 threads have same process ID, but LWP ID begins with 1, and increases contiguously (this is not same as Linux). By default, top command will only show processes:

Press ‘H‘ can display threads’ information (For ps command, -H option can be used to show threads):

P.S., the full code is here.

Reference:
How to get the thread number and every thread’s ID of a running process? .

 

Conditional variable takeaways

Conditional variable is a common concept in both user-space and kernel-space programming. IMHO, it is a little complicated synchronization mechanism. Recently, I came across Measuring context switching and memory overheads for Linux threads, and this article provides an example which I think is a good tutorial about how to understand and use conditional variable.

The parent thread code is like following:

  /* parent thread */
  pthread_mutex_lock(&si.mutex);
  pthread_t childt;
  pthread_create(&childt, NULL, threadfunc, (void*)&si);

  // Each iteration of this loop will switch context from the parent to the
  // child and back - two context switches. The child signals first.
  ......
  for (int i = 0; i < NUM_ITERATIONS; ++i) {
    pthread_cond_wait(&si.cv, &si.mutex);
    pthread_cond_signal(&si.cv);
  }
  pthread_mutex_unlock(&si.mutex);

And this is the child thread code:

// The child thread signals first
  pthread_mutex_lock(&si->mutex);
  pthread_cond_signal(&si->cv);
  for (int i = 0; i < NUM_ITERATIONS; ++i) {
    pthread_cond_wait(&si->cv, &si->mutex);
    pthread_cond_signal(&si->cv);
  }
  pthread_mutex_unlock(&si->mutex);

(1) The first takeaway is pthread_cond_signal() must be called after pthread_cond_wait() in timing sequence; otherwise the signal won’t be received.

Check preceding code, before launching child thread:

    ......
    pthread_t childt;
    pthread_create(&childt, NULL, threadfunc, (void*)&si);
    ......

The parent thread must hold mutex first:

    ......
    pthread_mutex_lock(&si.mutex);
    ......

Then in the first iteration of loop, release the mutex and wait for notification:

    ......
    pthread_cond_wait(&si.cv, &si.mutex);
    ......

This can guarantee when child thread sends signal, the parent thread is already in the wait queue:

  ......
  pthread_mutex_lock(&si->mutex);
  pthread_cond_signal(&si->cv);
  ......

(2) The other thing we should remember is before and after calling pthread_cond_wait(), the current thread must hold the mutex. I.e., before callingpthread_cond_wait(), the current thread get the mutex, then in pthread_cond_wait(), put the current thread in the wait queue and release the mutexatomically. Once another thread signals current thread, it will reacquire mutex and return from pthread_cond_wait().