Tag Archives: unix

The anatomy of ldd program on OpenBSD

Published / by nanxiao / 1 Comment on The anatomy of ldd program on OpenBSD

In the past week, I read the ldd source code on OpenBSD to get a better understanding of how it works. And this post should also be a reference for other*NIX OSs.

The ELF file is divided into 4 categories: relocatable, executable, shared, and core. Only the executable and shared object files may have dynamic object dependencies, so the ldd only check these 2 kinds of ELF file:

(1) Executable.

ldd leverages the LD_TRACE_LOADED_OBJECTS environment variable in fact, and the code is as following:

if (setenv("LD_TRACE_LOADED_OBJECTS", "true", 1) < 0)
    err(1, "setenv(LD_TRACE_LOADED_OBJECTS)");

When LD_TRACE_LOADED_OBJECTS is set to 1 or true, running executable file will show shared objects needed instead of running it, so you even not needldd to check executable file. See the following outputs:

# /usr/bin/ldd
usage: ldd program ...
# LD_TRACE_LOADED_OBJECTS=1 /usr/bin/ldd
        Start            End              Type Open Ref GrpRef Name
        00000b6ac6e00000 00000b6ac7003000 exe  1    0   0      /usr/bin/ldd
        00000b6dbc96c000 00000b6dbcc38000 rlib 0    1   0      /usr/lib/libc.so.89.3
        00000b6d6ad00000 00000b6d6ad00000 rtld 0    1   0      /usr/libexec/ld.so  

(2) Shared object.

The code to print dependencies of shared object is as following:

if (ehdr.e_type == ET_DYN && !interp) {
    if (realpath(name, buf) == NULL) {
        printf("realpath(%s): %s", name,
    dlhandle = dlopen(buf, RTLD_TRACE);
    if (dlhandle == NULL) {
        printf("%s\n", dlerror());

Why the condition of checking a ELF file is shared object or not is like this:

if (ehdr.e_type == ET_DYN && !interp) {

That’s because the file type of position-independent executable (PIE) is the same as shared object, but normally PIE contains a interpreter program header since it needs dynamic linker to load it while shared object lacks (refer this article). So the above condition will filter PIE file.

The dlopen(buf, RTLD_TRACE) is used to print dynamic object information. And the actual code is like this:

if (_dl_traceld) {

In fact, you can also implement a simple application which outputs dynamic object information for shared object yourself:

#include <dlfcn.h>

int main(int argc, char **argv)
    dlopen(argv[1], RTLD_TRACE);
    return 0;

Compile and use it to analyze /usr/lib/libssl.so.43.2:

# cc lddshared.c
# ./a.out /usr/lib/libssl.so.43.2
    Start            End              Type Open Ref GrpRef Name
    000010e2df1c5000 000010e2df41a000 dlib 1    0   0      /usr/lib/libssl.so.43.2
    000010e311e3f000 000010e312209000 rlib 0    1   0      /usr/lib/libcrypto.so.41.1

The same as using ldd directly:

# ldd /usr/lib/libssl.so.43.2
    Start            End              Type Open Ref GrpRef Name
    00001d9ffef08000 00001d9fff15d000 dlib 1    0   0      /usr/lib/libssl.so.43.2
    00001d9ff1431000 00001d9ff17fb000 rlib 0    1   0      /usr/lib/libcrypto.so.41.1

Through the studying of ldd source code, I also get many by-products: such as knowledge of ELF file, linking and loading, etc. So diving into code is a really good method to learn *NIX deeper!

The basics of Client/Server socket programming

Published / by nanxiao / Leave a Comment

While Client/Server communication model is ubiquitous nowadays, most of them involve socket programming knowledge. In this post, I will introduce some rudimentary aspects of it:

(1) Short/Long-lived TCP connection.
Short-lived TCP connection refers to following pattern: Client creates a connection to server; send message, then close the connection. If Client wants to transmit information again, repeat the above steps. Because establishing and destroying TCP sessions have overhead, if Client needs to have transactions with Server frequently, long-lived connection may be a better choice: connect Server; deliver message, deliver message, …, disconnect Server. A caveat about long-lived connection is Client may need to send heartbeat message to Server to keep the TCP session active.

(2) Synchronous/Asynchronous communication.
After Client sends the request, it can block here to wait for Server’s response, this is called synchronous mode. Certainly the Client should set a timer in case the response never come. The Client can also choose not to block, and continue to do other things. On the contrary, this is called asynchronous mode. If the Client can send multiple requests before receiving responses, it needs to add ID for every message, so it can distinguish the corresponding response for every request.

(3) Error handling.
A big pain point of socket programming is you need to consider so many exceptional cases. For example, the Server suddenly crashes; the network cable is plugged out, or the response message is half-received, etc. So your code should process as many abnormalities as possible. It is no exaggeration to say that error-handling code quality is the cornerstone of program’s robustness.

(4) Portability.
Different *NIX flavors may have small divergences on socket programming, so the program works well on Linux may not guarantee it also run as you expect on FreeBSD. BTW, I summarized a post about tips of Solaris/illumos socket programming before, and you can read it if you happen to work on these platforms.

(5) Leverage sniffer tools.
Tcpdump/Wireshark/snoop are amazing tools for debugging network programming, and they can tell you what really happens under the hood. Try to be sophisticated at these tools, and they will save you at one day, trust me!


Use clang to build OpenBSD on amd64/i386

Published / by nanxiao / Leave a Comment

I install the newest OpenBSD 6.1, and try to build -curr kernel. But unfortunately the make reports following errors:

# make
cat /usr/src/sys/arch/amd64/amd64/genassym.cf /usr/src/sys/arch/amd64/amd64/genassym.cf |  sh /usr/src/sys/kern/genassym.sh cc -no-integrated-as -g -Werror -Wall -Wimplicit-function-declaration  -Wno-uninitialized -Wno-pointer-sign  -Wno-address-of-packed-member -Wno-constant-conversion  -Wframe-larger-than=2047 -mcmodel=kernel -mno-red-zone -mno-sse2 -mno-sse -mno-3dnow  -mno-mmx -msoft-float -fno-omit-frame-pointer -ffreestanding -fno-pie -O2 -pipe -nostdinc -I/usr/src/sys -I/usr/src/sys/arch/amd64/compile/GENERIC.MP/obj -I/usr/src/sys/arch -DDDB -DDIAGNOSTIC -DKTRACE -DACCOUNTING -DKMEMSTATS -DPTRACE -DPOOL_DEBUG -DCRYPTO -DSYSVMSG -DSYSVSEM -DSYSVSHM -DUVM_SWAP_ENCRYPT -DFFS -DFFS2 -DFFS_SOFTUPDATES -DUFS_DIRHASH -DQUOTA -DEXT2FS -DMFS -DNFSCLIENT -DNFSSERVER -DCD9660 -DUDF -DMSDOSFS -DFIFO -DFUSE -DSOCKET_SPLICE -DTCP_SACK -DTCP_ECN -DTCP_SIGNATURE -DINET6 -DIPSEC -DPPP_BSDCOMP -DPPP_DEFLATE -DPIPEX -DMROUTING -DMPLS -DBOOT_CONFIG -DUSER_PCICONF -DAPERTURE -DMTRR -DNTFS -DHIBERNATE -DPCIVERBOSE -DUSBVERBOSE -DWSDISPLAY_COMPAT_USL -DWSDISPLAY_COMPAT_RAWKBD -DWSDISPLAY_DEFAULTSCREENS="6" -DX86EMU -DONEWIREVERBOSE -DMULTIPROCESSOR -DMAXUSERS=80 -D_KERNEL -MD -MP -MF assym.P > assym.h.tmp
cc: unrecognized option '-no-integrated-as'
cc1: error: unrecognized command line option "-Wno-address-of-packed-member"
cc1: error: unrecognized command line option "-Wno-constant-conversion"
*** Error 1 in /usr/src/sys/arch/amd64/compile/GENERIC.MP (Makefile:938 'assym.h')

From this mail, I learn that clang has been the default compiler on amd64/i386 platforms for OpenBSD, so I switch to use clang to build kernel:

# CC=clang make

Now it can compile!

Create a symbol link for python after installing it on OpenBSD

Published / by nanxiao / Leave a Comment

After installing Python on FreeBSD:

# pkg_add python
quirks-2.304 signed on 2017-04-02T15:01:33Z
Ambiguous: choose package for python
a       0: <None>
        1: python-2.7.13p0
        2: python-3.4.5p2
        3: python-3.5.2p2
        4: python-3.6.0p0
Your choice: 4

It won’t create a symbol link by default:

# python
ksh: python: not found

So for using it handily, you can create a symbol link yourself:

# cd /usr/local/bin
# ln -s python3.6 python

Use Source Insight as the editor to develop Unix softwares

Published / by nanxiao / Leave a Comment

Source Insight is my favorite editor, and I have used it for more than 10 years. But when employing it to develop Unix software, you will run into annoying line break issue, which is on windows, the newline is \r\n while in Unix it is \n only. Therefore you will see the file edited in Source Insight will display an extra ^M in Unix environment:

#include <stdio.h>^M
int main(void)^M

To resolve this problem, you can refer this topic in stackoverflow:

To save a file with a specific end-of-line type in Source Insight, select File -> Save As…, then where it says “Save as type”, select the desired end-of-line type.

To set the end-of-line type for new files you create in Source Insight, select Options -> Preferences and click the Files tab. Where it says “Default file format” select the desired end-of-line type.

So you can set Unix file format as you wanted:


Another caveat you should pay attention is if you use git Windows client, by default, it will convert the newline of project from \n to \r\n directly. My solution is just disabling this auto conversion feature:

git config --global core.autocrlf false