Technology | Nan Xiao's Blog

Customize ksh display for OpenBSD

The default shell for OpenBSD is ksh, and it looks a little monotonous:

To make its user-experience more friendly, I need to do some customizations:

(1) Modify the “Prompt String” to display the user name and current directory:

PS1='$USER:$PWD# '

(2) Install colorls package:

# pkg_add colorls

Use it to replace the shipped ls command:

alias ls='colorls -G'

(3) Change LSCOLORS environmental variable to make your favorite color. For example, I don’t want the directory is displayed in default blue, change it to magenta:

LSCOLORS=fxexcxdxbxegedabagacad

For detailed explanation of LSCOLORS, please refer manual of colorls:

# man colorls
......
LSCOLORS        The value of this variable describes what color to use
                     for which attribute when colors are enabled with
                     CLICOLOR.  This string is a concatenation of pairs of the
                     format fb, where f is the foreground color and b is the
                     background color.

                     The color designators are as follows:

                           a     black
                           b     red
......

This is my final modification of .profile:

......
PS1='$USER:$PWD# '
export PS1
LSCOLORS=fxexcxdxbxegedabagacad
export LSCOLORS
alias ls='colorls -G'
......

And this is the final effect:

References:
Add Color to Your Terminal;
Why doesn’t alias work in AIX (Korn shell, .profile)?;
ksh.kshrc.

My first patch to OpenBSD

Several days ago, I come across 2 posts which introduce OpenBSD Hackathon: Historical: My first OpenBSD Hackathon and Michael W. Lucas: Visiting the OpenBSD t2k13 Hackathon. From the reading, I find OpenBSD is a really distinctive and amazing project, especially after browsing its neat code mirror in github, which makes sure my assumption. So I decided to make my hands dirty on it.

Last weekend, I installed OpenBSD on an old machine, and checked out its fresh code from CVS repository. Because I think reading its code can let me get a better understanding about OpenBSD, even Unix philosophy, I decides to take the dmesg as the first experiment. During the reading, the following code gave me a hit unexpectedly:

......
mib[1] = startupmsgs ? KERN_CONSBUFSIZE : KERN_MSGBUFSIZE;
len = sizeof(msgbufsize);
if (sysctl(mib, 2, &msgbufsize, &len, NULL, 0))
    err(1, "sysctl: KERN_MSGBUFSIZE");
......
mib[1] = startupmsgs ? KERN_CONSBUF : KERN_MSGBUF;
len = msgbufsize;
if (sysctl(mib, 2, bufdata, &len, NULL, 0))
    err(1, "sysctl: KERN_MSGBUF");
    ......

The err() function always reports KERN_MSGBUFSIZE error even it is actually KERN_CONSBUFSIZE. So it should be a bug, at least should be an enhancement. I followed Preparing a diff and Making your first patch (OpenBSD) to submit my first OpenBSD patch. Guess what? just few hours later, dmesg source file was changed base on my code. Although the final modification is not my code, it is still a great pleasure that I contribute my own effort to help make OpenBSD better!

Use pdb to help understand python program

As I have mentioned in Why do I need a debugger?:

(3) Debugger is a good tool to help you understand code.

So when I come across difficulty to understand vfscount.py code in bcc project, I know it is time to resort to pdb, python‘s debugger, to help me.

The thing which confuses me is here:

counts = b.get_table("counts")
for k, v in sorted(counts.items(), key=lambda counts: counts[1].value):
    print("%-16x %-26s %8d" % (k.ip, b.ksym(k.ip), v.value))

From previous code:

BPF_HASH(counts, struct key_t, u64, 256);

It seems the v‘s type is u64, and I can’t figure out why use v.value to fetch its data here.

pdb‘s manual is very succinct and its command is so similar with gdb‘s, so it is no learning curve for me. Just launch a debugging session and set breakpoint at “counts = b.get_table("counts")” line:

# python -m pdb vfscount.py
> /root/Project/bcc/tools/vfscount.py(14)<module>()
-> from __future__ import print_function
(Pdb) b vfscount.py:49

Start the program and press Ctrl-C after seconds; the breakpoint will be hit:

(Pdb) r
Tracing... Ctrl-C to end.
^C
ADDR             FUNC                          COUNT
> /root/Project/bcc/tools/vfscount.py(49)<module>()
-> counts = b.get_table("counts")

Step into get_table method, and single-step every line. Before leaving method, check the type of keytype and leaftype:

-> counts = b.get_table("counts")
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/bcc/__init__.py(416)get_table()
-> def get_table(self, name, keytype=None, leaftype=None, reducer=None):
(Pdb) n
> /usr/lib/python3.6/site-packages/bcc/__init__.py(417)get_table()
-> map_id = lib.bpf_table_id(self.module, name.encode("ascii"))
......
(Pdb) p leaf_desc
b'"unsigned long long"'
(Pdb) n
> /usr/lib/python3.6/site-packages/bcc/__init__.py(430)get_table()
-> leaftype = BPF._decode_table_type(json.loads(leaf_desc.decode()))
(Pdb)
> /usr/lib/python3.6/site-packages/bcc/__init__.py(431)get_table()
-> return Table(self, map_id, map_fd, keytype, leaftype, reducer=reducer)
(Pdb) p leaftype
<class 'ctypes.c_ulong'>
(Pdb) p keytype
<class 'bcc.key_t'>

Yeah! The magic is here: leaftype‘s type is not pure u64, but ctypes.c_ulong. According to document:

>>> print(i.value)
42

We should use v.value to get its internal data.

Happy pdbing! Happy python debugging!

The timezone issue of installing OpenBSD in VirtualBox

I tried to use OpenBSD in VirtualBox. During installation, it prompted me:

Time appears wrong. Set to ‘Sat Aug 19 11:56:42 +08 2017’? [yes]

Since my server is UTC+8 timezone, I select “yes”. But after rebooting, the date command showed wrong info:

#date
Sat Aug 19 20:01:00 +8 2017
#date
Sat Aug 19 12:01:05 UTC 2017

Actually, my current host time should be 12:01:05 UTC+8. The VirtualBox seemed consider the host time as UTC time, and added another 8 hours. After discussing in the mailing list, the correct answer was found. I should tick “Hardware clock in UTC time” in setting:

According the manual:

If checked, VirtualBox will report the system time in UTC format to the guest instead of local (host) time. This affects how the virtual real-time clock (RTC) operates and may be useful for Unix-like guest operating systems, which typically expect the hardware clock to be set to UTC.

OpenBSD should consider time reported by VirtualBox as UTC time. So if this option is not checked, the VirtualBox will report local time to OpenBSD, OpenBSD misunderstands it as UTC time and add additional 8 hour to local time. This can explain what I have seen.

Reference:
Set date during OpenBSD installation.

Include “stdio.h” before Readline library header files

I install Readline library and write a simple program to play with it:

#include <readline/readline.h>
#include <readline/history.h>

int main(void)
{
    char *line_read = readline (">>  ");
    if (line_read && *line_read)
    {
        add_history (line_read);
    }
    ......
}

Build it will generate following errors:

In file included from /usr/include/readline/readline.h:35:0,
                 from readline.c:3:
/usr/include/readline/rltypedefs.h:71:28: error: unknown type name 'FILE'
 typedef int rl_getc_func_t PARAMS((FILE *));
                            ^
/usr/include/readline/readline.h:429:20: error: unknown type name 'FILE'
 extern int rl_getc PARAMS((FILE *));
                    ^
In file included from readline.c:3:0:
/usr/include/readline/readline.h:558:8: error: unknown type name 'FILE'
 extern FILE *rl_instream;
        ^~~~
/usr/include/readline/readline.h:559:8: error: unknown type name 'FILE'
 extern FILE *rl_outstream;
        ^~~~
/usr/include/readline/readline.h:588:8: error: unknown type name 'rl_getc_func_t'
 extern rl_getc_func_t *rl_getc_function;
        ^~~~~~~~~~~~~~
/usr/include/readline/readline.h:917:3: error: unknown type name 'FILE'
   FILE *inf;
   ^~~~
/usr/include/readline/readline.h:918:3: error: unknown type name 'FILE'
   FILE *outf;

The solution is including <stdio.h> before Readline library’s header files:

#include <stdio.h>
#include <readline/readline.h>
#include <readline/history.h>
......