Linux kernel 笔记 (53)——为什么“interrupt handler”不能被抢占?

Interrupt handler会复用当前被中断taskkernel stack,它并不是一个真正的task,也不拥有task_struct。因此一旦被调度出去,就无法再被调度回来继续执行。所以interrupt handler不允许被抢占。

参考资料:
Why can’t you sleep in an interrupt handler in the Linux kernel? Is this true of all OS kernels?
Why kernel code/thread executing in interrupt context cannot sleep?;
Are there any difference between “kernel preemption” and “interrupt”?;
Why can not processes switch in atomic context?

 

Linux kernel 笔记 (52)——使用“spinlock”的进程不能被抢占

以下摘自这封邮件

A process cannot be preempted nor sleep while holding a spinlock due spinlocks behavior. If a process grabs a spinlock and goes to sleep before releasing it. A second process (or an interrupt handler) that to grab the spinlock will busy wait. On an uniprocessor machine the second process will lock the CPU not allowing the first process to wake up and release the spinlock so the second process can continue, it is basically a deadlock.

This happens since grabbing an spinlocks also disables interrupts and this is required to synchronize threads with interrupt handlers.

当一个task获得spinlock以后,它不能被抢占(比如调用sleep)。因为如果这时有另外一个task也想获得这个spinlock,在UP系统上,这个task就会一直占据CPU,并且不停地尝试获得锁。而第一个task没有机会重新执行来释放锁,这就造成“死锁”。Interrupt handler也是同样道理。

 

Linux kernel 笔记 (51)——”atomic context”

以下摘自这篇文章

Kernel code generally runs in one of two fundamental contexts. Process context reigns when the kernel is running directly on behalf of a (usually) user-space process; the code which implements system calls is one example. When the kernel is running in process context, it is allowed to go to sleep if necessary. But when the kernel is running in atomic context, things like sleeping are not allowed. Code which handles hardware and software interrupts is one obvious example of atomic context.

There is more to it than that, though: any kernel function moves into atomic context the moment it acquires a spinlock. Given the way spinlocks are implemented, going to sleep while holding one would be a fatal error; if some other kernel function tried to acquire the same lock, the system would almost certainly deadlock forever.

“Deadlocking forever” tends not to appear on users’ wishlists for the kernel, so the kernel developers go out of their way to avoid that situation. To that end, code which is running in atomic context carefully follows a number of rules, including (1) no access to user space, and, crucially, (2) no sleeping. Problems can result, though, when a particular kernel function does not know which context it might be invoked in. The classic example is kmalloc() and friends, which take an explicit argument (GFPKERNEL or GFPATOMIC) specifying whether sleeping is possible or not.

处理中断代码属于atomic context,必须遵守下面的原则:
a)不能访问user space
b)不能sleep

 

Linux kernel 笔记 (50)——”context switch”和”mode switch”

以下内容摘自stackoverflow上的这个帖子

At a high level, there are two separate mechanisms to understand. The first is the kernel entry/exit mechanism: this switches a single running thread from running usermode code to running kernel code in the context of that thread, and back again. The second is the context switch mechanism itself, which switches in kernel mode from running in the context of one thread to another.

So, when Thread A calls sched_yield() and is replaced by Thread B, what happens is:

Thread A enters the kernel, changing from user mode to kernel mode;
Thread A in the kernel context-switches to Thread B in the kernel;
Thread B exits the kernel, changing from kernel mode back to user mode.

Each user thread has both a user-mode stack and a kernel-mode stack. When a thread enters the kernel, the current value of the user-mode stack (SS:ESP) and instruction pointer (CS:EIP) are saved to the thread’s kernel-mode stack, and the CPU switches to the kernel-mode stack – with the int $80 syscall mechanism, this is done by the CPU itself. The remaining register values and flags are then also saved to the kernel stack.

When a thread returns from the kernel to user-mode, the register values and flags are popped from the kernel-mode stack, then the user-mode stack and instruction pointer values are restored from the saved values on the kernel-mode stack.

When a thread context-switches, it calls into the scheduler (the scheduler does not run as a separate thread – it always runs in the context of the current thread). The scheduler code selects a process to run next, and calls the switchto() function. This function essentially just switches the kernel stacks – it saves the current value of the stack pointer into the TCB for the current thread (called struct taskstruct in Linux), and loads a previously-saved stack pointer from the TCB for the next thread. At this point it also saves and restores some other thread state that isn’t usually used by the kernel – things like floating point/SSE registers.

So you can see that the core user-mode state of a thread isn’t saved and restored at context-switch time – it’s saved and restored to the thread’s kernel stack when you enter and leave the kernel. The context-switch code doesn’t have to worry about clobbering the user-mode register values – those are already safely saved away in the kernel stack by that point.

总结如下:
mode switch”是一个运行的taskuser-mode切换到kernel-mode,或者切换回来。而“context switch”一定发生在kernel mode,进行task的切换。

每个user task有一个user-mode stack和一个kernel-mode stack,当从user-mode切换到kernel-mode时,寄存器的值要保存到kernel-mode stack,反之,从kernel-mode切换回user-mode时,要把寄存器的值恢复出来。

进行“context switch”时,scheduler将当前kernel-mode stack中的值保存在task_struct中,并把下一个将要运行tasktask_struct值恢复到kernel-mode stack中。这样,从kernel-mode返回到user-mode,就会运行另外一个task

 

SystemTap 笔记 (11)—— 命令行参数

SystemTap使用$@传递命令行参数:$传递整数参数,@传递字符串参数。举例如下:

# cat test.stp
#!/usr/bin/stap

probe begin
{
        printf("arg1 is %d, arg2 is %s\n", $1, @2)
        exit()
}

执行如下:

 # ./test.stp 100 "test"
arg1 is 100, arg2 is test

参考资料:
Command-Line Arguments

 

SystemTap 笔记 (10)—— “@defined”和“@choose_defined”

随着代码的不断变化,有些target variable可能在新的版本里不存在了。@defined用来检查target variable是否存在。举例如下:

probe vm.pagefault = kernel.function("__handle_mm_fault@mm/memory.c") ?,
                     kernel.function("handle_mm_fault@mm/memory.c") ?
{
        write_access = (@defined($flags) ? $flags & FAULT_FLAG_WRITE : $write_access)
}

上述代码则用来根据flags是否存在,来赋给write_access不同的值。

此外还有@choose_defined@choose_defined($a, $b)相当于@defined($a)? $a : $b。举例如下:

probe vm.pagefault = kernel.function("handle_mm_fault@mm/memory.c")
{
        write_access = @choose_defined($write_access, 0)
}

 

参考资料:
Checking Target Variable Availability

Arguments

 

最孤独的人?最幸福的人?

上周看到一篇报道,讲的是一名俄罗斯气象员独自一人在极地附近工作和生活的故事。看完以后,我还特意找到了这则消息最原始的英文文章出处,详细地读了一下。有人说,他是世界上最孤独的人,不过在我看来,他同时也是最幸福的人。他可以把整个世俗抛在身后,每天专心致志地从事自己想做的事情,过着一种几乎“与世无争”的生活,在自己的“桃花源”中尽情地享受着。在当下这个世界,有几个能像他这般“幸福”?真是让人羡慕不已。。。

Crash工具笔记 (3)—— 在Xen环境使用crash

这两周一直在crash邮件列表里讨论如何在SuSE Xen上使用crash调试Dom0 kernel。邮件来来回回讨论很多(参见这里),最后还发现了一个bug。细节不说了,把最后的结果总结一下:

(1)由于SuSE kerenl默认编译打开CONFIG_STRICT_DEVMEM编译开关,所以crash工具无法完全访问/dev/mem,可以使用/proc/kcore作为代替;

(2)SuSE带有crash.ko驱动(位于:“/lib/modules/uname -r/updates/crash.ko”),但默认没有安装,可以自己手动安装(使用insmod命令),然后就可以使用了:

# crash

crash 7.1.3
Copyright (C) 2002-2014  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

crash: /boot/xen-4.5.gz: original filename unknown
       Use "-f /boot/xen-4.5.gz" on command line to prevent this message.

WARNING: machine type mismatch:
         crash utility: X86_64
         /var/tmp/xen-4.5.gz_ud3IRy: X86

crash: /boot/symtypes-3.12.49-6-default.gz: original filename unknown
       Use "-f /boot/symtypes-3.12.49-6-default.gz" on command line to
prevent this message.

crash: /boot/symvers-3.12.49-6-default.gz: original filename unknown
       Use "-f /boot/symvers-3.12.49-6-default.gz" on command line to
prevent this message.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: /boot/vmlinux-3.12.49-6-xen.gz
   DEBUGINFO: /usr/lib/debug/boot/vmlinux-3.12.49-6-xen.debug
    DUMPFILE: /dev/crash
        CPUS: 128
        DATE: Fri Nov 20 06:55:06 2015
      UPTIME: 18:51:36
LOAD AVERAGE: 1.76, 1.48, 1.21
       TASKS: 1230
    NODENAME: dl980-5
     RELEASE: 3.12.49-6-xen
     VERSION: #1 SMP Mon Oct 26 16:05:37 UTC 2015 (11560c3)
     MACHINE: x86_64  (1995 Mhz)
      MEMORY: 125.9 GB
         PID: 6618
     COMMAND: "crash"
        TASK: ffff881ea93b2140  [THREAD_INFO: ffff881e869f2000]
         CPU: 112
       STATE: TASK_RUNNING (ACTIVE)

 

Linux kernel 笔记 (49)——ERESTARTSYS和EINTR

LDD3中提到驱动代码返回ERESTARTSYSEINTR时如何选择:

Note the check on the return value of down_interruptible; if it returns nonzero, the operation was interrupted. The usual thing to do in this situation is to return -ERESTARTSYS。 Upon seeing this return code, the higher layers of the kernel will either restart the call from the beginning or return the error to the user. If you return -ERESTARTSYS , you must first undo any user-visible changes that might have been made, so that the right thing happens when the system call is retried. If you cannot undo things in this manner, you should return -EINTR instead.

即如果可以把用户看到的设备状态完全回滚到执行驱动代码之前,则返回ERESTARTSYS,否则返回EINTR。因为EINTR错误可以使系统调用失败,并且返回错误码为EINTR给应用程序。而ERESTARTSYS有可能会让kernel重新发起操作,而不会惊动应用程序。可以参考这篇帖子

 

Linux kernel 笔记 (48)——CONFIG_STRICT_DEVMEM和/dev/crash

CONFIG_STRICT_DEVMEM配置项的作用是控制对/dev/mem的访问:一旦置成yes,则只能访问一段特定的区域。比如在X86平台,只能访问内存开始的1M区域:

# dd if=/dev/mem of=/dev/null
dd: error reading ‘/dev/mem’: Operation not permitted
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.0349979 s, 30.0 MB/s

RedHat开发了一个驱动:/dev/crash,可以用来取代/dev/mem,方便调试器(例如crash)访问物理内存区域。

参考资料:
/dev/crash Driver
Tools:Memory Imaging