UP VS SMP

UP(Uni-Processor):系统只有一个处理器单元,即单核CPU系统。

SMP(Symmetric Multi-Processors):系统有多个处理器单元。各个处理器之间共享总线,内存等等。在操作系统看来,各个处理器之间没有区别。

要注意,这里提到的“处理器单元”是指“logic CPU”,而不是“physical CPU”。举个例子,如果一个“physical CPU”包含2core,并且一个core包含2hardware thread。则一个“处理器单元”就是一个hardware thread

Linux kernel IOMMU代码分析笔记(1)

Linux kernel代码版本是3.10

intel-iommu.h头文件定义了root-entry table address寄存器:

#define DMAR_RTADDR_REG 0x20    /* Root entry table */

DMAR_RTADDR_REG只在iommu_set_root_entry这个函数中使用(这个函数作用是更新root-entry table的地址):

static void iommu_set_root_entry(struct intel_iommu *iommu)
{
    void *addr;
    u32 sts;
    unsigned long flag;

    addr = iommu->root_entry;

    raw_spin_lock_irqsave(&iommu->register_lock, flag);
    dmar_writeq(iommu->reg + DMAR_RTADDR_REG, virt_to_phys(addr));

    writel(iommu->gcmd | DMA_GCMD_SRTP, iommu->reg + DMAR_GCMD_REG);

    /* Make sure hardware complete it */
    IOMMU_WAIT_OP(iommu, DMAR_GSTS_REG,
              readl, (sts & DMA_GSTS_RTPS), sts);

    raw_spin_unlock_irqrestore(&iommu->register_lock, flag);
}

DMAR_RTADDR_REG存储的是root-entry table的物理地址(virt_to_phys()函数把virtual address转成physical address)。整个root-entry table4KiB大小,并且要求起始地址是“页面对齐的(page-alignedpage长度是4KiB)”,所以DMAR_RTADDR_REG12 bits0。更新DMAR_RTADDR_REG代码如下:

dmar_writeq(iommu->reg + DMAR_RTADDR_REG, virt_to_phys(addr));

更新完DMAR_RTADDR_REG寄存器,还要把global command寄存器的SRTPSet Root Table Pointer)位置1

writel(iommu->gcmd | DMA_GCMD_SRTP, iommu->reg + DMAR_GCMD_REG);

最后读取Global Status寄存器的RTPSRoot Table Pointer Status)位,确认更新成功:

IOMMU_WAIT_OP(iommu, DMAR_GSTS_REG,
                  readl, (sts & DMA_GSTS_RTPS), sts);

*NIX & Hacking —— 第5期

做一本我感兴趣的杂志,就这么简单!

C

Integer Overflow

Golang

breaking out of a select statement when all channels are closed

Kernel

Feature: High Memory In The Linux Kernel
What’s Next for Containers? User Namespaces

Network

Recent advancements in Linux TCP congestion control
SO_REUSEPORT – Scaling Techniques for Servers with High Connection Rates

Rust

Rust in Detail: Writing Scalable Chat Service from Scratch

Shell

fish shell

Easter egg

Announcing the first Art of Computer Programming eBooks
Goodbye Moto
Intel DDIO, LLC cache, buffer alignment, prefetching, shared locks and packet rates
Jokes
List of freely available Programming Books
Mandelbrot Set with SIMD Intrinsics

Linux kernel 笔记 (3) ——page和zone

MMU(Memory Management Unit)用来管理内存的最小单位是一个物理页面(physical page,在32-bit系统上通常每个页面为4k64-bit系统上为8k)。Linux kernel使用struct page结构体(定义在<linux/mm_types.h>)来代表每个物理页面:

struct page {
    /* First double word block */
    unsigned long flags;        /* Atomic flags, some possibly
                     * updated asynchronously */
    struct address_space *mapping;  /* If low bit clear, points to
                     * inode address_space, or NULL.
                     * If page mapped as anonymous
                     * memory, low bit is set, and
                     * it points to anon_vma object:
                     * see PAGE_MAPPING_ANON below.
                     */
    ......
}  

受制于一些硬件限制,有些物理页面不能用来进行某种操作。所以kernel把页面分成不同的zone
ZONE_DMA:用来进行DMA操作的页面;
ZONE_DMA32:也是用来进行DMA操作的页面,不过仅针对32-bit设备;
ZONE_NORMAL:包含常规的,用来映射的页面;
ZONE_HIGHMEM:包含不能永久地映射到kernel的地址空间(address space)的页面。
还有其它的zone定义(例如:ZONE_MOVABLE)。zone_type定义在 <linux/mmzone.h>文件里:

enum zone_type {
#ifdef CONFIG_ZONE_DMA
    /*
     * ZONE_DMA is used when there are devices that are not able
     * to do DMA to all of addressable memory (ZONE_NORMAL). Then we
     * carve out the portion of memory that is needed for these devices.
     * The range is arch specific.
     *
     * Some examples
     *
     * Architecture     Limit
     * ---------------------------
     * parisc, ia64, sparc  <4G
     * s390         <2G
     * arm          Various
     * alpha        Unlimited or 0-16MB.
     *
     * i386, x86_64 and multiple other arches
     *          <16M.
     */
    ZONE_DMA,
#endif
#ifdef CONFIG_ZONE_DMA32
    /*
     * x86_64 needs two ZONE_DMAs because it supports devices that are
     * only able to do DMA to the lower 16M but also 32 bit devices that
     * can only do DMA areas below 4G.
     */
    ZONE_DMA32,
#endif
    /*
     * Normal addressable memory is in ZONE_NORMAL. DMA operations can be
     * performed on pages in ZONE_NORMAL if the DMA devices support
     * transfers to all addressable memory.
     */
    ZONE_NORMAL,
#ifdef CONFIG_HIGHMEM
    /*
     * A memory area that is only addressable by the kernel through
     * mapping portions into its own address space. This is for example
     * used by i386 to allow the kernel to address the memory beyond
     * 900MB. The kernel will set up special mappings (page
     * table entries on i386) for each page that the kernel needs to
     * access.
     */
    ZONE_HIGHMEM,
#endif
    ZONE_MOVABLE,
    __MAX_NR_ZONES
};  

如果CPU体系结构允许DMA操作访问任何地址空间,那就没有ZONE_DMA。同理,Intel X86_64处理器可以访问所有的内存空间,也就不存在ZONE_HIGHMEM。 关于“High memory”,可以参考这篇文章

Linux kernel 笔记 (2) ——kernel代码的一些特性

Linux kernel代码相对于user-space程序的一些特性:
(1)kernel代码不能访问标准C程序库(libc)和头文件(例如打印使用printk,而不是printf);
(2)kernel代码是用GNU C实现的,不是ANSI C(比如:内联函数,内联汇编,分支预测等等);
(3)kernel程序没有内存保护;
(4)避免在kernel代码中引入浮点运算;
(5)kernel程序为每个进程预分配的堆栈空间很小;
(6)因为kernel程序是可抢占的(preemptive),支持symmetrical multiprocessing (SMP)和异步中断(asynchronous interrupt),所以开发kernel程序时要时刻考虑并发和同步问题;
(7)可移植性对于kernel代码非常重要。

*NIX & Hacking —— 第4期

做一本我感兴趣的杂志,就这么简单!

Compiler

Notes on How Parsers and Compilers Work

Golang

GopherCon 2015 live blog
Testing in Go
Why does adding parentheses in if condition results in compile error?

Kernel

Choosing a Linux Tracer (2015)
Debugging by printing
Learning Kernel Programming
LTTng Packages now Available for Red Hat Enterprise Linux 7

Lua

Cloning a function in Lua

Network

Unit testing a TCP stack

Unix

Where the Unix philosophy breaks down

Easter egg

Starting from scratch: How do you build a world-class research lab?
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Linux kernel 笔记 (1) ——CPU在做什么?

In fact, in Linux, we can generalize that each processor is doing exactly one of three things at any given moment:
a) In user-space, executing user code in a process
b) In kernel-space, in process context, executing on behalf of a specific process
c) In kernel-space, in interrupt context, not associated with a process, handling an
interrupt

Linux中,任何时候,CPU都在做下面三件事中的一件:

a)运行进程的用户空间代码;
b)运行进程的内核空间代码;
c)处理中断(也是工作在内核空间,但不与任何进程关联)。

printk函数简介

printk函数是Linux kernel开发中用来打印log的常用函数,同使用在用户空间程序的printf函数很类似。实现代码如下:

asmlinkage int printk(const char *fmt, ...)
{
    va_list args;
    int r;

#ifdef CONFIG_KGDB_KDB
    if (unlikely(kdb_trap_printk)) {
        va_start(args, fmt);
        r = vkdb_printf(fmt, args);
        va_end(args);
        return r;
    }
#endif
    va_start(args, fmt);
    r = vprintk_emit(0, -1, NULL, 0, fmt, args);
    va_end(args);

    return r;
}

举个例子(注意KERN_ALERT"DEBUG ..."之间没有逗号):

printk(KERN_ALERT "DEBUG: Passed %s %d \n",__FUNCTION__,__LINE__);

KERN_ALERT代表log的级别,参考kern_levels.h

#define KERN_EMERG  KERN_SOH "0"    /* system is unusable */
#define KERN_ALERT  KERN_SOH "1"    /* action must be taken immediately */
#define KERN_CRIT   KERN_SOH "2"    /* critical conditions */
#define KERN_ERR    KERN_SOH "3"    /* error conditions */
#define KERN_WARNING    KERN_SOH "4"    /* warning conditions */
#define KERN_NOTICE KERN_SOH "5"    /* normal but significant condition */
#define KERN_INFO   KERN_SOH "6"    /* informational */
#define KERN_DEBUG  KERN_SOH "7"    /* debug-level messages */ 

数字越大,代表优先级越低。

另外关于printk格式化字符串形式,参考printk-formats.txt

/proc/sys/kernel/printk这个文件显示了和printk相关的log级别设置:

[root@localhost kernel]# cat /proc/sys/kernel/printk
7       4       1       7

4个值的含义依次如下:
console_loglevel:当前consolelog级别,只有更高优先级的log才被允许打印到console
default_message_loglevel:当不指定log级别时,printk默认使用的log级别;
minimum_console_loglevelconsole能设定的最高log级别;
default_console_loglevel:默认的consolelog级别。

请看下面这个例子:

[root@localhost kernel]# echo 8 > /proc/sys/kernel/printk
[root@localhost kernel]# cat /proc/sys/kernel/printk
8       4       1       7

可以看到,console_loglevel的级别修改成了8

参考资料:
Debugging by printing