Go语言利用goroutine实现递归

Recursion And Tail Calls In Go这篇文章讲了用goroutine实现函数递归调用:这样做可以避免过多函数调用引起的堆栈空间的不断增大,感觉很巧妙。以下是例子代码:

package main
import "fmt"

func recursiveCall(product int, num int, ch chan int)  {
    product += num

    if num == 1 {
        ch <- product
        return
    }

    go recursiveCall(product, num - 1, ch)
}

func main()  {
    ch := make(chan int)
    go recursiveCall(0, 4, ch)
    product := <-ch
    fmt.Printf("Product is %d\n", product)
}  

执行结果如下:

Product is 10

 

Go语言的string和byte slice之间的转换

以下摘自The Go Programming Language

A string contains an array of bytes that, once created, is immutable. By contrast, the elements of a byte slice can be freely modified.

Strings can be converted to byte slices and back again:
s := “abc”
b := []byte(s)
s2 := string(b)

Conceptually, the []byte(s) conversion allocates a new byte array holding a copy of the bytes of s, and yields a slice that references the entirety of that array. An optimizing compiler may be able to avoid the allocation and copying in some cases, but in general copying is required to ensure that the bytes of s remain unchanged even if those of b are subsequently modified. The conversion from byte slice back to string with string(b) also makes a copy, to ensure immutability of the resulting string s2.

由于Go语言中字符串是不可修改的,因此如果要修改其中内容,就要把其转化成byte slice。此外,byte slice也可以转化成字符串。这两种转化都需要分配一块新的内存,然后进行内容拷贝。

 

Idle状态的CPU在做什么?

What Does an Idle CPU Do?这篇文章介绍了CPUIdle状态下在干什么,讲解地很清晰。我总结一下:

当操作系统“无事可做”时,就会运行Idle进程。以LinuxIdle进程代码为例:

while (1) {
    while(!need_resched()) {
        cpuidle_idle_call();
    }

    /*
      [Note: Switch to a different task. We will return to this loop when the
      idle task is again selected to run.]
    */
    schedule_preempt_disabled();
}

可以看到只要没有“调度”需求(need_resched),就会执行cpuidle_idle_call函数。而对于Intel处理器而言,保持Idle状态意味着执行hlt指令。

 

使用vmstat命令监控CPU使用

vmstat命令可以用来监控CPU的使用状况。举例如下:

# vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 5201924   1328 5578060    0    0     0     0 1582 6952  2  1 98  0  0
 1  0      0 5200984   1328 5577996    0    0     0     0 2020 20567  9  1 90  0  0
 0  0      0 5198668   1328 5577952    0    0     0     0 1568 7617  5  1 94  0  0
 0  0      0 5194844   1328 5578000    0    0     0   187 1249 7057  1  1 98  0  0
 0  0      0 5199956   1328 5578232    0    0     0     0 1496 7306  4  1 95  0  0

上述命令每隔1秒输出系统状态,最后5列是描述的是CPU状况。man手册上关于这5列的含义描述的很清楚:

CPU
       These are percentages of total CPU time.
       us: Time spent running non-kernel code.  (user time, including nice time)
       sy: Time spent running kernel code.  (system time)
       id: Time spent idle.  Prior to Linux 2.5.41, this includes IO-wait time.
       wa: Time spent waiting for IO.  Prior to Linux 2.5.41, included in idle.
       st: Time stolen from a virtual machine.  Prior to Linux 2.6.11, unknown.

vmstat实质上是从/proc/stat文件获得系统状态:

# cat /proc/stat
cpu  381584 711 299364 1398303520 429839 0 251 0 0 0
cpu0 90740 58 44641 174627550 131209 0 120 0 0 0
cpu1 43141 26 22925 174746812 108219 0 10 0 0 0
cpu2 41308 35 25097 174831161 25877 0 40 0 0 0
cpu3 39301 70 27514 174836084 27792 0 4 0 0 0
cpu4 39187 78 46191 174750027 109013 0 0 0 0 0
......

需要注意的是这里数字的单位是Jiffies

另外,vmstat计算CPU时间百分比使用的是“四舍五入”算法(vmstat.c):

static void new_format(void){
    ......
    duse = *cpu_use + *cpu_nic;
    dsys = *cpu_sys + *cpu_xxx + *cpu_yyy;
    didl = *cpu_idl;
    diow = *cpu_iow;
    dstl = *cpu_zzz;
    Div = duse + dsys + didl + diow + dstl;
    if (!Div) Div = 1, didl = 1;
    divo2 = Div / 2UL;
    printf(w_option ? wide_format : format,
           running, blocked,
           unitConvert(kb_swap_used), unitConvert(kb_main_free),
           unitConvert(a_option?kb_inactive:kb_main_buffers),
           unitConvert(a_option?kb_active:kb_main_cached),
           (unsigned)( (unitConvert(*pswpin  * kb_per_page) * hz + divo2) / Div ),
           (unsigned)( (unitConvert(*pswpout * kb_per_page) * hz + divo2) / Div ),
           (unsigned)( (*pgpgin        * hz + divo2) / Div ),
           (unsigned)( (*pgpgout           * hz + divo2) / Div ),
           (unsigned)( (*intr          * hz + divo2) / Div ),
           (unsigned)( (*ctxt          * hz + divo2) / Div ),
           (unsigned)( (100*duse            + divo2) / Div ),
           (unsigned)( (100*dsys            + divo2) / Div ),
           (unsigned)( (100*didl            + divo2) / Div ),
           (unsigned)( (100*diow            + divo2) / Div ),
           (unsigned)( (100*dstl            + divo2) / Div )
    );
    ......
}

所以会出现CPU利用百分比相加大于100的情况:2 + 1 + 98 = 101

另外,在Linux系统上,r字段表示的是当前正在运行和等待运行的task的总和。

 

参考资料:
/proc/stat explained
procps

 

Go语言的浮点数

以下摘自The Go Programming Language

A float32 provides approximately six decimal digits of precision, whereas a float64 provides about 15 digits; float64 should be preferred for most purposes because float32 computations accumulate error rapidly unless one is quite careful, and the smallest positive integer that cannot be exactly represented as a float32 is not large:

var f float32 = 16777216 // 1 << 24
fmt.Println(f == f+1) // “true”!

因为float32计算时累计错误很快,所以大多数时候推荐使用float64

The math package has functions for creating and detecting the special values defined by IEEE 754: the positive and negative infinities, which represent numbers of excessive magnitude and the result of division by zero; and NaN (“not a number”), the result of such mathematically dubious operations as 0/0 or Sqrt(-1).

var z float64
fmt.Println(z, -z, 1/z, -1/z, z/z) // “0 -0 +Inf -Inf NaN”

要注意正负0,正负无穷和NaN

The function math.IsNaN tests whether its argument is a not-a-number value, and math.NaN returns such a value. It’s tempting to use NaN as a sentinel value in a numeric computation, but testing whether a specific computational result is equal to NaN is fraught with peril because any comparison with NaN always yields false except !=, which is always the negation of ==:

nan := math.NaN()
fmt.Println(nan != nan, nan == nan, nan < nan, nan > nan) // “true false false false”

要留心数值和NaN比较时的返回值。

 

Go语言中使用fmt.Printf的小技巧

以下摘自The Go Programming Language

When printing numbers using the fmt package, we can control the radix and format with the %d, %o, and %x verbs, as shown in this example:

o := 0666
fmt.Printf(“%d %[1]o %#[1]o\n”, o) // “438 666 0666”
x := int64(0xdeadbeef)
fmt.Printf(“%d %[1]x %#[1]x %#[1]X\n”, x)
// Output:
// 3735928559 deadbeef 0xdeadbeef 0XDEADBEEF

Note the use of two fmt tricks. Usually a Printf format string containing multiple % verbs would require the same number of extra operands, but the [1] “adverbs” after % tell Printf to use the first operand over and over again. Second, the # adverb for %o or %x or %X tells Printf to emit a 0 or 0x or 0X prefix respectively.

Rune literals are written as a character within single quotes. The simplest example is an ASCII character like ‘a’, but it’s possible to write any Unicode code point either directly or with numeric escapes, as we will see shortly.

Runes are printed with %c, or with %q if quoting is desired: ascii := ‘a’
unicode := ‘ ‘
newline := ‘\n’
fmt.Printf(“%d %[1]c %[1]q\n”, ascii) // “97 a ‘a'”
fmt.Printf(“%d %[1]c %[1]q\n”, unicode) // “22269 ‘ ‘”
fmt.Printf(“%d %[1]q\n”, newline) // “10 ‘\n'”

总结一下,“%[1]”还是格式化第一个参数;“%#”会打印出数值的前缀;而“%q”会加上引号。举例如下:

package main

import "fmt"

func main() {
    var c rune = '楠'
    fmt.Printf("%c %[1]d %#[1]x %[1]q", c)
}

执行结果:

楠 26976 0x6960 '楠'

 

Go语言的bit clear操作

以下摘自The Go Programming Language

The &^ operator is bit clear (AND NOT): in the expression z = x &^ y, each bit of z is 0 if the corresponding bit of y is 1; otherwise it equals the corresponding bit of x.

z = x &^ y运算相当于先把y取反(针对y的每个bit0变成11变成0),然后再和x进行&运算。参考下例:

package main
import "fmt"

func main(){
    var x uint8 = 1
    var y uint8 = 1 << 2

    fmt.Printf("%08b\n", x &^ y);

}  

执行结果如下:

00000001

 

Bash中的测试表达式

Bash shell中,每个执行命令都有一个返回值表示其退出状态:0表示true1表示falsetest命令是专门测试执行命令返回值,其格式如下:

test expression
或:
[ expression ]

目前test只支持3种测试对象:字符串,整数(0和正整数,不包含负数和小数点)和文件。当expression测试为“真”时,test命令就返回0true),反之返回非0false)。 关于test表达式的例子和解释,可以参考How to understand if condition in bash?

参考资料:
Shell十三问

 

GNU Parallel简介

GNU parallel可以并行地执行shell命令。看一个简单的例子:

# ls
a  b  c
# cat a
aaaa

# cat b
bbbb

# cat c
cccc

# parallel cat ::: *
aaaa

bbbb

cccc

上面的例子输出了abc3个文件的内容。:::告诉GNU parallel从命令行而不是stdin读取参数,而shell会把*扩展成当前目录下的文件名。

再看一个例子:

# parallel sleep {}\; echo {} ::: 2 1 4 3
1
2
3
4
# parallel -k sleep {}\; echo {} ::: 2 1 4 3
2
1
4
3

正常情况下,完成一个jobparallel就会把这个job的内容输出。-k选项保证输出顺序和输入顺序一致。而{}会替换成input line

GNU parallel不会把多个输出的内容混杂在一起,对比下列两个命令输出:

# traceroute foss.org.my & traceroute debian.org & traceroute freenetproject.org & wait
[1] 4920
[2] 4921
[3] 4922
traceroute to debian.org (149.20.20.20), 30 hops max, 60 byte packets
traceroute to freenetproject.org (80.68.94.117), 30 hops max, 60 byte packets
foss.org.my: Name or service not known
Cannot handle "host" cmdline arg `foss.org.my' on position 1 (argc 1)
[1]   Exit 2                  traceroute foss.org.my
 1  16.187.248.2 (16.187.248.2)  1.705 ms  1.709 ms  2.067 ms
 1  16.187.248.2 (16.187.248.2)  1.315 ms  1.314 ms  1.618 ms
......

# parallel traceroute ::: foss.org.my debian.org freenetproject.org
foss.org.my: Name or service not known
Cannot handle "host" cmdline arg `foss.org.my' on position 1 (argc 1)
traceroute to debian.org (140.211.15.34), 30 hops max, 60 byte packets
 1  16.187.248.2 (16.187.248.2)  1.871 ms  1.857 ms  2.151 ms
......
traceroute to freenetproject.org (80.68.94.117), 30 hops max, 60 byte packets
 1  16.187.248.2 (16.187.248.2)  2.175 ms  2.471 ms  2.471 ms
 2  16.160.221.81 (16.160.221.81)  0.456 ms  0.463 ms  0.463 ms
......

参考资料:
GNU Parallel: The Command-Line Power Tool
GNU Parallel manual