Go语言的string和byte slice之间的转换

以下摘自The Go Programming Language

A string contains an array of bytes that, once created, is immutable. By contrast, the elements of a byte slice can be freely modified.

Strings can be converted to byte slices and back again:
s := “abc”
b := []byte(s)
s2 := string(b)

Conceptually, the []byte(s) conversion allocates a new byte array holding a copy of the bytes of s, and yields a slice that references the entirety of that array. An optimizing compiler may be able to avoid the allocation and copying in some cases, but in general copying is required to ensure that the bytes of s remain unchanged even if those of b are subsequently modified. The conversion from byte slice back to string with string(b) also makes a copy, to ensure immutability of the resulting string s2.

由于Go语言中字符串是不可修改的,因此如果要修改其中内容,就要把其转化成byte slice。此外,byte slice也可以转化成字符串。这两种转化都需要分配一块新的内存,然后进行内容拷贝。

 

使用vmstat命令监控CPU使用

vmstat命令可以用来监控CPU的使用状况。举例如下:

# vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 5201924   1328 5578060    0    0     0     0 1582 6952  2  1 98  0  0
 1  0      0 5200984   1328 5577996    0    0     0     0 2020 20567  9  1 90  0  0
 0  0      0 5198668   1328 5577952    0    0     0     0 1568 7617  5  1 94  0  0
 0  0      0 5194844   1328 5578000    0    0     0   187 1249 7057  1  1 98  0  0
 0  0      0 5199956   1328 5578232    0    0     0     0 1496 7306  4  1 95  0  0

上述命令每隔1秒输出系统状态,最后5列是描述的是CPU状况。man手册上关于这5列的含义描述的很清楚:

CPU
       These are percentages of total CPU time.
       us: Time spent running non-kernel code.  (user time, including nice time)
       sy: Time spent running kernel code.  (system time)
       id: Time spent idle.  Prior to Linux 2.5.41, this includes IO-wait time.
       wa: Time spent waiting for IO.  Prior to Linux 2.5.41, included in idle.
       st: Time stolen from a virtual machine.  Prior to Linux 2.6.11, unknown.

vmstat实质上是从/proc/stat文件获得系统状态:

# cat /proc/stat
cpu  381584 711 299364 1398303520 429839 0 251 0 0 0
cpu0 90740 58 44641 174627550 131209 0 120 0 0 0
cpu1 43141 26 22925 174746812 108219 0 10 0 0 0
cpu2 41308 35 25097 174831161 25877 0 40 0 0 0
cpu3 39301 70 27514 174836084 27792 0 4 0 0 0
cpu4 39187 78 46191 174750027 109013 0 0 0 0 0
......

需要注意的是这里数字的单位是Jiffies

另外,vmstat计算CPU时间百分比使用的是“四舍五入”算法(vmstat.c):

static void new_format(void){
    ......
    duse = *cpu_use + *cpu_nic;
    dsys = *cpu_sys + *cpu_xxx + *cpu_yyy;
    didl = *cpu_idl;
    diow = *cpu_iow;
    dstl = *cpu_zzz;
    Div = duse + dsys + didl + diow + dstl;
    if (!Div) Div = 1, didl = 1;
    divo2 = Div / 2UL;
    printf(w_option ? wide_format : format,
           running, blocked,
           unitConvert(kb_swap_used), unitConvert(kb_main_free),
           unitConvert(a_option?kb_inactive:kb_main_buffers),
           unitConvert(a_option?kb_active:kb_main_cached),
           (unsigned)( (unitConvert(*pswpin  * kb_per_page) * hz + divo2) / Div ),
           (unsigned)( (unitConvert(*pswpout * kb_per_page) * hz + divo2) / Div ),
           (unsigned)( (*pgpgin        * hz + divo2) / Div ),
           (unsigned)( (*pgpgout           * hz + divo2) / Div ),
           (unsigned)( (*intr          * hz + divo2) / Div ),
           (unsigned)( (*ctxt          * hz + divo2) / Div ),
           (unsigned)( (100*duse            + divo2) / Div ),
           (unsigned)( (100*dsys            + divo2) / Div ),
           (unsigned)( (100*didl            + divo2) / Div ),
           (unsigned)( (100*diow            + divo2) / Div ),
           (unsigned)( (100*dstl            + divo2) / Div )
    );
    ......
}

所以会出现CPU利用百分比相加大于100的情况:2 + 1 + 98 = 101

另外,在Linux系统上,r字段表示的是当前正在运行和等待运行的task的总和。

 

参考资料:
/proc/stat explained
procps

 

Go语言的浮点数

以下摘自The Go Programming Language

A float32 provides approximately six decimal digits of precision, whereas a float64 provides about 15 digits; float64 should be preferred for most purposes because float32 computations accumulate error rapidly unless one is quite careful, and the smallest positive integer that cannot be exactly represented as a float32 is not large:

var f float32 = 16777216 // 1 << 24
fmt.Println(f == f+1) // “true”!

因为float32计算时累计错误很快,所以大多数时候推荐使用float64

The math package has functions for creating and detecting the special values defined by IEEE 754: the positive and negative infinities, which represent numbers of excessive magnitude and the result of division by zero; and NaN (“not a number”), the result of such mathematically dubious operations as 0/0 or Sqrt(-1).

var z float64
fmt.Println(z, -z, 1/z, -1/z, z/z) // “0 -0 +Inf -Inf NaN”

要注意正负0,正负无穷和NaN

The function math.IsNaN tests whether its argument is a not-a-number value, and math.NaN returns such a value. It’s tempting to use NaN as a sentinel value in a numeric computation, but testing whether a specific computational result is equal to NaN is fraught with peril because any comparison with NaN always yields false except !=, which is always the negation of ==:

nan := math.NaN()
fmt.Println(nan != nan, nan == nan, nan < nan, nan > nan) // “true false false false”

要留心数值和NaN比较时的返回值。

 

Go语言中使用fmt.Printf的小技巧

以下摘自The Go Programming Language

When printing numbers using the fmt package, we can control the radix and format with the %d, %o, and %x verbs, as shown in this example:

o := 0666
fmt.Printf(“%d %[1]o %#[1]o\n”, o) // “438 666 0666”
x := int64(0xdeadbeef)
fmt.Printf(“%d %[1]x %#[1]x %#[1]X\n”, x)
// Output:
// 3735928559 deadbeef 0xdeadbeef 0XDEADBEEF

Note the use of two fmt tricks. Usually a Printf format string containing multiple % verbs would require the same number of extra operands, but the [1] “adverbs” after % tell Printf to use the first operand over and over again. Second, the # adverb for %o or %x or %X tells Printf to emit a 0 or 0x or 0X prefix respectively.

Rune literals are written as a character within single quotes. The simplest example is an ASCII character like ‘a’, but it’s possible to write any Unicode code point either directly or with numeric escapes, as we will see shortly.

Runes are printed with %c, or with %q if quoting is desired: ascii := ‘a’
unicode := ‘ ‘
newline := ‘\n’
fmt.Printf(“%d %[1]c %[1]q\n”, ascii) // “97 a ‘a'”
fmt.Printf(“%d %[1]c %[1]q\n”, unicode) // “22269 ‘ ‘”
fmt.Printf(“%d %[1]q\n”, newline) // “10 ‘\n'”

总结一下,“%[1]”还是格式化第一个参数;“%#”会打印出数值的前缀;而“%q”会加上引号。举例如下:

package main

import "fmt"

func main() {
    var c rune = '楠'
    fmt.Printf("%c %[1]d %#[1]x %[1]q", c)
}

执行结果:

楠 26976 0x6960 '楠'

 

Go语言的bit clear操作

以下摘自The Go Programming Language

The &^ operator is bit clear (AND NOT): in the expression z = x &^ y, each bit of z is 0 if the corresponding bit of y is 1; otherwise it equals the corresponding bit of x.

z = x &^ y运算相当于先把y取反(针对y的每个bit0变成11变成0),然后再和x进行&运算。参考下例:

package main
import "fmt"

func main(){
    var x uint8 = 1
    var y uint8 = 1 << 2

    fmt.Printf("%08b\n", x &^ y);

}  

执行结果如下:

00000001

 

Bash中的测试表达式

Bash shell中,每个执行命令都有一个返回值表示其退出状态:0表示true1表示falsetest命令是专门测试执行命令返回值,其格式如下:

test expression
或:
[ expression ]

目前test只支持3种测试对象:字符串,整数(0和正整数,不包含负数和小数点)和文件。当expression测试为“真”时,test命令就返回0true),反之返回非0false)。 关于test表达式的例子和解释,可以参考How to understand if condition in bash?

参考资料:
Shell十三问

 

GNU Parallel简介

GNU parallel可以并行地执行shell命令。看一个简单的例子:

# ls
a  b  c
# cat a
aaaa

# cat b
bbbb

# cat c
cccc

# parallel cat ::: *
aaaa

bbbb

cccc

上面的例子输出了abc3个文件的内容。:::告诉GNU parallel从命令行而不是stdin读取参数,而shell会把*扩展成当前目录下的文件名。

再看一个例子:

# parallel sleep {}\; echo {} ::: 2 1 4 3
1
2
3
4
# parallel -k sleep {}\; echo {} ::: 2 1 4 3
2
1
4
3

正常情况下,完成一个jobparallel就会把这个job的内容输出。-k选项保证输出顺序和输入顺序一致。而{}会替换成input line

GNU parallel不会把多个输出的内容混杂在一起,对比下列两个命令输出:

# traceroute foss.org.my & traceroute debian.org & traceroute freenetproject.org & wait
[1] 4920
[2] 4921
[3] 4922
traceroute to debian.org (149.20.20.20), 30 hops max, 60 byte packets
traceroute to freenetproject.org (80.68.94.117), 30 hops max, 60 byte packets
foss.org.my: Name or service not known
Cannot handle "host" cmdline arg `foss.org.my' on position 1 (argc 1)
[1]   Exit 2                  traceroute foss.org.my
 1  16.187.248.2 (16.187.248.2)  1.705 ms  1.709 ms  2.067 ms
 1  16.187.248.2 (16.187.248.2)  1.315 ms  1.314 ms  1.618 ms
......

# parallel traceroute ::: foss.org.my debian.org freenetproject.org
foss.org.my: Name or service not known
Cannot handle "host" cmdline arg `foss.org.my' on position 1 (argc 1)
traceroute to debian.org (140.211.15.34), 30 hops max, 60 byte packets
 1  16.187.248.2 (16.187.248.2)  1.871 ms  1.857 ms  2.151 ms
......
traceroute to freenetproject.org (80.68.94.117), 30 hops max, 60 byte packets
 1  16.187.248.2 (16.187.248.2)  2.175 ms  2.471 ms  2.471 ms
 2  16.160.221.81 (16.160.221.81)  0.456 ms  0.463 ms  0.463 ms
......

参考资料:
GNU Parallel: The Command-Line Power Tool
GNU Parallel manual

 

Go语言的比较运算符

以下摘自The Go Programming Language

Two integers of the same type may be compared using the binary comparison operators below; the type of a comparison expression is a boolean.

== equal to
!= not equal to
< less than
<= less than or equal to
greater than
= greater than or equal to

In fact, all values of basic type—booleans, numbers, and strings—are comparable, meaning that two values of the same type may be compared using the == and != operators. Furthermore, integers, floating-point numbers, and strings are ordered by the comparison operators. The values of many other types are not comparable, and no other types are ordered.

 

Go语言的int,uint和uintptr类型

以下摘自The Go Programming Language

There are also two types called just int and uint that are the natural or most efficient size for signed and unsigned integers on a particular platform; int is by far the most widely used numeric type. Both these types have the same size, either 32 or 64 bits, but one must not make assumptions about which; different compilers may make different choices even on identical hardware.

Finally, there is an unsigned integer type uintptr, whose width is not specified but is sufficient to hold all the bits of a pointer value. The uintptr type is used only for low-level programming, such as at the boundary of a Go program with a C library or an operating system.

Regardless of their size, int, uint, and uintptr are different types from their explicitly sized siblings. Thus int is not the same type as int32, even if the natural size of integers is 32 bits, and an explicit conversion is required to use an int value where an int32 is needed, and vice versa.

另外关于intuint类型的使用:

Although Go provides unsigned numbers and arithmetic, we tend to use the signed int form even for quantities that can’t be negative, such as the length of an array, though uint might seem a more obvious choice. Indeed, the built-in len function returns a signed int, as in this loop which announces prize medals in reverse order:

medals := []string{“gold”, “silver”, “bronze”}
for i := len(medals) – 1; i >= 0; i– {
fmt.Println(medals[i]) // “bronze”, “silver”, “gold”
}

The alternative would be calamitous. If len returned an unsigned number, then i too would be a uint, and the condition i >= 0 would always be true by definition. After the third iteration, in which i == 0, the i– statement would cause i to become not −1, but the maximum uint value, and the evaluation of medals[i] would fail at run time, or panic, by attempting to access an element outside the bounds of the slice.

For this reason, unsigned numbers tend to be used only when their bitwise operators or peculiar arithmetic operators are required, as when implementing bit sets, parsing binary file formats, or for hashing and cryptography. They are typically not used for merely non-negative quantities.

多数时候,在Golang中应该使用int类型,即使有些变量不可能为负数。否则可能会出现严重的问题(就像上面的例子)。而uint仅仅是用在位运算,hash运算等等少数情况中。