time和/usr/bin/time

当我在bash中敲入time命令时,运行的其实是bash内置的time命令:

$ time

real    0m0.000s
user    0m0.000s
sys     0m0.000s
$ type time
time is a shell keyword

这个time命令有一个-p选项,表示以posix格式输出:

$ time -p
real 0.00
user 0.00
sys 0.00

除此以外,还有一个time命令。不过我当前的机器并没有安装这个程序:

$ which time
which: no time in (/home/xiaonan/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl)

安装一下,对比bash内置的time命令:

$ sudo pacman -S time
$ type time
time is a shell keyword
$ which time
/usr/bin/time

单独运行“/usr/bin/time -p”,只会输出命令的帮助选项:

$ /usr/bin/time -p
Usage: /usr/bin/time [-apvV] [-f format] [-o file] [--append] [--verbose]
       [--portability] [--format=format] [--output=file] [--version]
       [--help] command [arg...]

需要加上具体的需要度量时间的命令:

$ /usr/bin/time -p echo

real 0.00
user 0.00
sys 0.00

此外也可以给出命令执行的详细信息:

$ /usr/bin/time -v echo

    Command being timed: "echo"
    User time (seconds): 0.00
    System time (seconds): 0.00
    Percent of CPU this job got: 0%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 1536
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 70
    Voluntary context switches: 1
    Involuntary context switches: 1
    Swaps: 0
    File system inputs: 0
    File system outputs: 0
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0

参考资料:
/usr/bin/time: not the command you think you know

Bash中的测试表达式

Bash shell中,每个执行命令都有一个返回值表示其退出状态:0表示true1表示falsetest命令是专门测试执行命令返回值,其格式如下:

test expression
或:
[ expression ]

目前test只支持3种测试对象:字符串,整数(0和正整数,不包含负数和小数点)和文件。当expression测试为“真”时,test命令就返回0true),反之返回非0false)。 关于test表达式的例子和解释,可以参考How to understand if condition in bash?

参考资料:
Shell十三问

 

docker笔记(6)—— 在docker container中执行命令的脚本

下面脚本的功能是循环地在各个container中执行命令:

#!/bin/bash -x

for i in {1..2}
do
        docker exec -i hammerdb_net${i} bash <<-EOF
        su oracle
        source /tmp/ora_env
        cd /data/oracle/tablespaces/
        rm -f *.html
        ./create_awr.sh
        mv awr.html awr_${i}.html
        EOF
done

需要注意的是在dodone之间应该使用tab而不是空格。

参考资料:
Indenting bourne shell here documents
How to write a bash script which automate entering “docker container” and doing other things?
Can’t indent heredoc to match nesting’s indent

 

Bash quoting简介

Bash quoting可以关闭Bash中具有特殊含义的meta字符的功能:
a)单引号:所有meta字符的功能均被关闭;
b)双引号:大部分meta字符的功能被关闭,除了$等少数字符;
c)反斜线(\):仅跟着\后面的meta字符被关闭。
这样就可以理解为什么解压多个zip文件时,要使用“unzip '*.zip'”而不是“unzip *.zip”。因为第二种会首先把*.zip替换成所有的文件名,而第一种方法不会这样做。

参考资料:
Shell十三问
How do I unzip multiple / many files under Linux?

 

find命令的“-exec COMMAND \;”

下面这个find命令列出当前目录下的*.stp文件:

# find . -name '*.stp' -exec ls {} \;
./Documents/one.stp
./Documents/two.stp

关于find命令的“-exec COMMAND \;”:

find

-exec COMMAND \;

Carries out COMMAND on each file that find matches. The command sequence terminates with ; (the “;” is escaped to make certain the shell passes it to find literally, without interpreting it as a special character).

If COMMAND contains {}, then find substitutes the full path name of the selected file for “{}”.

;的作用是标示命令完结,\;是让shell;原封不动地传给find命令。而{}会使用查找出来的文件的全路径名。

参考资料:
16.2. Complex Commands

 

*NIX & Hacking —— 第5期

做一本我感兴趣的杂志,就这么简单!

C

Integer Overflow

Golang

breaking out of a select statement when all channels are closed

Kernel

Feature: High Memory In The Linux Kernel
What’s Next for Containers? User Namespaces

Network

Recent advancements in Linux TCP congestion control
SO_REUSEPORT – Scaling Techniques for Servers with High Connection Rates

Rust

Rust in Detail: Writing Scalable Chat Service from Scratch

Shell

fish shell

Easter egg

Announcing the first Art of Computer Programming eBooks
Goodbye Moto
Intel DDIO, LLC cache, buffer alignment, prefetching, shared locks and packet rates
Jokes
List of freely available Programming Books
Mandelbrot Set with SIMD Intrinsics

*NIX & Hacking —— 创刊号

做一本我感兴趣的杂志,就这么简单!

C

Docker

Golang

Kernel

Lua

Rust

Unix

Vim

X86

Easter egg

Bash shell进程问题浅析(续)

上文,再来看一下exec这个bash shell内置命令:

exec [-cl] [-a name] [command [arguments]]
              If command is specified, it replaces the shell.  No new process is created.  The arguments become the arguments to command.  If the -l option is supplied, the shell places a dash at the beginning of the zeroth argument passed to command.  This is what login(1) does.  The -c option causes command  to  be executed with an empty environment.  If -a is supplied, the shell passes name as the zeroth argument to the executed command.  If command cannot be executed for some reason,  a  non-interactive  shell  exits,  unless the shell option execfail is enabled, in which case it returns failure.  An interactive shell returns failure if the file cannot be executed.  If command is not specified,  any  redirections  take  effect in the current shell, and the return status is 0.  If there is a redirection error, the return status is 1.  

可以看到,当用exec执行一个命令时,不会产生新的进程,并且这个命令会替换掉当前的bash shell进程。让我们看个例子。在一个终端执行下列命令:

[root@localhost ~]# echo $$
22330
[root@localhost ~]# exec sleep 60

再在另一个终端执行下列命令:

[root@localhost ~]# ps -ef | grep 22330
root     22330 22329  0 05:50 pts/0    00:00:00 sleep 60
root     22361 22345  0 05:52 pts/1    00:00:00 grep --color=auto 22330

可以看到22330号进程变成了sleep 60,而不是bash shell进程了。60秒后,sleep 60进程结束了,终端也退出了:

[root@localhost ~]# ps -ef | grep 22330
root     22363 22345  0 05:56 pts/1    00:00:00 grep --color=auto 22330

最后,通过演示经典的《Shell十三问》《exec跟source差在哪?》一章结尾的例子,再好好理解一下bash shell进程的相关问题:
1.sh

#!/bin/bash
A=B
echo "PID for 1.sh before exec/source/fork:$$"
export A
echo "1.sh: \$A is $A"
case $1 in
    exec)
        echo "using exec..."
        exec ./2.sh ;;
    source)
        echo "using source..."
        . ./2.sh ;;
    *)
        echo "using fork by default..."
        ./2.sh ;;
esac
echo "PID for 1.sh after exec/source/fork:$$"
echo "1.sh: \$A is $A"

2.sh:

#!/bin/bash
echo "PID for 2.sh: $$"
echo "2.sh get \$A=$A from 1.sh"
A=C
export A
echo "2.sh: \$A is $A"

(1)执行“./1.sh fork”:

[root@localhost ~]# ./1.sh fork
PID for 1.sh before exec/source/fork:22390
1.sh: $A is B
using fork by default...
PID for 2.sh: 22391
2.sh get $A=B from 1.sh
2.sh: $A is C
PID for 1.sh after exec/source/fork:22390
1.sh: $A is B

可以看到,由于1.sh脚本(进程ID22390)会新起一个subshell进程去执行2.sh(进程ID22391),所以在2.sh脚本中对A的修改不会影响到1.sh脚本中A的值。

(2)执行“./1.sh source”:

[root@localhost ~]# ./1.sh source  
PID for 1.sh before exec/source/fork:22393
1.sh: $A is B
using source...
PID for 2.sh: 22393
2.sh get $A=B from 1.sh
2.sh: $A is C
PID for 1.sh after exec/source/fork:22393
1.sh: $A is C

可以看到,由于2.sh脚本会在1.sh脚本进程中运行(打印出的进程ID均为22393),所以在2.sh脚本中对A的修改会影响到1.sh脚本中A的值。

(3)执行“./1.sh exec”:

[root@localhost ~]# ./1.sh exec
PID for 1.sh before exec/source/fork:22396
1.sh: $A is B
using exec...
PID for 2.sh: 22396
2.sh get $A=B from 1.sh
2.sh: $A is C

2.sh脚本会在1.sh脚本进程中运行(打印出的进程ID均为22396),同时原有的1.sh脚本进程不会再运行。所以2.sh脚本运行结束后,不会再执行1.sh脚本的命令。

参考资料:
Shell十三问

Bash shell进程问题浅析

bash shell中执行一个命令时,其实是由bash shell fork出一个子进程,然后在这个子进程中运行相应的命令,直至退出。在一个终端执行下列操作:

[root@localhost ~]# echo $$
19954
[root@localhost ~]# sleep 100

在另一个终端执行下列操作:

[root@localhost bin]# ps -ef | grep 19954
root     19954 19353  0 03:01 pts/3    00:00:00 /bin/bash
root     20265 19954  0 04:39 pts/3    00:00:00 sleep 100
root     20267 19354  0 04:39 pts/0    00:00:00 grep --color=auto 19954

可以看到第一个终端的bash shell(进程ID19954fork产生了sleep 100这个进程。

当在bash shell中执行一个bash shell脚本时,其实先会fork出一个subshell子进程,再由这个子进程执行脚本中的命令。举个例子可能会解释的更清楚。

这是一个简单的bash shell脚本(test.sh):

#!/bin/bash
sleep 100

在一个终端执行这个脚本:

[root@localhost ~]# echo $$
19954
[root@localhost ~]# ./test.sh

在另一个终端执行下列操作:

[root@localhost bin]# ps -ef | grep test
root     20309 19954  0 05:01 pts/3    00:00:00 /bin/bash ./test.sh
root     20316 19354  0 05:02 pts/0    00:00:00 grep --color=auto test
[root@localhost bin]# ps -ef | grep 20309
root     20309 19954  0 05:01 pts/3    00:00:00 /bin/bash ./test.sh
root     20310 20309  0 05:01 pts/3    00:00:00 sleep 100
root     20318 19354  0 05:02 pts/0    00:00:00 grep --color=auto 20309

可以看到第一个终端的bash shell(进程ID19954fork产生了test.sh这个进程(进程ID20309),而test.sh这个进程又fork产生了sleep 100这个进程。

接下来要提一下source(“.”命令作用是一样的)这个bash shell内建命令。man source是这样解释的:

.  filename [arguments]
source filename [arguments]
      Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename.  If filename does not contain a slash, file names in  PATH  are used to find the directory containing filename.  The file searched for in PATH need not be executable.  When bash is not in posix mode, the current directory is searched if no  file is  found in PATH.  If the sourcepath option to the shopt builtin command is turned off, the PATH is not searched.  If any arguments are supplied, they become the positional parameters when filename  is  executed.  Otherwise the positional parameters are unchanged.  The return status is the status of the last command exited within the script (0 if no commands are executed), and false if filename is not found or cannot be read.

可以看出,“source filename [arguments]”会在当前的shell环境执行文件中的命令,也就是不会产生一个subshell子进程。仍利用test.sh脚本演示一下:

在一个终端执行这个脚本:

[root@localhost ~]# echo $$
19954
[root@localhost ~]# source ./test.sh

在另一个终端执行下列操作:

[root@localhost bin]# ps -ef | grep test
root     20349 19354  0 05:24 pts/0    00:00:00 grep --color=auto test
[root@localhost bin]# ps -ef | grep 19954
root     19954 19353  0 03:01 pts/3    00:00:00 /bin/bash
root     20345 19954  0 05:24 pts/3    00:00:00 sleep 100
root     20347 19354  0 05:24 pts/0    00:00:00 grep --color=auto 19954

可以看到并没有test.sh这个进程,sleep 100这个进程是由bash shell(进程ID19954)进程直接fork产生的。

参考资料:
Shell十三问