Author: nanxiao

Append slice puzzle

This puzzle comes from go-traps.appspot.com:

package main

import "fmt"

func main() {
    a := make([]int, 3, 4)
    a[0], a[1], a[2] = 0, 1, 2

    b := append(a, 66)
    b[0] = 6
    c := append(a, 77)
    c[0] = 7
    d := append(a, 88, 99)
    d[0] = 9

    fmt.Println(a)
    fmt.Println(b)
    fmt.Println(c)
    fmt.Println(d)
}

I think this is a good tutorial to explain the nitty-gritty behind the slice in Go, so I will analyze this issue detailedly. But before our journey, let’s get into some preliminaries of slice first:

The internals of a slice is like the following diagram:

1

A slice consists of 3 components:
a) ptr: A pointer which points to the start position of slice in the underlying array;
b) len (also known as length, and its type is int): the number of the valid elements of the slice;
b) cap (also known as capacity, and its type is int): the total number of slots of the slice.

Now that we have known the construction of the slice, we can explain the problem step by step:
(1)

a := make([]int, 3, 4)
a[0], a[1], a[2] = 0, 1, 2

The above statements are very general. Now the a slice is as follows:

2

(2)

b := append(a, 66)

Since the capacity of a is 4, and the length of a is 3, so a has a free slot for storing the new element: 66. Besides this, the operation of “append(a, 66)” assigns to a new variable: b, so the value of a doesn’t change: len is still 3; cap is still 4; ptr still points the original array, but the content of original array is modified: the 4th element is 66 now. Because b is the assignee of “append(a, 66)“, it points to the same underlying array of a, but The len of b is 4:

3

After running “b[0] = 6“, now the memory layout is like this:

4

(3)

c := append(a, 77)
c[0] = 7

Since the len of a is 3 up to this time, the program will consider there is an available slot. The result is the 4th element in the array will be changed to 77 from 66. “c[0] = 7” will modify the 1st element. Now the ptrs of a, b and c all point to the same array:

5

(4)

d := append(a, 88, 99)
d[0] = 9

Since a only has 1 accessible slot, “append(a, 88, 99)” will relocating a new array (the size will be doubled to be be more efficient, that is from 4 to 8 in this case.). So now the ptr of d points to a new array, and the final result should be here:

6

To wrap out this discussion, I want to quote the following statement from go-traps.appspot.com:

A good rule of thumb for non-specialists would be “never call append without assigning back to the same variable”.

So if you really need using append but without assigning back the original value, please be more careful, or else it may bite you and give you a big unpleasant surprise!

Why do I need a debugger?

When I begin to learn a new programming language, I will try and master the debugger for it as early as possible. For example, in 2013, while I touched the Go, there seems only gdb for use. Although gdb itself is not a good choice (From Debugging Go Code with GDB):

As a consequence, although GDB can be useful in some situations, it is not a reliable debugger for Go programs, particularly heavily concurrent ones.

But at that time there was no other choice. So after delve came out, I switched to it without hesitation. Though I am not an expert of delvecode, I still try my best do some contributions to make delve become better: improve documents, report issues, etc. Why am I so keen on debugger? The answer is it is a really irreplaceable and necessary tool for software engineers. The reasons are as follows:

(1) If the print statements are the only debugging method of a programming language, it will make feel upset. I.e., if a bug is fully reproduceable, but from the logs you can’t figure out the reason. If there is a debugger which can help you step into every statement and inspect value of every variable, I think you can get the root cause quickly.

Certainly, the debugger isn’t omnipotent. E.g., the nasty multi-threads bug (If you are interested in this topic, you can read this post which describes an experience I have undergone.). If you can’t find reproduce condition of this issue, and adding logs still fails, you can try to add some assert statements which are triggered when the issue happens again. Then you can get the core dump file which records the scene of crime, and use debugger to analyze it. You can see the debugger plays an important part in this scenario yet.

(2) “Debugger is a perfect unit-test tool”, the words come form my director when I got my first full-time job after leaving school. The reason is when you finish a code segment, you can use debugger to check whether it is correct through step-in mode: check value of every variable, mock the conditions which can’t easily be constructed in black-box test, inspect the stack and registers, etc. By means of this, you can fix many corner bugs.

(3) Debugger is a good tool to help you understand code. When I dive into some big Go projects, I find so many channels, interfaces, goroutines, and they sometimes make me crazy. But by way of using debugger, I can set breakpoints, then once the program is stopped, I can understand the code logic better through watching the calling stack.

Based on the above, debugger is an invaluable tool for everyone who lives on writing code. Try know and master it better. Maybe one day, a colleague can’t find reasons for one bug, then you use a small debugger trick and spot the root cause immediately. Isn’t it a cool stuff? 🙂

Build the newest Docker environment

This tutorial explains how to build the newest Docker environment. My host is Ubuntu 16.04.1, and it is already shipped withDocker 1.12.0:

# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
   └─http-proxy.conf
   Active: active (running) since Tue 2016-08-09 03:49:08 EDT; 3min 24s ago
 Docs: https://docs.docker.com
 Main PID: 30465 (dockerd)
Tasks: 26
   Memory: 36.5M
  CPU: 2.394s
   CGroup: /system.slice/docker.service
   ├─30465 /usr/bin/dockerd -H fd://
   └─30473 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics

Aug 09 03:49:08 ubuntu dockerd[30465]: time="2016-08-09T03:49:08.114671045-04:00" level=info msg="Graph migration to content-addressability
......
# docker version
Client:
 Version:  1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:Thu Jul 28 22:11:10 2016
 OS/Arch:  linux/amd64

Server:
 Version:  1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:Thu Jul 28 22:11:10 2016
 OS/Arch:  linux/amd64

(1) The prerequisite is the Go environment is ready on your host, and GOPATH environment variable is also set. If not, please follow thisdocument to setup.

(2) Download the newest Docker code:

# go get -d -u github.com/docker/docker
package github.com/docker/docker: no buildable Go source files in /go/src/github.com/docker/docker

Build the Docker:

# cd $GOPATH/src/github.com/docker/docker/
# make DOCKER_BUILD_ARGS="--build-arg http_proxy=http://web-proxy.corp.xxxxxx.com:8080/ --build-arg https_proxy=https://web-proxy.corp.xxxxxx.com:8080/" DOCKER_DEBUG=1

Because my host works behind proxy, I need to specify proxy address in command line. Whether adding DOCKER_DEBUG or not depends on your personal flavor.

(3) After above building process succeeds, backup old Docker files:

# systemctl stop docker
# cd /usr/bin
# mkdir backup_docker
# mv docker* backup_docker

(4) Change back to $GOPATH/src/github.com/docker/docker/, and copy new Docker binaries:

# cd $GOPATH/src/github.com/docker/docker/
# cd bundles/latest/
# ls
binary-client  binary-daemon

binary-client contains Docker executable file:

# cd binary-client/
# ls
docker  docker-1.13.0-dev  docker-1.13.0-dev.md5  docker-1.13.0-dev.sha256
# cp docker /usr/bin/

Then copy Docker daemon related files:

# cd ../binary-daemon/
# ls
docker-containerd             docker-containerd.sha256       dockerd-1.13.0-dev         docker-proxy-1.13.0-dev.md5
docker-containerd-ctr         docker-containerd-shim         dockerd-1.13.0-dev.md5     docker-proxy-1.13.0-dev.sha256
docker-containerd-ctr.md5     docker-containerd-shim.md5     dockerd-1.13.0-dev.sha256  docker-runc
docker-containerd-ctr.sha256  docker-containerd-shim.sha256  docker-proxy               docker-runc.md5
docker-containerd.md5         dockerd                        docker-proxy-1.13.0-dev    docker-runc.sha256
# cp docker-containerd docker-containerd-ctr docker-containerd-shim docker-runc dockerd docker-proxy /usr/bin/

(5) Restart Docker and check it:

# systemctl start docker
# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─http-proxy.conf
   Active: active (running) since Tue 2016-08-09 04:26:16 EDT; 9s ago
     Docs: https://docs.docker.com
 Main PID: 4961 (dockerd)
    Tasks: 24
   Memory: 13.6M
      CPU: 367ms
   CGroup: /system.slice/docker.service
           ├─4961 /usr/bin/dockerd -H fd://
           └─4968 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-

Aug 09 04:26:15 ubuntu dockerd[4961]: time="2016-08-09T04:26:15.795281048-04:00" level=info msg="Graph migration to content-addressability
......
# docker version
Client:
 Version:      1.13.0-dev
 API version:  1.25
 Go version:   go1.6.3
 Git commit:   b2b41b2
 Built:        Tue Aug  9 07:49:54 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0-dev
 API version:  1.25
 Go version:   go1.6.3
 Git commit:   b2b41b2
 Built:        Tue Aug  9 07:49:54 2016
 OS/Arch:      linux/amd64

Now you are playing the freshest Docker! Enjoy it!

 

Fix “please format Go code with ‘gofmt -s'” issue in building Swarmkit

When you do modifications in Swarmkit source code, e.g., doc.go, sometimes executing “make” command will show the following errors:

# make
 fmt
doc.go
 please format Go code with 'gofmt -s'

But executing “gofmt -s” command on doc.go seems not take effect:

# gofmt -s doc.go
// Package swarmkit implements a framework for task orchestration.
package swarmkit
# make
 fmt
doc.go
 please format Go code with 'gofmt -s'
make: *** [fmt] Error 1

The solution should be running “go fmt” command on this file, then the build process is OK:

# go fmt doc.go
doc.go
# make
 fmt
 bin/swarmd
......