Monthly Archives: July 2016

Fix “please format Go code with ‘gofmt -s'” issue in building Swarmkit

Published / by nanxiao / Leave a Comment

When you do modifications in Swarmkit source code, e.g., doc.go, sometimes executing “make” command will show the following errors:

# make
 fmt
doc.go
 please format Go code with 'gofmt -s'

But executing “gofmt -s” command on doc.go seems not take effect:

# gofmt -s doc.go
// Package swarmkit implements a framework for task orchestration.
package swarmkit
# make
 fmt
doc.go
 please format Go code with 'gofmt -s'
make: *** [fmt] Error 1

The solution should be running “go fmt” command on this file, then the build process is OK:

# go fmt doc.go
doc.go
# make
 fmt
 bin/swarmd
......

Wget only recognizes http_proxy, not https_proxy

Published / by nanxiao / Leave a Comment

My Ubuntu 16.04 LTS works behind proxy. I have set HTTP_PROXY and HTTPS_PROXY environmental variables:

HTTP_PROXY=http://web-proxy.corp.xxxxxx.com:8080/
HTTPS_PROXY=https://web-proxy.corp.xxxxx.com:8080/

But wget can’t work:

# wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
--2016-07-14 22:51:12--  https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
Resolving github.com (github.com)... 192.30.253.112
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
ERROR: cannot verify github.com's certificate, issued by ‘O=Fortinet Ltd.,CN=FG3K6C3A15800021’:
  Self-signed certificate encountered.
    ERROR: certificate common name ‘FG3K6C3A15800021’ doesn't match requested host name ‘github.com’.
To connect to github.com insecurely, use `--no-check-certificate'.
root@ubuntu:~# wget --no-check-certificate -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb

After setting http_proxy and https_proxy:

http_proxy=http://web-proxy.corp.xxxxxx.com:8080/
https_proxy=https://web-proxy.corp.xxxxx.com:8080/
HTTP_PROXY=http://web-proxy.corp.xxxxxx.com:8080/
HTTPS_PROXY=https://web-proxy.corp.xxxxx.com:8080/

Now wget works:

# wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb     --2016-07-14 22:57:30--  https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
Resolving web-proxy.xxxxxx.hp.com (web-proxy.xxxxxx.hp.com)... xxx.xxx.xxx.xxx
Connecting to web-proxy.xxxxxx.hp.com (web-proxy.xxxxxx.hp.com)|xxx.xxx.xxx.xxx|:8080... connected.
Proxy request sent, awaiting response... 302 Found
......

So we can conclude that wget is picky about uppercase and lowercase words.

 

Build docker from source behind proxy

Published / by nanxiao / 3 Comments on Build docker from source behind proxy

If you want to build Docker from source like this:

# git clone https://github.com/docker/docker.git
# cd docker
# make

But your working server is actually behind a proxy, I think you may run into errors as the follows:

# make
mkdir bundles
docker build  -t "docker-dev:master" -f "Dockerfile" .
Sending build context to Docker daemon 145.8 MB
Step 1 : FROM debian:jessie
 ---> f854eed3f31f
Step 2 : RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61  || apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61
 ---> Running in fede03b56767
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.MjO7kIEOm8 --no-auto-check-trustdb --trust-model always --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-stable.gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61
gpg: requesting key F6B0FC61 from hkp server p80.pool.sks-keyservers.net
gpgkeys: key E871F18B51E0147C77796AC81196BA81F6B0FC61 can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.6clUfj4AwL --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-squeeze-stable.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-automatic.gpg --keyring /etc/apt/trusted.gpg.d/debian-archive-wheezy-stable.gpg --keyserver hkp://pgp.mit.edu:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61
gpg: requesting key F6B0FC61 from hkp server pgp.mit.edu
gpgkeys: key E871F18B51E0147C77796AC81196BA81F6B0FC61 can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
The command '/bin/sh -c apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61 || apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys E871F18B51E0147C77796AC81196BA81F6B0FC61' returned a non-zero code: 2
Makefile:70: recipe for target 'build' failed
make: *** [build] Error 1

Or:

......
RUN apt-get update && apt-get install -y       apparmor        apt-utils       aufs-tools      automake        bash-completion   binutils-mingw-w64       bsdmainutils    btrfs-tools     build-essential         clang   createrepo      curl    dpkg-sig        gcc-mingw-w64      git     iptables        jq      libapparmor-dev         libcap-dev      libltdl-dev     libsqlite3-dev  libsystemd-journal-dev  libtool    mercurial       net-tools       pkg-config      python-dev      python-mock     python-pip      python-websocket        ubuntu-zfs xfsprogs        libzfs-dev      tar     zip     --no-install-recommends         && pip install awscli==1.10.15
 ---> Running in 37080c364862
Get:1 http://ppa.launchpad.net trusty InRelease [8127 B]
Get:2 http://httpredir.debian.org jessie InRelease [8127 B]
Get:3 http://security.debian.org jessie/updates InRelease [8127 B]
Splitting up /var/lib/apt/lists/partial/ppa.launchpad.net_zfs-native_stable_ubuntu_dists_trusty_InRelease into data and signature failedIgn http://ppa.launchpad.net trusty InRelease
E: GPG error: http://ppa.launchpad.net trusty InRelease: Clearsigned file isn't valid, got 'NODATA' (does the network require authentication?)
The command '/bin/sh -c apt-get update && apt-get install -y    apparmor        apt-utils       aufs-tools      automake        bash-completion    binutils-mingw-w64      bsdmainutils    btrfs-tools     build-essential         clang   createrepo      curl    dpkg-sig        gcc-mingw-w64      git     iptables        jq      libapparmor-dev         libcap-dev      libltdl-dev     libsqlite3-dev  libsystemd-journal-dev     libtool         mercurial       net-tools       pkg-config      python-dev      python-mock     python-pip      python-websocket  ubuntu-zfs       xfsprogs        libzfs-dev      tar     zip     --no-install-recommends         && pip install awscli==1.10.15' returned a non-zero code: 100
make: *** [build] Error 1

These reports can make you crazy!

The solution is adding proxy into Dockerfile which resides in the root directory of Docker folder:

......
FROM debian:jessie
ENV http_proxy http://web-proxy.corp.xxxxxx.com:8080/
ENV https_proxy https://web-proxy.corp.xxxxxx.com:8080/
......

Then the make progress will be smooth!

P.S., after discussing in reddit, the correct and idiomatic method should be this:

make DOCKER_BUILD_ARGS="--build-arg http_proxy=http://web-proxy.corp.xxxxxx.com:8080/ --build-arg https_proxy=https://web-proxy.corp.xxxxxx.com:8080/"