Tag: Ubuntu

Downgrade boost on Ubuntu 16.04

In the past 2 days, I was tortured by boost. The default boostversion on Ubuntu 16.04 is 1.58, but I met following compile errors:

......
/usr/include/boost/multi_index/detail/bucket_array.hpp: In static member function ‘static std::size_t boost::multi_index::detail::bucket_array_base<_>::size_index(std::size_t)’:
/usr/include/boost/multi_index/detail/bucket_array.hpp:84:62: error: invalid use of non-lvalue array
     const std::size_t *bound=std::lower_bound(sizes,sizes+sizes_length,n);
                                                              ^~~~~~~~~~~~
/usr/include/boost/multi_index/detail/bucket_array.hpp:85:25: error: invalid use of non-lvalue array
     if(bound==sizes+sizes_length)--bound;
                         ^~~~~~~~~~~~
/usr/include/boost/multi_index/detail/bucket_array.hpp:86:22: error: invalid use of non-lvalue array
     return bound-sizes;
......

I downgraded boost to 1.55; and downloaded and built it:

$ ./bootstrap.sh --prefix=/usr/local  
$ sudo ./b2 -a -q install

This time I found the default gcc-5 could not compile successfully. So I followed this post to install gcc-6, and modified/home/nan/boost_1_55_0/tools/build/v2/user-config.jam file to use gcc-6 to compile boost:

......
# Configure specific gcc version, giving alternative name to use.
using gcc : 6 : g++-6 ;
......

Then my project can be compiled successfully. Check gcc search header file path and library path:

$ echo | gcc-6 -E -Wp,-v -
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/6/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/6/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
......

$ ldconfig -v 2>/dev/null | grep -v ^$'\t'
/usr/local/cuda-9.0/targets/x86_64-linux/lib:
/usr/lib/x86_64-linux-gnu/libfakeroot:
/usr/local/lib:
/lib/x86_64-linux-gnu:
/usr/lib/x86_64-linux-gnu:
/usr/lib/nvidia-384:
/usr/lib32/nvidia-384:
/lib32:
/usr/lib32:
/lib:
/usr/lib:
/usr/lib/nvidia-384/tls: (hwcap: 0x8000000000000000)
/usr/lib32/nvidia-384/tls: (hwcap: 0x8000000000000000)

You will find the fresh installed boost 1.55 (in /usr/local directory) always be found before default boost 1.58 (header files are in /usr/include/boostand libraries in /usr/lib/x86_64-linux-gnu).

Porting CUDA program from ArchLinux to Ubuntu 16.04

Today I ported a CUDA project from Arch Linux to Ubuntu 16.04, and this post records the pitfalls.

(0) Update cmake to newest version (follow this);

(1) Specify nvcc path in CMakeLists.txt:

SET(CMAKE_CUDA_COMPILER /usr/local/cuda-9.0/bin/nvcc)

otherwise, following error may generate:

......
No CMAKE_CUDA_COMPILER could be found.
......

(2) Since Ubuntu 16.04‘s default compiler is still gcc-5, install gcc-6 first, then pass gcc-6 as default compiler for nvcc:

......
SET(CMAKE_CXX_STANDARD 11)
SET(CMAKE_CUDA_FLAGS "-std=c++11 -ccbin gcc-6")
......

(3) Execute cmake command:

cmake -DCMAKE_C_COMPILER=gcc-6 -DCMAKE_CXX_COMPILER=g++-6 -DCMAKE_BUILD_TYPE=Release ..

References:
Not Locating CUDA Compiler;
CMake: How to pass mode dependent compile flags to nvcc in visual studio environment;
Tensorflow crashes on build on Ubuntu 16.04 when building for skylake (avx512).

 

Notice the linking library position on Ubuntu

This week, I ported tcpbench from OpenBSD to Linux. The idiomatic method of OpenBSD is putting the linking library in front of generating final target:

cc -g -O2 -Wall -levent -o tcpbench tcpbench.c

However this doesn’t work in Ubuntu since its the linker uses --as-needed option. So I change the Makefile to put the library at the end:

cc -g -O2 -Wall -o tcpbench tcpbench.c -levent

Please refer this discussion if you are interested.

Build the newest Docker environment

This tutorial explains how to build the newest Docker environment. My host is Ubuntu 16.04.1, and it is already shipped withDocker 1.12.0:

# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
   └─http-proxy.conf
   Active: active (running) since Tue 2016-08-09 03:49:08 EDT; 3min 24s ago
 Docs: https://docs.docker.com
 Main PID: 30465 (dockerd)
Tasks: 26
   Memory: 36.5M
  CPU: 2.394s
   CGroup: /system.slice/docker.service
   ├─30465 /usr/bin/dockerd -H fd://
   └─30473 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics

Aug 09 03:49:08 ubuntu dockerd[30465]: time="2016-08-09T03:49:08.114671045-04:00" level=info msg="Graph migration to content-addressability
......
# docker version
Client:
 Version:  1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:Thu Jul 28 22:11:10 2016
 OS/Arch:  linux/amd64

Server:
 Version:  1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:Thu Jul 28 22:11:10 2016
 OS/Arch:  linux/amd64

(1) The prerequisite is the Go environment is ready on your host, and GOPATH environment variable is also set. If not, please follow thisdocument to setup.

(2) Download the newest Docker code:

# go get -d -u github.com/docker/docker
package github.com/docker/docker: no buildable Go source files in /go/src/github.com/docker/docker

Build the Docker:

# cd $GOPATH/src/github.com/docker/docker/
# make DOCKER_BUILD_ARGS="--build-arg http_proxy=http://web-proxy.corp.xxxxxx.com:8080/ --build-arg https_proxy=https://web-proxy.corp.xxxxxx.com:8080/" DOCKER_DEBUG=1

Because my host works behind proxy, I need to specify proxy address in command line. Whether adding DOCKER_DEBUG or not depends on your personal flavor.

(3) After above building process succeeds, backup old Docker files:

# systemctl stop docker
# cd /usr/bin
# mkdir backup_docker
# mv docker* backup_docker

(4) Change back to $GOPATH/src/github.com/docker/docker/, and copy new Docker binaries:

# cd $GOPATH/src/github.com/docker/docker/
# cd bundles/latest/
# ls
binary-client  binary-daemon

binary-client contains Docker executable file:

# cd binary-client/
# ls
docker  docker-1.13.0-dev  docker-1.13.0-dev.md5  docker-1.13.0-dev.sha256
# cp docker /usr/bin/

Then copy Docker daemon related files:

# cd ../binary-daemon/
# ls
docker-containerd             docker-containerd.sha256       dockerd-1.13.0-dev         docker-proxy-1.13.0-dev.md5
docker-containerd-ctr         docker-containerd-shim         dockerd-1.13.0-dev.md5     docker-proxy-1.13.0-dev.sha256
docker-containerd-ctr.md5     docker-containerd-shim.md5     dockerd-1.13.0-dev.sha256  docker-runc
docker-containerd-ctr.sha256  docker-containerd-shim.sha256  docker-proxy               docker-runc.md5
docker-containerd.md5         dockerd                        docker-proxy-1.13.0-dev    docker-runc.sha256
# cp docker-containerd docker-containerd-ctr docker-containerd-shim docker-runc dockerd docker-proxy /usr/bin/

(5) Restart Docker and check it:

# systemctl start docker
# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─http-proxy.conf
   Active: active (running) since Tue 2016-08-09 04:26:16 EDT; 9s ago
     Docs: https://docs.docker.com
 Main PID: 4961 (dockerd)
    Tasks: 24
   Memory: 13.6M
      CPU: 367ms
   CGroup: /system.slice/docker.service
           ├─4961 /usr/bin/dockerd -H fd://
           └─4968 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-

Aug 09 04:26:15 ubuntu dockerd[4961]: time="2016-08-09T04:26:15.795281048-04:00" level=info msg="Graph migration to content-addressability
......
# docker version
Client:
 Version:      1.13.0-dev
 API version:  1.25
 Go version:   go1.6.3
 Git commit:   b2b41b2
 Built:        Tue Aug  9 07:49:54 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0-dev
 API version:  1.25
 Go version:   go1.6.3
 Git commit:   b2b41b2
 Built:        Tue Aug  9 07:49:54 2016
 OS/Arch:      linux/amd64

Now you are playing the freshest Docker! Enjoy it!

 

Wget only recognizes http_proxy, not https_proxy

My Ubuntu 16.04 LTS works behind proxy. I have set HTTP_PROXY and HTTPS_PROXY environmental variables:

HTTP_PROXY=http://web-proxy.corp.xxxxxx.com:8080/
HTTPS_PROXY=https://web-proxy.corp.xxxxx.com:8080/

But wget can’t work:

# wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
--2016-07-14 22:51:12--  https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
Resolving github.com (github.com)... 192.30.253.112
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
ERROR: cannot verify github.com's certificate, issued by ‘O=Fortinet Ltd.,CN=FG3K6C3A15800021’:
  Self-signed certificate encountered.
    ERROR: certificate common name ‘FG3K6C3A15800021’ doesn't match requested host name ‘github.com’.
To connect to github.com insecurely, use `--no-check-certificate'.
root@ubuntu:~# wget --no-check-certificate -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb

After setting http_proxy and https_proxy:

http_proxy=http://web-proxy.corp.xxxxxx.com:8080/
https_proxy=https://web-proxy.corp.xxxxx.com:8080/
HTTP_PROXY=http://web-proxy.corp.xxxxxx.com:8080/
HTTPS_PROXY=https://web-proxy.corp.xxxxx.com:8080/

Now wget works:

# wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb     --2016-07-14 22:57:30--  https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
Resolving web-proxy.xxxxxx.hp.com (web-proxy.xxxxxx.hp.com)... xxx.xxx.xxx.xxx
Connecting to web-proxy.xxxxxx.hp.com (web-proxy.xxxxxx.hp.com)|xxx.xxx.xxx.xxx|:8080... connected.
Proxy request sent, awaiting response... 302 Found
......

So we can conclude that wget is picky about uppercase and lowercase words.