使用LXC初体验

我使用的OSCentOS 7.1,需要安装lxclxc-templates。安装后的模板在/usr/share/lxc/templates目录下:

# ls
lxc-alpine    lxc-archlinux  lxc-centos  lxc-debian    lxc-fedora  lxc-openmandriva  lxc-oracle  lxc-sshd    lxc-ubuntu-cloud
lxc-altlinux  lxc-busybox    lxc-cirros  lxc-download  lxc-gentoo  lxc-opensuse      lxc-plamo   lxc-ubuntu

接下来以CentOS为模板创建一个container

lxc-create -t centos --name cn-centos

临时的root密码存在/var/lib/lxc/cn-01/tmp_root_pass

# cat /var/lib/lxc/cn-centos/tmp_root_pass
Root-cn-centos-EXb6bB

启动container

# lxc-start -n cn-centos

停止container

# lxc-stop -n cn-centos

参考资料:
Setup Linux Containers Using LXC On Ubuntu 15.04

 

LXC,cgroups和namespace简介

LXC is a userspace interface for the Linux kernel containment features. Through a powerful API and simple tools, it lets Linux users easily create and manage system or application containers.

The linux containers, lxc, aims to use these new functionalities to provide a userspace container object which provides full resource isolation and resource control for an application or a system.

Linux container技术的目标是为应用程序或系统提供完整的资源隔离和控制。LXC项目通过提供一组API接口和工具,可以让其他程序方便地使用Linux container技术。

The container technology is actively being pushed into the mainstream linux kernel. It provides the resource management through the control groups aka process containers and resource isolation through the namespaces.

Linux container技术cgroups(control groups)namespaces实现。两者的功能如下:

cgroups = limits how much you can use;
namespaces = limits what you can see (and therefore use)

Cgroups限制了你能够拥有的资源,而namespces限制了你能够看到的资源。

参考资料:
LXC
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic

 

SLES12版本上Xen的一些变化

今天在网上找到一篇文档,描述了SLES12版本上Xen的一些变化:

(1)grub2
SLES12上使用grub2配置Xen参数。以配置dom0_max_vcpus为例:
a)SLES11使用/boot/grub/menu.lst(参考这里);
b)SLES12使用/etc/default/grub(参考这里)。

(2)xm/xend已经废弃了,要使用xl/libxl。下图做了一个简单对比:

Capture

(3)SLES12 Xen Stack

• Xen 4.4.1  
• kernel 3.12.x  
• libvirt 1.2.5  
• virt-install 1.1.x, vm-install 1.x.x  
• virt-manager 1.1.x  

 

 

Crash工具笔记 (3)—— 在Xen环境使用crash

这两周一直在crash邮件列表里讨论如何在SuSE Xen上使用crash调试Dom0 kernel。邮件来来回回讨论很多(参见这里),最后还发现了一个bug。细节不说了,把最后的结果总结一下:

(1)由于SuSE kerenl默认编译打开CONFIG_STRICT_DEVMEM编译开关,所以crash工具无法完全访问/dev/mem,可以使用/proc/kcore作为代替;

(2)SuSE带有crash.ko驱动(位于:“/lib/modules/uname -r/updates/crash.ko”),但默认没有安装,可以自己手动安装(使用insmod命令),然后就可以使用了:

# crash

crash 7.1.3
Copyright (C) 2002-2014  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

crash: /boot/xen-4.5.gz: original filename unknown
       Use "-f /boot/xen-4.5.gz" on command line to prevent this message.

WARNING: machine type mismatch:
         crash utility: X86_64
         /var/tmp/xen-4.5.gz_ud3IRy: X86

crash: /boot/symtypes-3.12.49-6-default.gz: original filename unknown
       Use "-f /boot/symtypes-3.12.49-6-default.gz" on command line to
prevent this message.

crash: /boot/symvers-3.12.49-6-default.gz: original filename unknown
       Use "-f /boot/symvers-3.12.49-6-default.gz" on command line to
prevent this message.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: /boot/vmlinux-3.12.49-6-xen.gz
   DEBUGINFO: /usr/lib/debug/boot/vmlinux-3.12.49-6-xen.debug
    DUMPFILE: /dev/crash
        CPUS: 128
        DATE: Fri Nov 20 06:55:06 2015
      UPTIME: 18:51:36
LOAD AVERAGE: 1.76, 1.48, 1.21
       TASKS: 1230
    NODENAME: dl980-5
     RELEASE: 3.12.49-6-xen
     VERSION: #1 SMP Mon Oct 26 16:05:37 UTC 2015 (11560c3)
     MACHINE: x86_64  (1995 Mhz)
      MEMORY: 125.9 GB
         PID: 6618
     COMMAND: "crash"
        TASK: ffff881ea93b2140  [THREAD_INFO: ffff881e869f2000]
         CPU: 112
       STATE: TASK_RUNNING (ACTIVE)

 

libvirt笔记 (4) —— log配置

libvirt库通过以下三个环境变量配置log

The library configuration of logging is through 3 environment variables allowing to control the logging behaviour:

LIBVIRT_DEBUG: it can take the four following values:

1 or “debug”: asking the library to log every message emitted, though the filters can be used to avoid filling up the output

2 or “info”: log all non-debugging information

3 or “warn”: log warnings and errors, that’s the default value

4 or “error”: log only error messages

LIBVIRTLOGFILTERS: defines logging filters

LIBVIRTLOGOUTPUTS: defines logging outputs

Note that, for example, setting LIBVIRT_DEBUG= is the same as unset. If you specify an invalid value, it will be ignored with a warning. If you have an error in a filter or output string, some of the settings may be applied up to the point at which libvirt encountered the error.

libvirtd daemon程序也有三个类似的配置项(存储在配置文件libvirtd.conf):

log_level: accepts the following values:

4: only errors

3: warnings and errors

2: information, warnings and errors

1: debug and everything

log_filters: defines logging filters

log_outputs: defines logging outputs

对于libvirtd程序来讲,log配置项的优先级如下:

When starting the libvirt daemon, any logging environment variable settings will override settings in the config file. Command line options take precedence over all. If no outputs are defined for libvirtd, it will try to use

0.10.0 or later: systemd journal, if /run/systemd/journal/socket exists 0.9.0 or later: file /var/log/libvirt/libvirtd.log if running as a daemon before 0.9.0: syslog if running as a daemon all versions: to stderr stream if running in the foreground

参考资料:
Logging in the library and the daemon

 

libvirt笔记 (3) —— 得到virtualization host的能力信息

getCapabilities方法得到一个字符串,用来描述virtualization host的能力,以及能创建什么样的Guest OS。请看下面代码:

#!/usr/bin/python

from __future__ import print_function
import sys
import libvirt

conn = libvirt.open('xen:///')
if conn == None:
    print('Failed to open connection to xen:///', file=sys.stderr)
    exit(1)

caps = conn.getCapabilities() # caps will be a string of XML
print('Capabilities:\n'+caps)

conn.close()
exit(0)

执行如下:

Capabilities:
<capabilities>

  <host>
    <cpu>
      <arch>x86_64</arch>
      <features>
        <pae/>
      </features>
    </cpu>
    <power_management/>
    <migration_features>
      <live/>
    </migration_features>
    <topology>
      <cells num='1'>
        <cell id='0'>
          <memory unit='KiB'>1048512</memory>
          <cpus num='0'>
          </cpus>
        </cell>
      </cells>
    </topology>
  </host>

  <guest>
    <os_type>xen</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/lib/xen/bin/qemu-system-i386</emulator>
      <machine>xenpv</machine>
      <domain type='xen'/>
    </arch>
  </guest>

  <guest>
    <os_type>xen</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/lib/xen/bin/qemu-system-i386</emulator>
      <machine>xenpv</machine>
      <domain type='xen'/>
    </arch>
    <features>
      <pae/>
    </features>
  </guest>

</capabilities>

参考资料:
Capability information

 

libvirt笔记 (2) —— Hypervisor connections

Hypervisor connectionlibvirt一个核心概念(以下内容摘自这里):

A connection is the primary or top level object in the libvirt API and Python libvirt module. An instance of this object is required before attempting to use almost any of the classes or methods. A connection is associated with a particular hypervisor, which may be running locally on the same machine as the libvirt client application, or on a remote machine over the network. In all cases, the connection is represented by an instance of the virConnect class and identified by a URI. The URI scheme and path defines the hypervisor to connect to, while the host part of the URI determines where it is located.

An application is permitted to open multiple connections at the same time, even when using more than one type of hypervisor on a single machine. For example, a host may provide both KVM full machine virtualization and LXC container virtualization. A connection object may be used concurrently across multiple threads. Once a connection has been established, it is possible to obtain handles to other managed objects or create new managed objects.

以下代码测试Xen连接:

#!/usr/bin/python
from __future__ import print_function
import sys
import libvirt

conn = libvirt.open('xen:///')
if conn == None:
        print('Failed to open connection to xen:///', file=sys.stderr)
        exit(1)
else:
        print('Open connection success', file=sys.stdout)
        conn.close()
        exit(0)

 

libvirt笔记 (1) —— 术语

以下内容摘自Libvert terminology and goals

a node is a single physical machine

an hypervisor is a layer of software allowing to virtualize a node in a set of virtual machines with possibly different configurations than the node itself

a domain is an instance of an operating system (or subsystem in the case of container virtualization) running on a virtualized machine provided by the hypervisor Hypervisor and domains running on a node

 

node

Now we can define the goal of libvirt: to provide a common and stable layer sufficient to securely manage domains on a node, possibly remote.

libvirt中,node即指物理机器,domain可以理解为虚拟机。

 

Xen 笔记 (1)——为什么xen是32位的可执行程序?

今天编译了一下Xen,发现编出来的xen32位的可执行程序,而xen-syms却是64位的:

Linux:~/Downloads/xen-4.6.0/xen # file xen
xen: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped
Linux:~/Downloads/xen-4.6.0/xen # file xen-syms
xen-syms: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

Roger在邮件里给了答案:

The Xen entry point is in 32bits (because that’s what the multiboot specification requires). Xen then jumps into long mode (64bits) by itself, so there’s only a very small amount of 32bit code that’s used as a trampoline.

原来是为了支持multibootxen实际运行以后还是64位程序。

参考资料:
[Xen-users] Why the built xen file is 32-bit on 64-bit OS?

 

virt-manager/virsh调试技巧(不断更新)

(1)使用virt-manager --trace-libvirt --debug可以输出virt-manager的调试信息。

Linux:~ # virt-manager --trace-libvirt --debug
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (cli:246) Launched with command line: /usr/share/virt-manager/virt-manager --trace-libvirt --debug
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (virt-manager:153) virt-manager version: 1.2.1
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (virt-manager:154) virtManager import: <module 'virtManager' from '/usr/share/virt-manager/virtManager/__init__.pyc'>
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (virt-manager:157) Libvirt tracing requested
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (module_trace:66) wrapfunc <function _dispatchEventHandleCallback at 0x7ff4391fc050> _dispatchEventHandleCallback
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (module_trace:66) wrapfunc <function _dispatchEventTimeoutCallback at 0x7ff4391fc0c8> _dispatchEventTimeoutCallback
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (module_trace:66) wrapfunc <function _eventInvokeHandleCallback at 0x7ff439251ed8> _eventInvokeHandleCallback
[Thu, 22 Oct 2015 13:54:08 virt-manager 6124] DEBUG (module_trace:66) wrapfunc <function _eventInvokeTimeoutCallback at 0x7ff439251f50> _eventInvokeTimeoutCallback
......

也可以重定向输出到文件:

Linux:~ # virt-manager --trace-libvirt --debug > log.txt 2>&1

(2)通过virsh输出Guest OS日志:

Linux:~ # virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list
 Id    Name                           State
----------------------------------------------------
 0     Domain-0                       running
 3     sles11sp4-i686                 running

virsh # console 3
Connected to domain sles11sp4-i686
Escape character is ^]
[    0.000000] Reserving virtual address space above 0xf5800000
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.0.101-63-xen (geeko@buildhost) (gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux) ) #1 SMP Tue Jun 23 16:02:31 UTC 2015 (4b89d0c)
......