Mesos笔记 (9)—— Containerizer类代码解析

Containerizer类(定义在src/slave/containerizer/containerizer.hpp )是所有Containerizer的抽象父类。除了默认的构造函数和一个什么都没做的析构函数,其只实现了createresources方法。create方法代码如下(v0.26版本):

Try<Containerizer*> Containerizer::create(
    const Flags& flags,
    bool local,
    Fetcher* fetcher)
{
  if (flags.isolation == "external") {
    LOG(WARNING) << "The 'external' isolation flag is deprecated, "
                 << "please update your flags to"
                 << " '--containerizers=external'.";

    Try<ExternalContainerizer*> containerizer =
      ExternalContainerizer::create(flags);
    if (containerizer.isError()) {
      return Error("Could not create ExternalContainerizer: " +
                   containerizer.error());
    }

    return containerizer.get();
  }

  // TODO(benh): We need to store which containerizer or
  // containerizers were being used. See MESOS-1663.

  // Create containerizer(s).
  vector<Containerizer*> containerizers;

  foreach (const string& type, strings::split(flags.containerizers, ",")) {
    if (type == "mesos") {
      Try<MesosContainerizer*> containerizer =
        MesosContainerizer::create(flags, local, fetcher);
      if (containerizer.isError()) {
        return Error("Could not create MesosContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else if (type == "docker") {
      Try<DockerContainerizer*> containerizer =
        DockerContainerizer::create(flags, fetcher);
      if (containerizer.isError()) {
        return Error("Could not create DockerContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else if (type == "external") {
      Try<ExternalContainerizer*> containerizer =
        ExternalContainerizer::create(flags);
      if (containerizer.isError()) {
        return Error("Could not create ExternalContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else {
      return Error("Unknown or unsupported containerizer: " + type);
    }
  }

  if (containerizers.size() == 1) {
    return containerizers.front();
  }

  Try<ComposingContainerizer*> containerizer =
    ComposingContainerizer::create(containerizers);

  if (containerizer.isError()) {
    return Error(containerizer.error());
  }

  return containerizer.get();
}

默认情况下,containerizerstypemesos,所以会调用MesosContainerizer::create来生成containerizer。关于resources方法,参考Mesos笔记 (5)—— 资源

 

Mesos笔记 (8)—— message格式

MesosFrameworkMasterSlave之间使用http协议传递消息。参考下图:

Capture

由于Transfer-Encodingchunked格式,所以在http message body开始部分会包含data长度:0x44,也就是68个字节。另外,data的编码使用protobuf协议,也是key-value格式,具体编码请参考这里

参考资料:
《HTTP:The Definitive Guide》笔记(2)—— messages
HTTP – What does ‘Transfer-Encoding : Chunked’ mean?

Mesos笔记 (7)—— 打印VLOG函数输出的log

默认情况下,Mesos不会输出VLOG函数的log信息。如果想输出的话,需要加上GLOG_v=m

$ sudo GLOG_v=3 ./bin/mesos-master.sh --ip=15.242.100.56 --work_dir=/var/lib/mesos
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1229 22:42:38.818521 11830 process.cpp:2426] Spawned process __gc__@15.242.100.56:5050
I1229 22:42:38.818613 11846 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.818540032+00:00
I1229 22:42:38.818749 11847 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.818712832+00:00
I1229 22:42:38.818802 11844 process.cpp:2436] Resuming help@15.242.100.56:5050 at 2015-12-30 03:42:38.818746112+00:00
I1229 22:42:38.819092 11844 process.cpp:2393] Dropping event for process @0.0.0.0:0
I1229 22:42:38.819212 11830 process.cpp:2426] Spawned process help@15.242.100.56:5050
I1229 22:42:38.819373 11858 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.819341056+00:00
I1229 22:42:38.819762 11830 process.cpp:2426] Spawned process logging@15.242.100.56:5050
I1229 22:42:38.819864 11830 process.cpp:2426] Spawned process profiler@15.242.100.56:5050
I1229 22:42:38.819890 11854 process.cpp:2436] Resuming logging@15.242.100.56:5050 at 2015-12-30 03:42:38.819844096+00:00
I1229 22:42:38.819907 11849 process.cpp:2436] Resuming profiler@15.242.100.56:5050 at 2015-12-30 03:42:38.819878144+00:00
......

参考资料:
Re: How can mesos print logs from VLOG function?

 

Mesos笔记 (5)—— 资源

本文参考Building Applications on Mesos

Mesos Slave提供的通用资源包含:CPUsmemorydiskports。需要注意的是CPUs的值定义:

Capture

Slave提供资源的例子:

Capture Slave本身属性的定义:

Capture

Slave为不同的role预留资源有两种方式:
静态方式:在命令行指定不同的role获得不同的资源:

Capture

动态方式:从0.25版本开始,支持通过JSON配置文件和HTTP API方式动态预留资源。

获取资源的函数:

namespace mesos {
namespace internal {
namespace slave {

// TODO(idownes): Move this to the Containerizer interface to complete
// the delegation of containerization, i.e., external containerizers should be
// able to report the resources they can isolate.
Try<Resources> Containerizer::resources(const Flags& flags)
{
  Try<Resources> parsed = Resources::parse(
      flags.resources.getOrElse(""), flags.default_role);

  if (parsed.isError()) {
    return Error(parsed.error());
  }

  Resources resources = parsed.get();

  // NOTE: We need to check for the "cpus" string within the flag
  // because once Resources are parsed, we cannot distinguish between
  //  (1) "cpus:0", and
  //  (2) no cpus specified.
  // We only auto-detect cpus in case (2).
  // The same logic applies for the other resources!
  if (!strings::contains(flags.resources.getOrElse(""), "cpus")) {
    // No CPU specified so probe OS or resort to DEFAULT_CPUS.
    double cpus;
    Try<long> cpus_ = os::cpus();
    if (!cpus_.isSome()) {
      LOG(WARNING) << "Failed to auto-detect the number of cpus to use: '"
                   << cpus_.error()
                   << "'; defaulting to " << DEFAULT_CPUS;
      cpus = DEFAULT_CPUS;
    } else {
      cpus = cpus_.get();
    }

    resources += Resources::parse(
        "cpus",
        stringify(cpus),
        flags.default_role).get();
  }

  // Memory resource.
  ......
}

举个例子:启动Slave时参数为--resources=gpgpus:1,则Resources::parse函数会解析gpgpus资源;由于命令行没有提供CPUmemory之类的参数,需要调用os::cpus()等方法来获得。

 

Kubernetes笔记(7)—— 搭建”k8s on Mesos”注意事项

在本地搭建k8s on Mesos项目时,Mesos client脚本要以root身份运行。另外如果本地环境用到了proxy,一定要注意可能(不确定是否有例外,比如也许取决于你所使用的proxy或操作系统)需要把k8s或者MesosIP地址加入到no-proxy/NO_PROXY环境变量中。具体可参见下列issues
The kubernetes on Mesos can’t run successfully on the same machine
Why does “km controller-manager” think it is an invalid event?
The “km controller-manager” command doesn’t work successfully behind proxy.

 

Mesos笔记 (2)—— 启动Slave脚本时注意事项

启动Mesos slave脚本时,要使用root权限。如果和Mesos Master运行在不同Server上,要指定所在机器的IP

$ sudo ./bin/mesos-slave.sh --master=10.100.3.50:5050 --ip=10.100.3.39

下文引自Building Applications on Mesos

Since the slaves need to be able to launch containers and ensure that the executors are run as specific users, the slave process typically runs as root , so that it can create containers and run different executors as different users.

 

Mesos和Kubernetes的区别

stackocerflow上的这篇帖子描述了kubernetesmesos的区别:

Kubernetes is an open source project that brings ‘Google style’ cluster management capabilities to the world of virtual machines, or ‘on the metal’ scenarios. It works very well with modern operating system environments (like CoreOS or Red Hat Atomic) that offer up lightweight computing ‘nodes’ that are managed for you. It is written in Golang and is lightweight, modular, portable and extensible. We (the Kubernetes team) are working with a number of different technology companies (including Mesosphere who curate the Mesos open source project) to establish Kubernetes as the standard way to interact with computing clusters. The idea is to reproduce the patterns that we see people needing to build cluster applications based on our experience at Google. Some of these concepts include:

pods — a way to group containers together
replication controllers — a way to handle the lifecycle of containers
labels — a way to find and query containers, and
services — a set of containers performing a common function.
So with Kubernetes alone you will have something that is simple, easy to get up-and-running, portable and extensible that adds ‘cluster’ as a noun to the things that you manage in the lightest weight manner possible. Run an application on a cluster, and stop worrying about an individual machine. In this case, cluster is a flexible resource just like a VM. It is a logical computing unit. Turn it up, use it, resize it, turn it down quickly and easily.

With Mesos, there is a fair amount of overlap in terms of the basic vision, but the products are at quite different points in their lifecycle and have different sweet spots. Mesos is a distributed systems kernel that stitches together a lot of different machines into a logical computer. It was born for a world where you own a lot of physical resources to create a big static computing cluster. The great thing about it is that lots of modern scalable data processing application run well on Mesos (Hadoop, Kafka, Spark) and it is nice because you can run them all on the same basic resource pool, along with your new age container packaged apps. It is somewhat more heavy weight than the Kubernetes project, but is getting easier and easier to manage thanks to the work of folks like Mesosphere.

Now what gets really interesting is that Mesos is currently being adapted to add a lot of the Kubernetes concepts and to support the Kubernetes API. So it will be a gateway to getting more capabilities for your Kubernetes app (high availability master, more advanced scheduling semantics, ability to scale to a very large number of nodes) if you need them, and is well suited to run production workloads (Kubernetes is still in an alpha state).

When asked, I tend to say:

Kubernetes is a great place to start if you are new to the clustering world; it is the quickest, easiest and lightest way to kick the tires and start experimenting with cluster oriented development. It offers a very high level of portability since it is being supported by a lot of different providers (Microsoft, IBM, Red Hat, CoreOs, MesoSphere, VMWare, etc).

If you have existing workloads (Hadoop, Spark, Kafka, etc), Mesos gives you a framework that let’s you interleave those workloads with each other, and mix in a some of the new stuff including Kubernetes apps.

Mesos gives you an escape valve if you need capabilities that are not yet implemented by the community in the Kubernetes framework.

简而言之,Mesos是一个distributed systems kernel,它的作用可以理解成把一组physical机器抽象成一台logic机器供程序使用。Kubernetes是一个运行在Mesos上的framework,它是一个管理containercluster manager