分类：Mesos笔记

Mesos笔记 （9）—— Containerizer类代码解析

Containerizer类（定义在src/slave/containerizer/containerizer.hpp ）是所有Containerizer的抽象父类。除了默认的构造函数和一个什么都没做的析构函数，其只实现了create和resources方法。create方法代码如下（v0.26版本）：

Try<Containerizer*> Containerizer::create(
    const Flags& flags,
    bool local,
    Fetcher* fetcher)
{
  if (flags.isolation == "external") {
    LOG(WARNING) << "The 'external' isolation flag is deprecated, "
                 << "please update your flags to"
                 << " '--containerizers=external'.";

    Try<ExternalContainerizer*> containerizer =
      ExternalContainerizer::create(flags);
    if (containerizer.isError()) {
      return Error("Could not create ExternalContainerizer: " +
                   containerizer.error());
    }

    return containerizer.get();
  }

  // TODO(benh): We need to store which containerizer or
  // containerizers were being used. See MESOS-1663.

  // Create containerizer(s).
  vector<Containerizer*> containerizers;

  foreach (const string& type, strings::split(flags.containerizers, ",")) {
    if (type == "mesos") {
      Try<MesosContainerizer*> containerizer =
        MesosContainerizer::create(flags, local, fetcher);
      if (containerizer.isError()) {
        return Error("Could not create MesosContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else if (type == "docker") {
      Try<DockerContainerizer*> containerizer =
        DockerContainerizer::create(flags, fetcher);
      if (containerizer.isError()) {
        return Error("Could not create DockerContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else if (type == "external") {
      Try<ExternalContainerizer*> containerizer =
        ExternalContainerizer::create(flags);
      if (containerizer.isError()) {
        return Error("Could not create ExternalContainerizer: " +
                     containerizer.error());
      } else {
        containerizers.push_back(containerizer.get());
      }
    } else {
      return Error("Unknown or unsupported containerizer: " + type);
    }
  }

  if (containerizers.size() == 1) {
    return containerizers.front();
  }

  Try<ComposingContainerizer*> containerizer =
    ComposingContainerizer::create(containerizers);

  if (containerizer.isError()) {
    return Error(containerizer.error());
  }

  return containerizer.get();
}

默认情况下，containerizers的type是mesos，所以会调用MesosContainerizer::create来生成containerizer。关于resources方法，参考Mesos笔记 （5）—— 资源。

 

Mesos笔记 （8）—— message格式

Mesos的Framework，Master和Slave之间使用http协议传递消息。参考下图：

Capture

由于Transfer-Encoding是chunked格式，所以在http message body开始部分会包含data长度：0x44，也就是68个字节。另外，data的编码使用protobuf协议，也是key-value格式，具体编码请参考这里。

参考资料：
《HTTP:The Definitive Guide》笔记（2）—— messages；
HTTP – What does ‘Transfer-Encoding : Chunked’ mean?。

Mesos笔记 （7）—— 打印VLOG函数输出的log

默认情况下，Mesos不会输出VLOG函数的log信息。如果想输出的话，需要加上GLOG_v=m：

$ sudo GLOG_v=3 ./bin/mesos-master.sh --ip=15.242.100.56 --work_dir=/var/lib/mesos
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1229 22:42:38.818521 11830 process.cpp:2426] Spawned process __gc__@15.242.100.56:5050
I1229 22:42:38.818613 11846 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.818540032+00:00
I1229 22:42:38.818749 11847 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.818712832+00:00
I1229 22:42:38.818802 11844 process.cpp:2436] Resuming help@15.242.100.56:5050 at 2015-12-30 03:42:38.818746112+00:00
I1229 22:42:38.819092 11844 process.cpp:2393] Dropping event for process @0.0.0.0:0
I1229 22:42:38.819212 11830 process.cpp:2426] Spawned process help@15.242.100.56:5050
I1229 22:42:38.819373 11858 process.cpp:2436] Resuming __gc__@15.242.100.56:5050 at 2015-12-30 03:42:38.819341056+00:00
I1229 22:42:38.819762 11830 process.cpp:2426] Spawned process logging@15.242.100.56:5050
I1229 22:42:38.819864 11830 process.cpp:2426] Spawned process profiler@15.242.100.56:5050
I1229 22:42:38.819890 11854 process.cpp:2436] Resuming logging@15.242.100.56:5050 at 2015-12-30 03:42:38.819844096+00:00
I1229 22:42:38.819907 11849 process.cpp:2436] Resuming profiler@15.242.100.56:5050 at 2015-12-30 03:42:38.819878144+00:00
......

参考资料：
Re: How can mesos print logs from VLOG function?。

 

Mesos笔记 （5）—— 资源

本文参考Building Applications on Mesos：

Mesos Slave提供的通用资源包含：CPUs，memory，disk和ports。需要注意的是CPUs的值定义：

Capture

Slave提供资源的例子：

Capture Slave本身属性的定义：

Capture

Slave为不同的role预留资源有两种方式：
静态方式：在命令行指定不同的role获得不同的资源：

Capture

动态方式：从0.25版本开始，支持通过JSON配置文件和HTTP API方式动态预留资源。

获取资源的函数：

namespace mesos {
namespace internal {
namespace slave {

// TODO(idownes): Move this to the Containerizer interface to complete
// the delegation of containerization, i.e., external containerizers should be
// able to report the resources they can isolate.
Try<Resources> Containerizer::resources(const Flags& flags)
{
  Try<Resources> parsed = Resources::parse(
      flags.resources.getOrElse(""), flags.default_role);

  if (parsed.isError()) {
    return Error(parsed.error());
  }

  Resources resources = parsed.get();

  // NOTE: We need to check for the "cpus" string within the flag
  // because once Resources are parsed, we cannot distinguish between
  //  (1) "cpus:0", and
  //  (2) no cpus specified.
  // We only auto-detect cpus in case (2).
  // The same logic applies for the other resources!
  if (!strings::contains(flags.resources.getOrElse(""), "cpus")) {
    // No CPU specified so probe OS or resort to DEFAULT_CPUS.
    double cpus;
    Try<long> cpus_ = os::cpus();
    if (!cpus_.isSome()) {
      LOG(WARNING) << "Failed to auto-detect the number of cpus to use: '"
                   << cpus_.error()
                   << "'; defaulting to " << DEFAULT_CPUS;
      cpus = DEFAULT_CPUS;
    } else {
      cpus = cpus_.get();
    }

    resources += Resources::parse(
        "cpus",
        stringify(cpus),
        flags.default_role).get();
  }

  // Memory resource.
  ......
}

举个例子：启动Slave时参数为--resources=gpgpus:1，则Resources::parse函数会解析gpgpus资源；由于命令行没有提供CPU，memory之类的参数，需要调用os::cpus()等方法来获得。

 

Mesos笔记 （2）—— 启动Slave脚本时注意事项

启动Mesos slave脚本时，要使用root权限。如果和Mesos Master运行在不同Server上，要指定所在机器的IP：

$ sudo ./bin/mesos-slave.sh --master=10.100.3.50:5050 --ip=10.100.3.39

下文引自Building Applications on Mesos：

Since the slaves need to be able to launch containers and ensure that the executors are run as specific users, the slave process typically runs as root , so that it can create containers and run different executors as different users.

 

Mesos笔记 （1）—— 架构

本文内容摘自下列文章：
APACHE MESOS: THE TRUE OS FOR THE SOFTWARE DEFINED DATA CENTER?；
Mesos Architecture。

Imagine if instead of individual physical servers, we could aggregate all the resources in a data center into a single large virtual pool, exposing not virtual machines but primitives such as CPU, RAM, and I/O? In conjunction, imagine if we could break applications into small isolated units of tasks that could be dynamically assigned resources from our virtual data center pool, based on the needs of the applications in our data center? The analogy here would be a PC with an operating system that is pooling the PC’s processors and RAM and coordinating the allocation and deallocation of those resources for use by different processes. Now extend that analogy to make the data center the PC with Mesos as the operating system kernel. That, in a nutshell, is how Mesos is transforming the data center and making true SDDC a reality.

通俗地讲，可以把一个数据中心里所有的硬件资源看成一个整体。Mesos的功能就是为应用程序管理和分配这些资源。

Mesos结构如下图所示：

mesos-arch1

The modified diagram above from the Apache Mesos website shows how Mesos implements it’s two-level scheduling architecture for managing multiple types of applications. The first level is the master daemon which manages slave daemons running on each node in the Mesos cluster. The cluster consists of all servers, physical or virtual, that will be running applications tasks, such as Hadoop and MPI jobs. The second level consists of a component called a framework. A framework includes a scheduler and an executor process, the latter of which also runs on each node. Mesos is able to communicate with different types of frameworks with each one managing a different clustered application. The diagram above shows Hadoop and MPI but other frameworks have been written as well for other types of applications.

Mesos cluster里的每一个node都运行一个slave daemon程序，由一个master daemon程序统一管理。Mesos上运行的程序称之为framework，它包含两个部分：scheduler和executor process。分布式系统通常包含一个controller和多个worker，worker可以不依赖于controller而独立地运行。对于framework而言，scheduler就是controller，executor process就是worker。

 

A framework running on top of Mesos consists of two components: a scheduler that registers with the master to be offered resources, and an executor process that is launched on slave nodes to run the framework’s tasks (see the App/Framework development guide for more details about application schedulers and executors). While the master determines how many resources are offered to each framework, the frameworks’ schedulers select which of the offered resources to use. When a frameworks accepts offered resources, it passes to Mesos a description of the tasks it wants to run on them. In turn, Mesos launches the tasks on the corresponding slaves.

scheduler从master得到需要的资源，而executor process则会在slave node上运行framework task。