我的站点

一个系统软件工程师的随手涂鸦

Tag: 计算机常识 (Page 1 of 2)

什么是semaphore?

以下摘自The Little Book of Semaphores

A semaphore is like an integer, with three differences:
1. When you create the semaphore, you can initialize its value to any integer, but after that the only operations you are allowed to perform are increment (increase by one) and decrement (decrease by one). You cannot read the current value of the semaphore.
2. When a thread decrements the semaphore, if the result is negative, the thread blocks itself and cannot continue until another thread increments the semaphore.
3. When a thread increments the semaphore, if there are other threads waiting, one of the waiting threads gets unblocked.
To say that a thread blocks itself (or simply “blocks”) is to say that it notifies the scheduler that it cannot proceed. The scheduler will prevent the thread from running until an event occurs that causes the thread to become unblocked. In the tradition of mixed metaphors in computer science, unblocking is often called “waking”.
That’s all there is to the definition, but there are some consequences of the definition you might want to think about.
• In general, there is no way to know before a thread decrements a semaphore whether it will block or not (in specific cases you might be able to prove that it will or will not).
• After a thread increments a semaphore and another thread gets woken up, both threads continue running concurrently. There is no way to know which thread, if either, will continue immediately.
• When you signal a semaphore, you don’t necessarily know whether another thread is waiting, so the number of unblocked threads may be zero or one.
Finally, you might want to think about what the value of the semaphore means. If the value is positive, then it represents the number of threads that can decrement without blocking. If it is negative, then it represents the number of threads that have blocked and are waiting. If the value is zero, it means there are no threads waiting, but if a thread tries to decrement, it will block.

semaphore的初始值为1,通常就用来构建mutex

 

RESTful Web 服务简介

本文是RESTful Web Services: A Tutorial的笔记:

REST stands for Representational State Transfer, which is an architectural style for networked hypermedia applications, it is primarily used to build Web services that are lightweight, maintainable, and scalable. A service based on REST is called a RESTful service. REST is not dependent on any protocol, but almost every RESTful service uses HTTP as its underlying protocol.

尽管REST本身不依赖任何协议,但是事实上RESTful服务基本都使用HTTP协议。

RESTful服务本身的特性:

Every system uses resources. These resources can be pictures, video files, Web pages, business information, or anything that can be represented in a computer-based system. The purpose of a service is to provide a window to its clients so that they can access these resources. Service architects and developers want this service to be easy to implement, maintainable, extensible, and scalable. A RESTful design promises that and more. In general, RESTful services should have following properties and features, which I’ll describe in detail:

Representations
Messages
URIs
Uniform interface
Stateless
Links between resources
Caching

具体来说:
Representations:使用JSONXML
MessagesURIsUniform interface:使用HTTP协议;
Stateless:很重要,不同的request之间不能有依赖。

RPC和gRPC

关于RPC(Remote Procedure Call)的定义,可以参考这两篇文章:Remote procedure callRemote Procedure Call (RPC)。简单来讲,RPC指的就是一个进程(client)发起一个函数调用,但是实际用来执行这个函数调用是另一个进程(server)。这个函数调用会阻塞在那里,直到收到响应。Server进程可以与client进程可以位于同一台机器也可以是不同机器。Clientserver都有一个stub模块,clientstub负责发送request,并处理server返回的response;而serverstub则负责处理client发送的request,并返回response。 通常,我们使用Interface Description Language来定义RPC的消息格式。

gRPCgoogle实现的RPC,其默认使用protocol buffers,一个使用gRPC的例子如下(图出自Getting started):

grpc_concept_diagram_00

Go语言为例,搭建gPRC环境包括下列几步:

(1)安装gRPC runtime

go get google.golang.org/grpc

(2)安装protocol buffers支持(包含compilerruntime):从protocol buffers官网下载最新的源码包,解压缩:

./configure
make
make check
make install
ldconfig

(3)安装Go protoc plugin

go get -a github.com/golang/protobuf/protoc-gen-go

参考gRPC提供的例子,下载helloworld.proto文件,然后在当前目录执行“protoc --go_out=plugins=grpc:. *.proto”就会在当前目录下生成helloworld.pb.go文件。对于client来讲,发送request的函数已生成好,只需直接传入request内容以及option即可:

type GreeterClient interface {
    // Sends a greeting
    SayHello(ctx context.Context, in *HelloRequest, opts ...grpc.CallOption) (*HelloReply, error)
}

type greeterClient struct {
    cc *grpc.ClientConn
}

func NewGreeterClient(cc *grpc.ClientConn) GreeterClient {
    return &greeterClient{cc}
}

func (c *greeterClient) SayHello(ctx context.Context, in *HelloRequest, opts ...grpc.CallOption) (*HelloReply, error) {
    out := new(HelloReply)
    err := grpc.Invoke(ctx, "/helloworld.Greeter/SayHello", in, out, c.cc, opts...)
    if err != nil {
        return nil, err
    }
    return out, nil
} 

对于server,定义好了interface,需要自己实现满足这个interface的结构体:

// Server API for Greeter service

type GreeterServer interface {
    // Sends a greeting
    SayHello(context.Context, *HelloRequest) (*HelloReply, error)
}

可以看出,SayHello()函数只要实现如何构造response即可,其它如何传输消息等等,均不用管。

为了熟悉这个流程,我自己做了一遍,代码在这里:https://github.com/NanXiao/helloworld

“/dev/tty”,“/dev/console”和“/dev/tty0”的区别

这篇笔记来自于stackoverflow的一篇帖子,答案如下:

From the documentation(http://www.kernel.org/doc/Documentation/devices.txt):

    /dev/tty        Current TTY device
    /dev/console    System console
    /dev/tty0       Current virtual console

In the good old days /dev/console was System Administrator console. And TTYs were users' serial devices attached to a server.
Now /dev/console and /dev/tty0 represent current display and usually are the same. You can override it for example by adding console=ttyS0 to grub.conf. After that your /dev/tty0 is a monitor and /dev/console is /dev/ttyS0.

An exercise to show the difference between /dev/tty and /dev/tty0:

Switch to the 2nd console by pressing Ctrl+Alt+F2. Login as root. Type "sleep 5; echo tty0 > /dev/tty0". Press Enter and switch to the 3rd console by pressing Alt+F3.
Now switch back to the 2nd console by pressing Alt+F2. Type "sleep 5; echo tty > /dev/tty", press Enter and switch to the 3rd console.

You can see that "tty" is the console where process starts, and "tty0" is a always current console.

早些时候,/dev/console是系统管理员控制台,而TTYs则代表用户连接服务器的串行设备。而现在,/dev/console/dev/tty0均指当前的显示设备,并且通常情况下是一样的。你可以修改/dev/console所关联的设备。举个例子,在grub.conf中加入console=ttyS0。则现在,/dev/tty0所关联的是显示器,而dev/console则关联/dev/ttyS0

/dev/tty是当前进程控制的tty设备,而tty0则是当前的控制台。当你在一个终端执行“sleep 5; echo tty0 > /dev/tty0”命令后,切换到其它终端,则tty0会在你切换后的终端显示。而执行“sleep 5; echo tty > /dev/tty”命令后,无论切换到那个终端,tty始终会在输入命令的终端显示。

 

BIOS和UEFI

BIOS(Basic Input/Output System)是固化在主板芯片上的一段代码。电脑启动时,由BIOS负责启动各个硬件单元,并把“控制权”交给操作系统。BIOS使用MBRMaster Boot Record,存储电脑的分区表)来决定使用哪个操作系统。BIOS为用户提供了一个操作硬件的接口。

UEFI(Unified Extensible Firmware Interface)BIOS的继任者。它同BIOS的比较如下:

a)16-bit vs 32/64-bit
BIOS只能工作在16-bit处理器模式上,且只能访问1M内存。而UEFI则可工作在32/64-bit处理器模式上,且可访问更大的内存空间。

b)Booting
MBR限制每个磁盘只能有4个分区,且可以boot的磁盘大小限制在2.2TB。而UEFI使用GUID Partition Table,可以访问更大的磁盘。

c)Extensions
UEFI支持老的extensions(比如,ACPI)。

参考资料:
HTG Explains: Learn How UEFI Will Replace Your PC’s BIOS

 

“Emulation”和“Hardware virtualization”的比较

Emulation”是软件模拟硬件,即在你当前的host机器(举个例子:X86)上运行另一个SPARC平台的的虚拟机。软件需要把SPARC平台指令转化为X86指令,所以速度很慢。(qemu-system-x86_64

Hardware virtualization”是硬件支持虚拟化。即软件直接利用CPU和芯片组等硬件。由于没有指令转化,所以速度很快。当然,host机器和虚拟机是同样的指令集。(qemu-system-x86_64 -enable-kvm

参考资料:
qemu-kvm or qemu-system-x86_64?

 

PCI寻址笔记

Linux现在支持多个PCI domain,每个PCI domain管理256bus2^8),每个bus上可以挂载322^5)个设备,每个设备最多支持82^3)个function。同一PCI bus上的设备共享memory locationI/O port的地址空间,而configuration register则不是。每个PCI设备的configuration register256个字节。

执行“lspci -D”命令显示当前系统的PCI设备信息。格式为:domain:bus:dev:func

[root@localhost ~]# lspci -D
0000:00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
0000:00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
0000:00:01.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
0000:00:02.0 VGA compatible controller: InnoTek Systemberatung GmbH VirtualBox Graphics Adapter
0000:00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
0000:00:04.0 System peripheral: InnoTek Systemberatung GmbH VirtualBox Guest Service
0000:00:05.0 Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio Controller (rev 01)
0000:00:06.0 USB controller: Apple Inc. KeyLargo/Intrepid USB
0000:00:07.0 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
0000:00:0b.0 USB controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB2 EHCI Controller
0000:00:0d.0 SATA controller: Intel Corporation 82801HM/HEM (ICH8M/ICH8M-E) SATA Controller [AHCI mode] (rev 02)

*NIX & Hacking —— 第5期

做一本我感兴趣的杂志,就这么简单!

C

Integer Overflow

Golang

breaking out of a select statement when all channels are closed

Kernel

Feature: High Memory In The Linux Kernel
What’s Next for Containers? User Namespaces

Network

Recent advancements in Linux TCP congestion control
SO_REUSEPORT – Scaling Techniques for Servers with High Connection Rates

Rust

Rust in Detail: Writing Scalable Chat Service from Scratch

Shell

fish shell

Easter egg

Announcing the first Art of Computer Programming eBooks
Goodbye Moto
Intel DDIO, LLC cache, buffer alignment, prefetching, shared locks and packet rates
Jokes
List of freely available Programming Books
Mandelbrot Set with SIMD Intrinsics

什么是“Intel VT-d”?

Intel VT-d(以下简称VT-d)”代表“Intel Virtualization Technology for Directed I/O”。“VT(Virtualization Technology)”泛指Intel所有的虚拟化技术,而“VT-d”则是虚拟化技术解决方案中的一种。VT-d的整体思想就是用硬件方式支持隔离和限制对设备的访问。

VT-d有以下主要功能:
a)分配I/O设备:这个功能允许管理员根据需求,灵活地为虚拟机分配I/O设备。
b)DMA remapping:支持针对虚拟机DMA访问的地址转换。
c)Interrupt remapping:支持虚拟机对设备中断的路由和隔离。
d)可靠性功能:记录关于DMAInterrupt的错误访问。

参考资料:
Understanding VT-d: Intel Virtualization Technology for Directed I/O

什么是IOMMU?

在计算机领域,IOMMU(Input/Output Memory Management Unit)是一个内存管理单元(Memory Management Unit),它的作用是连接DMA-capable I/O总线(Direct Memory Access-capable I/O Bus)和主存(main memory)。传统的内存管理单元会把CPU访问的虚拟地址转化成实际的物理地址。而IOMMU则是把设备(device)访问的虚拟地址转化成物理地址。为了防止设备错误地访问内存,有些IOMMU还提供了访问内存保护机制。参考下图:

282px-MMU_and_IOMMU.svg

IOMMU的一个重要用途是在虚拟化技术(virtualization):虚拟机上运行的操作系统(guest OS)通常不知道它所访问的host-physical内存地址。如果要进行DMA操作,就有可能破坏内存,因为实际的硬件(hardware)不知道guest-physicalhost-physical内存地址之间的映射关系。IOMMU根据guest-physicalhost-physical内存地址之间的转换表(translation table),re-mapping硬件访问的地址,就可以解决这个问题。

另外,在AMDVIRTUALIZING IO THROUGH THE IO MEMORY MANAGEMENT UNIT (IOMMU)文档中,也有一个更全面的总结图:

iommu

参考资料:
IOMMU

Page 1 of 2

Powered by WordPress & Theme by Anders Norén