Category: Technology

Message length setting in gRPC

The default send/receive message length of gRPC is defined here:

/** Default send/receive message size limits in bytes. -1 for unlimited. */
/** TODO(roth) Make this match the default receive limit after next release */
#define GRPC_DEFAULT_MAX_SEND_MESSAGE_LENGTH -1
#define GRPC_DEFAULT_MAX_RECV_MESSAGE_LENGTH (4 * 1024 * 1024)

We can know that send message length is no limited (-1), while Server/Client can only receive 4 Mi bytes by default.

You can change receive message length to unlimited in Client:

grpc::ChannelArguments ch_args;
ch_args.SetMaxReceiveMessageSize(-1);
std::shared_ptr<grpc::Channel> ch = 
        grpc::CreateCustomChannel("localhost:50051", grpc::InsecureChannelCredentials(), ch_args);

But this doesn’t work in Server:

ServerBuilder builder;
builder.SetMaxReceiveMessageSize(-1);

Because in server_builder.cc, the parameter only takes effect when it is positive:

std::unique_ptr<Server> ServerBuilder::BuildAndStart() {
    ......
    if (max_receive_message_size_ >= 0) {
      args.SetInt(GRPC_ARG_MAX_RECEIVE_MESSAGE_LENGTH, max_receive_message_size_);
    }
    ......
}

So we can use builder.SetMaxReceiveMessageSize(INT_MAX); as a work-around.

BTW, check message length limit is in get_message_size_limits function.

Use clang to build OpenBSD on amd64/i386

I install the newest OpenBSD 6.1, and try to build -curr kernel. But unfortunately the make reports following errors:

# make
cat /usr/src/sys/arch/amd64/amd64/genassym.cf /usr/src/sys/arch/amd64/amd64/genassym.cf |  sh /usr/src/sys/kern/genassym.sh cc -no-integrated-as -g -Werror -Wall -Wimplicit-function-declaration  -Wno-uninitialized -Wno-pointer-sign  -Wno-address-of-packed-member -Wno-constant-conversion  -Wframe-larger-than=2047 -mcmodel=kernel -mno-red-zone -mno-sse2 -mno-sse -mno-3dnow  -mno-mmx -msoft-float -fno-omit-frame-pointer -ffreestanding -fno-pie -O2 -pipe -nostdinc -I/usr/src/sys -I/usr/src/sys/arch/amd64/compile/GENERIC.MP/obj -I/usr/src/sys/arch -DDDB -DDIAGNOSTIC -DKTRACE -DACCOUNTING -DKMEMSTATS -DPTRACE -DPOOL_DEBUG -DCRYPTO -DSYSVMSG -DSYSVSEM -DSYSVSHM -DUVM_SWAP_ENCRYPT -DFFS -DFFS2 -DFFS_SOFTUPDATES -DUFS_DIRHASH -DQUOTA -DEXT2FS -DMFS -DNFSCLIENT -DNFSSERVER -DCD9660 -DUDF -DMSDOSFS -DFIFO -DFUSE -DSOCKET_SPLICE -DTCP_SACK -DTCP_ECN -DTCP_SIGNATURE -DINET6 -DIPSEC -DPPP_BSDCOMP -DPPP_DEFLATE -DPIPEX -DMROUTING -DMPLS -DBOOT_CONFIG -DUSER_PCICONF -DAPERTURE -DMTRR -DNTFS -DHIBERNATE -DPCIVERBOSE -DUSBVERBOSE -DWSDISPLAY_COMPAT_USL -DWSDISPLAY_COMPAT_RAWKBD -DWSDISPLAY_DEFAULTSCREENS="6" -DX86EMU -DONEWIREVERBOSE -DMULTIPROCESSOR -DMAXUSERS=80 -D_KERNEL -MD -MP -MF assym.P > assym.h.tmp
cc: unrecognized option '-no-integrated-as'
cc1: error: unrecognized command line option "-Wno-address-of-packed-member"
cc1: error: unrecognized command line option "-Wno-constant-conversion"
*** Error 1 in /usr/src/sys/arch/amd64/compile/GENERIC.MP (Makefile:938 'assym.h')

From this mail, I learn that clang has been the default compiler on amd64/i386 platforms for OpenBSD, so I switch to use clang to build kernel:

# CC=clang make
.....

Now it can compile!

Configure Thunderbird to send patch friendly

Recently, I have tried outlook and web gmail to send patches to OpenBSD, but neither of them is an ideal choice. For example, the original patch is:

Index: dmesg.c
===================================================================
RCS file: /cvs/src/sbin/dmesg/dmesg.c,v
retrieving revision 1.29
diff -u -p -r1.29 dmesg.c
--- dmesg.c     1 Sep 2017 07:31:45 -0000       1.29
+++ dmesg.c     4 Sep 2017 08:55:50 -0000
@@ -65,12 +65,12 @@ main(int argc, char *argv[])
        int ch, newl, skip, i;
        char *p;
        struct msgbuf cur;
-       char *memf, *nlistf, *bufdata = NULL;
+       char *memf = NULL, *nlistf = NULL, *bufdata = NULL;
        char *allocated = NULL;
        int startupmsgs = 0;
        char buf[5];

-       memf = nlistf = NULL;
+       memset(&cur, 0, sizeof(cur));
        while ((ch = getopt(argc, argv, "sM:N:")) != -1)
                switch(ch) {
                case 's':

The actual effect of sending patch using above tools:

Index: dmesg.c
===================================================================
RCS file: /cvs/src/sbin/dmesg/dmesg.c,v
retrieving revision 1.29
diff -u -p -r1.29 dmesg.c
--- dmesg.c 1 Sep 2017 07:31:45 -0000   1.29
+++ dmesg.c 4 Sep 2017 08:55:50 -0000
@@ -65,12 +65,12 @@ main(int argc, char *argv[])
    int ch, newl, skip, i;
    char *p;
    struct msgbuf cur;
-   char *memf, *nlistf, *bufdata = NULL;
+   char *memf = NULL, *nlistf = NULL, *bufdata = NULL;
    char *allocated = NULL;
    int startupmsgs = 0;
    char buf[5];

-   memf = nlistf = NULL;
+   memset(&cur, 0, sizeof(cur));
    while ((ch = getopt(argc, argv, "sM:N:")) != -1)
    switch(ch) {
    case 's':

We can see the spaces in the left part of code are lost. After referring Completely plain email and Thunderbird (GUI), I decide to useThunderbird as the email client, but need further customization:

(1) All sent mails are plain text: “Account Settings” -> “Composition & Addressing“, untick “Compose message in HTML format“:

1

Or before sending mail: “Options” -> “Delivery Format” -> “Plain Text Only“.

(2) “Tools” -> “Options” -> “Advanced” -> “Config Editor...“:

2

Set mailnews.send_plaintext_flowed to false:

3

Now the patch format is correct.

Create a symbol link for python after installing it on OpenBSD

After installing Python on FreeBSD:

# pkg_add python
quirks-2.304 signed on 2017-04-02T15:01:33Z
Ambiguous: choose package for python
a       0: <None>
        1: python-2.7.13p0
        2: python-3.4.5p2
        3: python-3.5.2p2
        4: python-3.6.0p0
Your choice: 4

It won’t create a symbol link by default:

# python
ksh: python: not found

So for using it handily, you can create a symbol link yourself:

# cd /usr/local/bin
# ln -s python3.6 python

Build gRPC on ArchLinux

Today, I followed Build from Source to compile gRPC on ArchLinux:

 $ git clone -b $(curl -L https://grpc.io/release) https://github.com/grpc/grpc
 $ cd grpc
 $ git submodule update --init
 $ make

Current gRPC‘s release version is v1.4.x:

$ curl -L https://grpc.io/release
v1.4.x

The build flow will generate the errors like this:

......
src/core/lib/support/murmur_hash.c: In function ‘gpr_murmur_hash3’:
src/core/lib/support/murmur_hash.c:79:10: error: this statement may fall through [-Werror=implicit-fallthrough=]
       k1 ^= ((uint32_t)tail[2]) << 16;
       ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/core/lib/support/murmur_hash.c:80:5: note: here
     case 2:
     ^~~~
src/core/lib/support/murmur_hash.c:81:10: error: this statement may fall through [-Werror=implicit-fallthrough=]
       k1 ^= ((uint32_t)tail[1]) << 8;
       ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
src/core/lib/support/murmur_hash.c:82:5: note: here
     case 1:
     ^~~~
cc1: all warnings being treated as errors
......

After referring Fix warnings with GCC 7, I finally make the compilation successful. To facilitate others to build gRPC v1.4.x source code on ArchLinux, I create a patch, and hope it can help others.

P.S.
(1) You should fallback to OpenSSL 1.0. Please refer here:

PKG_CONFIG_PATH=/usr/lib/openssl-1.0/pkgconfig make

Otherwise you may encounter following errors:

src/core/tsi/ssl_transport_security.c: In function ‘tsi_create_ssl_client_handshaker_factory’:
src/core/tsi/ssl_transport_security.c:1281:3: error: ‘TLSv1_2_method’ is deprecated [-Werror=deprecated-declarations]
   ssl_context = SSL_CTX_new(TLSv1_2_method());
   ^~~~~~~~~~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from src/core/tsi/ssl_transport_security.c:45:
/usr/include/openssl/ssl.h:1624:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *TLSv1_2_method(void)) /* TLSv1.2 */
 ^
src/core/tsi/ssl_transport_security.c: In function ‘tsi_create_ssl_server_handshaker_factory_ex’:
src/core/tsi/ssl_transport_security.c:1389:7: error: ‘TLSv1_2_method’ is deprecated [-Werror=deprecated-declarations]
       impl->ssl_contexts[i] = SSL_CTX_new(TLSv1_2_method());
       ^~~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from src/core/tsi/ssl_transport_security.c:45:
/usr/include/openssl/ssl.h:1624:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *TLSv1_2_method(void)) /* TLSv1.2 */
 ^
At top level:
src/core/tsi/ssl_transport_security.c:118:22: error: ‘openssl_thread_id_cb’ defined but not used [-Werror=unused-functio ]
 static unsigned long openssl_thread_id_cb(void) {
                      ^~~~~~~~~~~~~~~~~~~~
src/core/tsi/ssl_transport_security.c:110:13: error: ‘openssl_locking_cb’ defined but not used [-Werror=unused-function]
 static void openssl_locking_cb(int mode, int type, const char *file, int line) {
             ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

(2) You may need to change installation directory from /usr/local to /usr:

make prefix=/usr install

This lets you process pkg-config path easily.