Tag: gRPC

Be careful of using grpc::ClientStreamingInterface::Finish() function

No matter server-side streaming RPC or bidirectional streaming RPC, if there is still message for Client to read, but Client callsgrpc::ClientStreamingInterface::Finish(), it will block here forever. The call stack is like this:

#0  0x00007ffff5b24076 in epoll_pwait () from /usr/lib/libc.so.6
#1  0x00007ffff71c6ae5 in ?? () from /usr/lib/libgrpc.so.4
#2  0x00007ffff71e3a26 in ?? () from /usr/lib/libgrpc.so.4
#3  0x000055555558300f in grpc::CompletionQueue::Pluck (this=0x555555fd5340, tag=0x7fffffffd800)
    at /usr/include/grpc++/impl/codegen/completion_queue.h:224
#4  0x0000555555585e46 in grpc::ClientReaderWriter<..., ...>::Finish (
    this=0x555555fd5320) at /usr/include/grpc++/impl/codegen/sync_stream.h:494

So please pay attention to it. Before calling grpc::ClientStreamingInterface::Finish(), make sure the grpc::ReaderInterface::Read() return false:

while (stream->Read(&server_note)) {
  ...
}
Status status = stream->Finish();

First taste of gRPC

In the past month, I made my hand dirty on gRPC, and tried to use it in my current project. The effect seems good until now, I want to share some feeling here:

(1) gRPC lets you concentrate on message definition, and it will generate all necessary code for you. This is really very neat! The APIs are also simple and handy: they help process all kinds of exceptional cases which will save users a lot of energy.

(2) Now gRPC‘s maximum message length is INT_MAX (please refer this post) , so on general 64-bit platform, its value is 2^31 - 1, nearly 2GiB. Personally, I think maybe LONG_MAX, 2^63 - 1, is better, since it is nearly equal to say there is no limitation for message length. BTW, gRPC useProtocol Buffers under the hood. From Protocol Buffers document:

Protocol Buffers are not designed to handle large messages. As a general rule of thumb, if you are dealing in messages larger than a megabyte each, it may be time to consider an alternate strategy.

I haven’t dived into Protocol Buffers code, so not sure what is the negative effect of transmitting large message. But anyway, whether support and whether support well are 2 different things.

 

Be careful of clear/release methods in gRPC

My project uses gRPC and generates code like this:

......
inline void AddDBResponse::clear_last_record() {
  if (has_last_record()) {
    delete msg_.last_record_;
    clear_has_msg();
  }
}
......
inline ::privdb::DB* AddDBResponse::release_last_record() {
  // @@protoc_insertion_point(field_release:privdb.AddDBResponse.last_record)
  if (has_last_record()) {
    clear_has_msg();
    ::privdb::DB* temp = msg_.last_record_;
    msg_.last_record_ = NULL;
    return temp;
  } else {
    return NULL;
  }
}
inline void AddDBResponse::set_allocated_last_record(::privdb::DB* last_record) {
  clear_msg();
  if (last_record) {
    set_has_last_record();
    msg_.last_record_ = last_record;
  }
  // @@protoc_insertion_point(field_set_allocated:privdb.AddDBResponse.last_record)
}
......

In clear_last_record() method, it will assume the msg_.last_record_ is allocated in heap, so free it; while release_last_record() not. So if you callset_allocated_last_record() function and last_record points the data which resides on stack literally, please use release_last_record(). Otherwise, the scary memory corruption will occur!:-)

Message length setting in gRPC

The default send/receive message length of gRPC is defined here:

/** Default send/receive message size limits in bytes. -1 for unlimited. */
/** TODO(roth) Make this match the default receive limit after next release */
#define GRPC_DEFAULT_MAX_SEND_MESSAGE_LENGTH -1
#define GRPC_DEFAULT_MAX_RECV_MESSAGE_LENGTH (4 * 1024 * 1024)

We can know that send message length is no limited (-1), while Server/Client can only receive 4 Mi bytes by default.

You can change receive message length to unlimited in Client:

grpc::ChannelArguments ch_args;
ch_args.SetMaxReceiveMessageSize(-1);
std::shared_ptr<grpc::Channel> ch = 
        grpc::CreateCustomChannel("localhost:50051", grpc::InsecureChannelCredentials(), ch_args);

But this doesn’t work in Server:

ServerBuilder builder;
builder.SetMaxReceiveMessageSize(-1);

Because in server_builder.cc, the parameter only takes effect when it is positive:

std::unique_ptr<Server> ServerBuilder::BuildAndStart() {
    ......
    if (max_receive_message_size_ >= 0) {
      args.SetInt(GRPC_ARG_MAX_RECEIVE_MESSAGE_LENGTH, max_receive_message_size_);
    }
    ......
}

So we can use builder.SetMaxReceiveMessageSize(INT_MAX); as a work-around.

BTW, check message length limit is in get_message_size_limits function.

Build gRPC on ArchLinux

Today, I followed Build from Source to compile gRPC on ArchLinux:

 $ git clone -b $(curl -L https://grpc.io/release) https://github.com/grpc/grpc
 $ cd grpc
 $ git submodule update --init
 $ make

Current gRPC‘s release version is v1.4.x:

$ curl -L https://grpc.io/release
v1.4.x

The build flow will generate the errors like this:

......
src/core/lib/support/murmur_hash.c: In function ‘gpr_murmur_hash3’:
src/core/lib/support/murmur_hash.c:79:10: error: this statement may fall through [-Werror=implicit-fallthrough=]
       k1 ^= ((uint32_t)tail[2]) << 16;
       ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/core/lib/support/murmur_hash.c:80:5: note: here
     case 2:
     ^~~~
src/core/lib/support/murmur_hash.c:81:10: error: this statement may fall through [-Werror=implicit-fallthrough=]
       k1 ^= ((uint32_t)tail[1]) << 8;
       ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
src/core/lib/support/murmur_hash.c:82:5: note: here
     case 1:
     ^~~~
cc1: all warnings being treated as errors
......

After referring Fix warnings with GCC 7, I finally make the compilation successful. To facilitate others to build gRPC v1.4.x source code on ArchLinux, I create a patch, and hope it can help others.

P.S.
(1) You should fallback to OpenSSL 1.0. Please refer here:

PKG_CONFIG_PATH=/usr/lib/openssl-1.0/pkgconfig make

Otherwise you may encounter following errors:

src/core/tsi/ssl_transport_security.c: In function ‘tsi_create_ssl_client_handshaker_factory’:
src/core/tsi/ssl_transport_security.c:1281:3: error: ‘TLSv1_2_method’ is deprecated [-Werror=deprecated-declarations]
   ssl_context = SSL_CTX_new(TLSv1_2_method());
   ^~~~~~~~~~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from src/core/tsi/ssl_transport_security.c:45:
/usr/include/openssl/ssl.h:1624:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *TLSv1_2_method(void)) /* TLSv1.2 */
 ^
src/core/tsi/ssl_transport_security.c: In function ‘tsi_create_ssl_server_handshaker_factory_ex’:
src/core/tsi/ssl_transport_security.c:1389:7: error: ‘TLSv1_2_method’ is deprecated [-Werror=deprecated-declarations]
       impl->ssl_contexts[i] = SSL_CTX_new(TLSv1_2_method());
       ^~~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from src/core/tsi/ssl_transport_security.c:45:
/usr/include/openssl/ssl.h:1624:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *TLSv1_2_method(void)) /* TLSv1.2 */
 ^
At top level:
src/core/tsi/ssl_transport_security.c:118:22: error: ‘openssl_thread_id_cb’ defined but not used [-Werror=unused-functio ]
 static unsigned long openssl_thread_id_cb(void) {
                      ^~~~~~~~~~~~~~~~~~~~
src/core/tsi/ssl_transport_security.c:110:13: error: ‘openssl_locking_cb’ defined but not used [-Werror=unused-function]
 static void openssl_locking_cb(int mode, int type, const char *file, int line) {
             ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

(2) You may need to change installation directory from /usr/local to /usr:

make prefix=/usr install

This lets you process pkg-config path easily.