Skip to main content

Deep dive on why gRPC Stream can not work with Cocos Creator

· 3 min read
tony

Version​

What is gRPC stream?​

A gRPC stream refers to a communication channel that allows bidirectional data flow between a client and a server in real-time. It's one of the key features of gRPC and is used for various purposes, including real-time updates, event-driven communication, and efficient data streaming

We can follow the older post in here: gRPC stream vs Websocket

What is Cocos Creator?​

Cocos Creator is both an efficient, lightweight, free and open source cross-platform 2D & 3D graphics engine and a real-time interactive 2D & 3D digital content creation platform. With many advantages such as High Performance, Low Power Consumption, Streaming Loading and Cross-Platform Publishing, we can use it to create projects in fields such as Games, Cars, XR, MetaVerse etc.

Problem​

We are having issues using gRPC Stream in our project because it is not supported by Cocos Creator; let's take a deeper look at why this is the case.

Testing gRPC stream with Cocos Creator​

Crearte streaming_test.proto

service StreamingTest {
    rpc BiDirectional(stream Message) returns (stream Message);
    rpc SeverStreaming(SeverStreamingRequest) returns (stream Message);
    rpc ClientStreaming(stream Message) returns (ClientStreamReply);
}

message Message {
    string id = 1;    
    bytes metadata = 2;
    google.protobuf.Timestamp created_at = 3;
}

message SeverStreamingRequest {}

message ClientStreamReply {}

grpc/grpc-web​

  1. Set up gRPC-web
  1. Generate
$ protoc -I=$DIR streaming_test.proto \
    --js_out=import_style=commonjs:$OUT_DIR \
    --grpc-web_out=import_style=commonjs,mode=grpcwebtext:$OUT_DIR

But... Client-side and Bi-directional streaming is not currently supported

Alt Text

improbable-eng/grpc-web​

gRPC Web implementation for Golang and TypeScript, a custom lib forked from grpc/grpc-web

It sill not working with Cocos Creator 😞

Root cause​

The issue we're facing pertains to using the "connect" method in Cocos, which is challenging due to Cocos using an older JavaScript engine that doesn't support newer APIs. grpc-web stream specifically require the Fetch API for out project.

We attempted to implement the Fetch API through XMLHttpRequest using a polyfill. While we were successful in making unary (single) method calls, we encountered difficulties with streaming. This is because XMLHttpRequest doesn't support reading a partial response body (chunk) - we must wait for the request to complete before accessing the entire body.

To fully address this issue, we would need to dive deeper into native code and create a custom implementation of the Fetch API that supports streaming. Keep in mind that this is our experience with Android, and we would need to separately implement and test for iOS.

It's also worth noting that the grpc-web library faces a similar limitation as it uses XMLHttpRequest, it is unable to perform asynchronous operations, which could be the reason it doesn't fully support streaming.

KaKa

Therefore... we use websocket instead 😁

Alt Text

References​