TEP-7: Streaming Support in Timeleap RPC

Abstract

This document proposes an extension to the Timeleap RPC protocol to support streamed responses. The current request-response model marks a request as complete upon the first response. This TEP introduces a new opcode Stream, which allows returning data in multiple chunks while keeping the request alive until a final result or status is sent through the normal return mechanism.

Motivation

The current RPC protocol in the Timeleap network only supports single-response semantics. This limitation makes it difficult to serve large payloads or long-running computations, especially when intermediate results can be useful to the client.

Supporting streamed responses allows the network to:

• Deliver partial responses : Useful when dealing with large datasets or real-time data.
• Terminate early on failure : Servers can end the stream with a failure response through the normal RPC return.
• Reduce memory pressure : Clients can process results incrementally instead of waiting for a full response.
• Preserve status semantics : Final outcome (success or failure) is still returned as a standard RPC response after the stream ends.

This streaming mechanism is particularly valuable for AI and ML applications. In the current Timeleap AI plugin, inference jobs such as text-to-image, image-to-image, or language model (LLM) generation benefit significantly from the ability to stream tokens or intermediate outputs as they are produced.

Specification

This section defines the proposed changes to the RPC protocol to support streaming.

Stream Opcode

A new opcode Stream is introduced to send chunks of data during an active RPC call, without terminating the request.

Each Stream opcode carries a ID field, which maps the chunk to the corresponding user socket or session in the broker. This ensures the broker can route the stream data to the correct destination.

Multiple Stream messages can be sent during the lifetime of an RPC call. The stream ends when the handler returns a final result via the normal Return opcode, or a failure is raised. The request is only considered complete when this final message is sent.

Use Cases

• Paginated queries : Fetching large result sets across multiple pages.
• Real-time metrics : Streaming logs or status updates as they happen.
• Blob or file delivery : Transferring large files or datasets chunk by chunk.
• Long computations : Returning intermediate results before finalization.

Rationale

The decision to keep StreamMessage simple, ensures minimal overhead and easy routing. Since the lifecycle of a stream is already tied to the RPC call, there is no need for extra flags to indicate termination or success.

This design keeps the protocol clean and consistent: the Stream opcode is purely for intermediate transmission, while the normal Return or error opcode finalizes the request, just like in current implementations.

Backwards Compatibility

The streaming extension is not backward compatible. Timeleap RPC is a versioned protocol, and support for the Stream opcode is only available starting from the version in which this TEP is implemented.

Reference Implementation

Pending

References

N/A