Migrate stream non-gRPC protos to LightProto#4781
Merged
lhotari merged 3 commits intoapache:masterfrom May 8, 2026
Merged
Conversation
The batchReadUnconfirmedAsync method added in apache#4739 calls LOG.error(...), but LedgerHandle was migrated to slog and only has a lowercase `log` field. Master fails to compile. Convert the call to the slog builder style used elsewhere in the file.
Migrates the stream module's non-gRPC proto definitions to LightProto: - stream/statelib/src/main/proto/kv.proto (KV state-store commands) - stream/proto/src/main/proto/cluster.proto (cluster metadata/assignment) Both files use proto3, including oneof (kv.proto) and map (cluster.proto), which LightProto handles directly. For stream/proto, cluster.proto is moved to a parallel proto-lightproto/ directory so the existing protobuf-maven-plugin keeps generating the rest of the protos (which still depend on the gRPC service stubs). Java sources that touched the migrated types are updated to use the LightProto mutable-instance API instead of the protobuf-java builder pattern. This change is independent of the other in-flight LightProto work in bookkeeper-proto and bookkeeper-server. gRPC service migration is left for a follow-up.
merlimat
force-pushed
the
lightproto-stream
branch
from
b0914bb to
5f41b36
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates the stream module's non-gRPC proto definitions from
protobuf-javato LightProto:stream/statelib/src/main/proto/kv.proto— KV state-store commands (uses proto3oneof)stream/proto/src/main/proto/cluster.proto— cluster metadata / server assignment (uses proto3map)Both proto files are proto3; LightProto handles
oneofandmapdirectly.Approach
Branched off
apache/masterand does not stack on top of #4779 / #4780 (the in-flight bookkeeper-proto migrations). The pom changes are minimal and contained to stream:pom.xml— adds<lightproto-maven-plugin.version>property.stream/statelib/pom.xml— replacesprotobuf-maven-pluginwithlightproto-maven-plugin(statelib has onlykv.proto).stream/proto/pom.xml— keepsprotobuf-maven-pluginfor the gRPC-bearing protos and addslightproto-maven-pluginforcluster.proto. To avoid both plugins generating the same types,cluster.protois moved to a parallelsrc/main/proto-lightproto/directory.Java sources that referenced the migrated types are updated to use the LightProto mutable-instance API (no Builder pattern, fields populated by chained setters or via the nested-message
setX()returning the inner message to mutate).Out of scope (follow-ups)
storage.proto,kv_rpc.proto,tests-common/rpc.proto) and the messages they import (common.proto,kv.proto,kv_store.proto,stream.proto) — these still need a custom gRPC marshaller story for LightProto types.tests-common/proto2_coder_test_messages.proto— uses proto2extensions, not supported by LightProto, and is dead code.Test plan
stream/statelib:TestKVUtils,TestRocksdbKVStore(35 tests pass)stream/storage/impl:InMemClusterMetadataStoreTest,ZkClusterMetadataStoreTest,DefaultStorageContainerControllerTest,ClusterControllerImplTest,ClusterControllerLeaderImplTest,ZkStorageContainerManagerTest(28 tests pass)