Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/proto.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
on:
push:
paths:
- 'proto/**'
- '.github/workflows/proto.yml'
pull_request:
paths:
- 'proto/**'
- '.github/workflows/proto.yml'

name: Proto
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
proto:
runs-on: ubuntu-latest
env:
BUF_VERSION: 1.55.1
PROTOC_GEN_GO_VERSION: v1.36.11
PROTOC_GEN_GO_GRPC_VERSION: v1.6.1
steps:
# actions/checkout v6
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10
with:
fetch-depth: 0
persist-credentials: false
- name: Fetch main for breaking baseline
if: github.ref != 'refs/heads/main'
run: git fetch --no-tags --prune origin +refs/heads/main:refs/heads/buf-breaking-base
# actions/setup-go v6
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c
with:
go-version-file: 'go.mod'
- name: Install buf
run: go install github.com/bufbuild/buf/cmd/buf@v${{ env.BUF_VERSION }}
- name: Install protoc-gen-go
run: go install google.golang.org/protobuf/cmd/protoc-gen-go@${{ env.PROTOC_GEN_GO_VERSION }}
- name: Install protoc-gen-go-grpc
run: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@${{ env.PROTOC_GEN_GO_GRPC_VERSION }}
- name: buf breaking (against main)
if: github.ref != 'refs/heads/main'
working-directory: proto
run: buf breaking --against '../.git#subdir=proto,branch=buf-breaking-base'
- name: Clean generated proto outputs
run: find proto -maxdepth 1 -type f \( -name '*.pb.go' -o -name '*_grpc.pb.go' \) -delete
- name: buf generate
working-directory: proto
run: buf generate

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Detect stale generated files after proto deletions

When a .proto file is removed or renamed, this in-place buf generate leaves the old tracked *.pb.go and *_grpc.pb.go files untouched; because those files are already tracked and unchanged, the later git add -N/git diff -- proto/ still exits successfully and stale generated APIs can remain committed. Clean or explicitly remove only the generated outputs before regenerating so deletions are surfaced by the drift check.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clean stale generated files before diffing

Because this runs a plain buf generate before diffing, generated Go files for a .proto that was removed or renamed are left untouched; the Buf docs describe clean generation as the mechanism that prevents removed .proto types from leaving stale generated files. In a PR that deletes proto/foo.proto but forgets foo.pb.go, the following git diff remains clean and go build can pass against the stale API, so this gate misses that drift; add a safe generated-file cleanup (not a blind clean of out: .) before generating.

Useful? React with 👍 / 👎.

- name: Fail on regenerated diff
run: |
# intent-to-add surfaces newly generated .pb.go that would
# otherwise be invisible to `git diff` (adding a .proto without
# committing its generated .pb.go must fail this gate).
git add -N proto/
if ! git diff --exit-code -- proto/; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Detect untracked generated files in proto drift check

The Fail on regenerated diff step can miss missing generated artifacts because git diff --exit-code -- proto/ only compares tracked changes (index/worktree) and returns success when buf generate creates new untracked *.pb.go files. In that case CI can pass even though generation output is not fully committed (e.g., adding a new .proto file and forgetting to git add the new generated file), so this gate does not reliably enforce the intended “committed output matches generated output” invariant.

Useful? React with 👍 / 👎.

echo "::error::Generated proto files differ from committed output. Run 'make -C proto gen' and commit the result."
exit 1
fi
- name: go build
run: go build ./...
93 changes: 57 additions & 36 deletions proto/Makefile
Original file line number Diff line number Diff line change
@@ -1,48 +1,69 @@
PROTOC_VERSION := libprotoc 29.3
PROTOC_GEN_GO_VERSION := protoc-gen-go v1.36.11
PROTOC_GEN_GO_GRPC_VERSION := protoc-gen-go-grpc 1.6.1
BUF_VERSION := 1.55.1
PROTOC_GEN_GO_VERSION := 1.36.11
PROTOC_GEN_GO_GRPC_VERSION := 1.6.1

.PHONY: all gen check-tools
BUF ?= buf
# BREAKING_AGAINST lets callers override the auto-detected baseline.
BREAKING_AGAINST ?=

.PHONY: all gen check-tools breaking clean-generated

all: gen

check-tools:
@if [ "$$(protoc --version)" != "$(PROTOC_VERSION)" ]; then \
echo "expected $(PROTOC_VERSION), got $$(protoc --version)"; \
@command -v $(BUF) >/dev/null 2>&1 || { \
echo "buf not found; install with:"; \
echo " go install github.com/bufbuild/buf/cmd/buf@v$(BUF_VERSION)"; \
exit 1; \
}
@installed=$$($(BUF) --version 2>/dev/null); \
if [ "$$installed" != "$(BUF_VERSION)" ]; then \
echo "buf version mismatch: expected $(BUF_VERSION), got $$installed"; \
echo "install with: go install github.com/bufbuild/buf/cmd/buf@v$(BUF_VERSION)"; \
exit 1; \
fi
@if [ "$$(protoc-gen-go --version)" != "$(PROTOC_GEN_GO_VERSION)" ]; then \
echo "expected $(PROTOC_GEN_GO_VERSION), got $$(protoc-gen-go --version)"; \
@command -v protoc-gen-go >/dev/null 2>&1 || { \
echo "protoc-gen-go not found; install with:"; \
echo " go install google.golang.org/protobuf/cmd/protoc-gen-go@v$(PROTOC_GEN_GO_VERSION)"; \
exit 1; \
}
@installed=$$(protoc-gen-go --version 2>/dev/null); \
if [ "$$installed" != "protoc-gen-go v$(PROTOC_GEN_GO_VERSION)" ]; then \
echo "protoc-gen-go version mismatch: expected v$(PROTOC_GEN_GO_VERSION), got $$installed"; \
echo "install with: go install google.golang.org/protobuf/cmd/protoc-gen-go@v$(PROTOC_GEN_GO_VERSION)"; \
exit 1; \
fi
@if [ "$$(protoc-gen-go-grpc --version)" != "$(PROTOC_GEN_GO_GRPC_VERSION)" ]; then \
echo "expected $(PROTOC_GEN_GO_GRPC_VERSION), got $$(protoc-gen-go-grpc --version)"; \
@command -v protoc-gen-go-grpc >/dev/null 2>&1 || { \
echo "protoc-gen-go-grpc not found; install with:"; \
echo " go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v$(PROTOC_GEN_GO_GRPC_VERSION)"; \
exit 1; \
}
@installed=$$(protoc-gen-go-grpc --version 2>/dev/null); \
if [ "$$installed" != "protoc-gen-go-grpc $(PROTOC_GEN_GO_GRPC_VERSION)" ]; then \
echo "protoc-gen-go-grpc version mismatch: expected $(PROTOC_GEN_GO_GRPC_VERSION), got $$installed"; \
echo "install with: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v$(PROTOC_GEN_GO_GRPC_VERSION)"; \
exit 1; \
fi

gen: check-tools
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
service.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
internal.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
distribution.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
admin.proto
protoc --go_out=. --go_opt=paths=source_relative \
dynamodb_internal.proto
protoc --go_out=. --go_opt=paths=source_relative \
redis_internal.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
etcd_raft.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
admin_forward.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
encryption_admin.proto
breaking: check-tools
@baseline='$(BREAKING_AGAINST)'; \
if [ -z "$$baseline" ]; then \
if git -C .. rev-parse --verify origin/main >/dev/null 2>&1; then \
git -C .. branch --force buf-breaking-base origin/main >/dev/null; \
baseline='../.git#subdir=proto,branch=buf-breaking-base'; \
elif git -C .. rev-parse --verify main >/dev/null 2>&1; then \
baseline='../.git#subdir=proto,branch=main'; \
else \
echo "no breaking baseline found: neither 'main' nor 'origin/main' is reachable"; \
echo "fetch main first (git fetch origin main) or override with BREAKING_AGAINST=..."; \
exit 1; \
fi; \
fi; \
$(BUF) breaking --against "$$baseline"

clean-generated:
find . -maxdepth 1 -type f \( -name '*.pb.go' -o -name '*_grpc.pb.go' \) -delete

gen: check-tools breaking
$(MAKE) clean-generated
$(BUF) generate
2 changes: 1 addition & 1 deletion proto/admin.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/admin_forward.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/admin_forward_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/admin_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions proto/buf.gen.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: v2
plugins:
- local: protoc-gen-go
out: .
opt: paths=source_relative
- local: protoc-gen-go-grpc
out: .
opt: paths=source_relative
6 changes: 6 additions & 0 deletions proto/buf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: v2
modules:
- path: .
breaking:
use:
- WIRE

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Protect generated gRPC APIs, not just wire format

Using only WIRE means this new required breaking gate ignores generated source/API breakage; Buf's rules docs state that WIRE detects binary encoding breakage and does not check generated source code. In this repo's public gRPC protos, a PR can delete or rename an RPC such as RawKV.RawPut and still pass buf breaking, while generated clients/servers and existing gRPC callers break; use PACKAGE/FILE or add the RPC/service deletion rules if the gate is meant to prevent API-breaking proto changes.

Useful? React with 👍 / 👎.

2 changes: 1 addition & 1 deletion proto/distribution.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/distribution_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/dynamodb_internal.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/encryption_admin.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/encryption_admin_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/etcd_raft.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/etcd_raft_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/internal.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions proto/internal.proto
Original file line number Diff line number Diff line change
Expand Up @@ -80,5 +80,3 @@ message RelayPublishResponse {
}




6 changes: 3 additions & 3 deletions proto/internal_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/redis_internal.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/service.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion proto/service_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading