Skip to content

Use grpc-okhttp instead of grpc-netty-shaded in the runtime jars - #548

Open
ludoch wants to merge 2 commits into
mainfrom
cloudtasks-runtime-okhttp
Open

ludoch wants to merge 2 commits into
mainfrom
cloudtasks-runtime-okhttp

Conversation

@ludoch

@ludoch ludoch commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #547 (base branch cloudtasks-runtime-jar-session-test). Only the last commit is new; retarget to main once #547 is merged.

Change

#547 bundles the Cloud Tasks client into runtime_impl_jetty12/runtime_impl_jetty121, which grows each jar by about 10.4 MB. grpc-netty-shaded alone is about 4 MB compressed of that.

Nothing in gax, gax-grpc, grpc-auth or the Cloud Tasks stubs references netty directly: the transport is found through the io.grpc.ManagedChannelProvider SPI. So this PR:

  • Bundles io.grpc:grpc-okhttp (version from grpc-bom) instead of io.grpc:grpc-netty-shaded. META-INF/services/io.grpc.ManagedChannelProvider now lists only io.grpc.okhttp.OkHttpChannelProvider.
  • Excludes okio 3.x from grpc-okhttp. okio 3 is written in Kotlin and pulls in kotlin-stdlib, about 1.5 MB compressed. Instead it uses com.squareup.okio:okio:1.17.6 (pure Java, 92 KB, managed in the root pom).
  • Drops the now-unneeded grpc-netty-shaded META-INF/native/** filter.

okio 1.17.6 compatibility

grpc-java upstream builds grpc-okhttp against okio 3, so this is an unsupported combination. I checked it at the bytecode level: every okio method and field referenced by grpc-okhttp-1.85.0.jar (61 members of Buffer, BufferedSink, BufferedSource, ByteString, Okio, Sink, Source, Timeout) exists in okio 1.17.6. okio 1.17.6 includes the fix for CVE-2023-3635. A future grpc-okhttp upgrade could start using an okio 3-only API; CloudTasksSessionTest would then fail with a class loading error. This is guarded so an automatic update can't bring Kotlin back (see Keeping okio on 1.x below).

Keeping okio on 1.x

  • okio.version (root pom) holds the pinned version. A maven-enforcer requireProperty rule in both runtime_impl poms fails the build unless it is 1.x, with a message explaining why. With okio.version=3.4.0 the build fails: okio.version must stay 1.x: the runtime jar bundles grpc-okhttp with the pure Java okio 1.x, while okio 2+ is written in Kotlin and needs kotlin-stdlib, which this jar does not bundle.
  • .github/dependabot.yml ignores com.squareup.okio:okio >= 2.0.0. Majors are already ignored globally; this entry makes the okio case explicit and survives a change to that global rule. The same is mirrored in the (disabled) renovate.json.
  • bannedDependencies was not used: maven-enforcer's dependency collection skips optional dependencies, and all dependencies of the runtime_impl poms are optional, so it never saw okio or Kotlin.

Size (mvn package, JDK 25)

runtime-impl-jetty121.jar Size Δ vs main
main 23.3 MB —
#547 (netty) 33.7 MB +10.4 MB
this PR (okhttp + okio 1.17.6) 29.3 MB +6.0 MB

runtime-impl-jetty12.jar: 23.1 → 33.5 (#547) → 29.1 MB. The increase over main drops by about 43%, saving about 4.4 MB per jar and about 8.9 MB across the runtime deployment.

Testing

JDK 25, Jetty 12.1 EE8/EE11: CloudTasksSessionTest 2/2 and JavaRuntimeAllInOneTest 12/12 pass. No ClassNotFoundException/NoClassDefFoundError appears. All Cloud Tasks calls from the runtime received PERMISSION_DENIED: Permission denied on resource project 1 from cloudtasks.googleapis.com, so TLS and HTTP/2 over okhttp work. Netty is no longer in the jars.

@ludoch

ludoch commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Note for the internal (google3) build

This PR, and #547 underneath it, only change the Maven build. The GitHub repo has no Bazel BUILD files, so the internal google3 build of the runtime jars won't pick up these changes automatically. It may not work internally unless the matching deps are also added to the corresponding BUILD targets for the runtime jars (the google3 equivalents of runtime_impl_jetty12 / runtime_impl_jetty121):

  • From Bundle the Cloud Tasks client in the runtime jars and add an e2e session test #547: the Cloud Tasks client stack used by the runtime's private class loader (e.g. DeferredDatastoreSessionStore with async session persistence and APPENGINE_USE_CLOUDTASK_PUSH_QUEUE=true): google-cloud-tasks, the v2/v2beta3 protos and gRPC stubs, proto-google-iam-v1, gax, gax-grpc, gax-httpjson, api-common, protobuf-java-util, grpc-api/core/stub/protobuf/protobuf-lite/util/auth/context, google-auth-library-credentials/oauth2-http, google-http-client/-gson, opencensus-api/-contrib-http-util, opentelemetry-api/-context, threetenbp.
  • From this PR: a gRPC transport. Here that's grpc-okhttp with okio 1.17.6, replacing grpc-netty-shaded. Internally, use whatever transport is standard in google3 (it may well already be netty there), as long as one is linked into the runtime and registered through META-INF/services/io.grpc.ManagedChannelProvider.
  • Service file merging: the internal jar build must merge META-INF/services files (the Maven build now uses ServicesResourceTransformer). Otherwise gRPC can fail at runtime to find its name resolver, load balancer or channel provider even when all the classes are present.

Without these, the internal runtime fails the same way as before #547: NoClassDefFoundError: com/google/auth/Credentials (or a missing gax/gRPC class) as soon as the runtime itself adds a task with the Cloud Tasks flag on. CloudTasksSessionTest covers this scenario if it can run internally.

Base automatically changed from cloudtasks-runtime-jar-session-test to main October 24, 2026 11:39
@ludoch
ludoch requested a review from gajjarc October 24, 2026 12:10
@ludoch

ludoch commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

not sure if this is helping to reduce jar sizes. Investigate more @gajjarc

@ludoch
ludoch force-pushed the cloudtasks-runtime-okhttp branch from 15e9a3f to 73920f3 Compare October 24, 2026 12:50
The Cloud Tasks client bundled in runtime_impl_jetty12/121 only needs a gRPC
transport found through the ManagedChannelProvider SPI; nothing in gax,
gax-grpc, grpc-auth or the Cloud Tasks stubs references netty directly.
Replace grpc-netty-shaded (about 4 MB compressed in each jar) with
grpc-okhttp.

grpc-okhttp depends on okio 3.x, which is written in Kotlin and pulls in
kotlin-stdlib (another 1.5 MB). Exclude it and use okio 1.17.6 (pure Java,
92 KB) instead: every okio method and field referenced by grpc-okhttp 1.85.0
exists in okio 1.17.6, which also has the fix for CVE-2023-3635.

runtime-impl-jetty121.jar: 33.7 MB with netty -> 29.3 MB.
The runtime jars bundle grpc-okhttp with the pure Java okio 1.17.6. okio 2+
is written in Kotlin and needs kotlin-stdlib, which the runtime jars don't
bundle (the shade artifactSet is an allowlist), so an okio upgrade would
break the Cloud Tasks client at runtime.

- Move the okio version to an okio.version property in the root pom.
- Add a maven-enforcer requireProperty rule to the runtime_impl poms that
  fails the build, with an explanation, unless okio.version is 1.x.
- Tell Dependabot to ignore okio >= 2.0.0. Majors are already ignored
  globally; this makes the okio case explicit. Mirror it in the (disabled)
  renovate.json.
@ludoch
ludoch force-pushed the cloudtasks-runtime-okhttp branch from 73920f3 to e8f9158 Compare October 24, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant