Kroxylicious release 0.9.0

December 17, 2024 by Sam Barker

The Kroxylicious project is very pleased to announce the release of Kroxylicious 0.9.0. This release introduces support for Apache Kafka 3.9.

We generally expect Kroxylicious to be both forwards- and backwards-compatible across Kafka versions just like the Apache Kafka client and broker. While we tested with pre-release versions of Apache Kafka 3.9 we discovered after it was released that we had missed a case in the protocol version negotiation. When the version of the ApiVersions RPC was updated we were unable to decode the response. In the 0.9.0 release we adopt the same behaviour as the Kafka broker and respond with API version 0 if we do not support the protocol version requested by the client - we have also expanded our test coverage to get earlier warnings of similar issues in the future.

This release also marks the introduction of our new state machine for handling connections. We have evolved the state machine at the core of the proxy to separate out the state carried at various stages of the connection lifecycle and provide stronger rules around transitions between states. This is still a work in progress due to the wide-ranging implications it has, however we think it gives us a stronger core to base everything else around. This does, for the moment, limit our ability to offer authentication offload support, we do intend to restore this eventually (please do get in touch if this use case is important to you via Slack or GitHub or even bsky).

There are many quality of life improvements in this release (in no particular order)

  • Ensure we now respond with errors instead of closing the connection when there are problems encrypting records (#1630).
  • We’ve made it easier for Filter authors to generate error responses of their own (#1630).
  • We realised forwarding partial requests from the record validation filter in the face of validation failures was a bad idea so it now rejects the whole batch (#1657).
  • We can now use EC2 instance metadata to authenticate against AWS Key Management Service (#1442).
  • We now support mutual TLS authentication on the downstream (or client side) of the proxy (#1631).
  • Key Management Systems (KMS) now have more lifecycle callbacks to better manage shared resources (#1498).
  • We now default to Log4J 2 Async logging - this is mostly a performance optimisation, but we think it should address some issues we have observed with logging in kubernetes environments (#1673).

This release upgrades Kroxylicious to Jackson 2.18 which changes how Jackson handles constructor detection, which may lead to issues when parsing config for custom filters. If after the upgrade you observe issues similar to

 com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Invalid type definition for type `com.fasterxml.jackson.databind.tofix.CreatorResolutionTest$HostPort`: Argument #0 of Creator [method com.fasterxml.jackson.databind.tofix.CreatorResolutionTest$HostPort#parse(java.lang.String)] has no property name (and is not Injectable): can not use as property-based Creator

then you need to add @JsonCreator(mode = JsonCreator.Mode.DELEGATING) to the constructor one expects Jackson to use.

See the Changelog for a full list of changes and summary of Deprecations, Changes and Removals.

Lastly, we are very pleased to announce that this Kroxylicious release contains first time contributions from:

Feedback

Please let us know, through Slack, GitHub or even bsky), if you find the project interesting or helpful and especially if you have deployed it in production.