Kroxylicious' composable filter chains and pluggable API mean that you can write your own filters to apply your own rules to the Kafka protocol, using the Java programming language.

In this quick start you will build a custom filter and use it to modify messages being sent to/consumed from Kafka, learn about filter configuration and running custom filters, and find a starting point for developing your own custom filters with your own rules and logic.

1. Getting started

1.1. Prerequisites

To start developing your own custom filters for Kroxylicious, you will need to install JDK 21.

You’ll also need to install the Apache Maven CLI and one of either Podman or Docker.

If you are using Podman, you may encounter issues with the integration tests. There are instructions here to resolve this.

1.2. Get the code

The easiest way to learn how to build custom filters is with our kroxylicious-sample module, which contains some basic find-and-replace filters for you to experiment with. Begin by downloading the latest kroxylicious-sample sources from the Kroxylicious repository.

git clone https://github.com/kroxylicious/kroxylicious.git
cd kroxylicious
git checkout v0.15.0-SNAPSHOT

2. Build the sample Filter project

Building the sample project is easy! You can build the kroxylicious-sample jar either on its own or with the rest of the Kroxylicious project.

To build all of Kroxylicious, including the sample:

mvn verify -Pdist -Dquick

Note that the quick flag prevents tests from running. For more instructions about working with the build, see the Development Guide.

3. Run the built sample Filter in a Proxy

Build both kroxylicious-sample and kroxylicious-app with the dist profile as above, then run the following command:

KROXYLICIOUS_CLASSPATH="kroxylicious-sample/target/*" kroxylicious-app/target/kroxylicious-app-0.15.0-SNAPSHOT-bin/kroxylicious-app-0.15.0-SNAPSHOT/bin/kroxylicious-start.sh --config kroxylicious-sample/sample-proxy-config.yaml

4. Configure the Filter

Filters can be added and removed by altering the filterDefinitions list in the kroxylicious-sample/sample-proxy-config.yaml file. You can also reconfigure the sample filters by changing the configuration values in this file.

The SampleFetchResponseFilter and SampleProduceRequestFilter each have two configuration values that must be specified for them to work:

  • findValue - the string the filter will search for in the produce/fetch data

  • replacementValue - the string the filter will replace the value above with

4.1. Default Configuration

The default configuration for SampleProduceRequestFilter is:

filterDefinitions:
  - name: produce-request-filter
    type: SampleProduceRequestFilterFactory
    config:
      findValue: foo
      replacementValue: bar

This means that it will search for the string foo in the produce data and replace all occurrences with the string bar. For example, if a Kafka Producer sent a produce request with data {"myValue":"foo"}, the filter would transform this into {"myValue":"bar"} and Kroxylicious would send that to the Kafka Broker instead.

The default configuration for SampleFetchResponseFilter is:

filterDefinitions:
  - name: fetch-response-filter
    type: SampleFetchResponseFilterFactory
    config:
      findValue: bar
      replacementValue: baz

This means that it will search for the string bar in the fetch data and replace all occurrences with the string baz. For example, if a Kafka Broker sent a fetch response with data {"myValue":"bar"}, the filter would transform this into {"myValue":"baz"} and Kroxylicious would send that to the Kafka Consumer instead.

4.2. Modify

Now that you know how the sample filters work, you can start modifying them! Replace the SampleFilterTransformer logic with your own code, change which messages they apply to, or whatever else you like!