Kafka开发服务(Dev Services)
If any Kafka-related extension is present (e.g. quarkus-messaging-kafka
), Dev Services for Kafka automatically starts a Kafka broker in dev mode and when running tests.
So, you don’t have to start a broker manually.
The application is configured automatically.
Because starting a Kafka broker can be long, Dev Services for Kafka uses Redpanda, a Kafka compatible broker which starts in ~1 second. |
启用/禁用Kafka开发服务
Kafka的开发服务是自动启用的,除非:
-
quarkus.kafka.devservices.enabled
被设置为false
-
配置了
kafka.bootstrap.servers
-
所有的Reactive Messaging Kafka通道都设置了
bootstrap.servers
属性
Kafka的开发服务依赖于Docker来启动broker。如果您的环境不支持Docker,您需要手动启动broker,或者连接到一个已经运行的broker。您可以使用 kafka.bootstrap.servers
来配置broker地址。
共享的broker
大多数情况下,您需要在应用程序之间共享broker。Kafka的开发服务实现了一个 服务发现(service discovery) 机制,可以让您的多个在 开发 模式下运行的Quarkus应用程序共享一个broker。
Kafka开发服务用 quarkus-dev-service-kafka 标签来启动容器,该标签用于识别容器。
|
如果您需要多个(共享的)broker,您可以配置 quarkus.kafka.devservices.service-name
属性并指明broker的名称。它会查询一个具有相同名称的容器,如果找不到的话就启动一个新的容器。默认的服务名称是 kafka
。
在开发模式下,共享是默认启用的,但在测试模式下是禁用的。您可以用 quarkus.kafka.devservices.shared=false
停用共享。
设置端口
默认情况下,Kafka开发服务会随机挑选一个端口并配置应用程序。您可以通过配置 quarkus.kafka.devservices.port
属性来设置端口。
注意,Kafka的广告地址(advertised address)会自动配置为所选择的端口。
配置镜像
Dev Services for Kafka supports Redpanda, kafka-native and Strimzi (in Kraft mode) images.
Redpanda is a Kafka compatible event streaming platform.
Because it provides a fast startup times, Dev Services defaults to Redpanda images from redpandadata/redpanda
.
You can select any version from https://hub.docker.com/r/redpandadata/redpanda.
kafka-native provides images of standard Apache Kafka distribution compiled to native binary using Quarkus and GraalVM. While still being experimental, it provides very fast startup times with small footprint.
Image type can be configured using
quarkus.kafka.devservices.provider=kafka-native
Strimzi provides container images and Operators for running Apache Kafka on Kubernetes. While Strimzi is optimized for Kubernetes, the images work perfectly in classic container environments. Strimzi container images run "genuine" Kafka broker on JVM, which is slower to start.
quarkus.kafka.devservices.provider=strimzi
对于Strimzi,您可以从 https://quay.io/repository/strimzi-test-container/test-container?tab=tags ,选择任何可以获得Kraft支持的Kafka版本(2.8.1及以上)的镜像
quarkus.kafka.devservices.image-name=quay.io/strimzi-test-container/test-container:0.106.0-kafka-3.7.0
配置Kafka主题
您可以配置Kafka开发服务来在broker启动后创建主题。主题以给定的分区数量以及1个副本创建。
下面的例子创建了一个名为 test
,有3个分区的主题,以及另一个名为 messages
,有2个分区的主题。
quarkus.kafka.devservices.topic-partitions.test=3
quarkus.kafka.devservices.topic-partitions.messages=2
如果具有某个指定名称的主题已经存在,那么创建会被跳过,并且不会尝试对已经存在的主题重新分区到不同的数量。
您可以通过 quarkus.kafka.devservices.topic-partitions-timeout
来配置在主题创建中所使用的Kafka管理员客户端调用的超时时间,默认为2秒。
Transactional and Idempotent producers support
By default, the Redpanda broker is configured to enable transactions and idempotence features. You can disable those using:
quarkus.kafka.devservices.redpanda.transaction-enabled=false
Redpanda transactions does not support exactly-once processing. |
Configuration reference
Configuration property fixed at build time - All other configuration properties are overridable at runtime
Configuration property |
类型 |
默认 |
---|---|---|
If Dev Services for Kafka has been explicitly enabled or disabled. Dev Services are generally enabled by default, unless there is an existing configuration present. For Kafka, Dev Services starts a broker unless Environment variable: Show more |
boolean |
|
Optional fixed port the dev service will listen to. If not defined, the port will be chosen randomly. Environment variable: Show more |
int |
|
Kafka dev service container type. Redpanda, Strimzi and kafka-native container providers are supported. Default is redpanda. For Redpanda: See https://docs.redpanda.com/current/get-started/quick-start/ and https://hub.docker.com/r/redpandadata/redpanda For Strimzi: See https://github.com/strimzi/test-container and https://quay.io/repository/strimzi-test-container/test-container For Kafka Native: See https://github.com/ozangunalp/kafka-native and https://quay.io/repository/ogunalp/kafka-native Note that Strimzi and Kafka Native images are launched in Kraft mode. Environment variable: Show more |
|
|
The Kafka container image to use. Dependent on the provider. Environment variable: Show more |
string |
|
Indicates if the Kafka broker managed by Quarkus Dev Services is shared. When shared, Quarkus looks for running containers using label-based service discovery. If a matching container is found, it is used, and so a second one is not started. Otherwise, Dev Services for Kafka starts a new container. The discovery uses the Container sharing is only used in dev mode. Environment variable: Show more |
boolean |
|
The value of the This property is used when you need multiple shared Kafka brokers. Environment variable: Show more |
string |
|
The topic-partition pairs to create in the Dev Services Kafka broker. After the broker is started, given topics with partitions are created, skipping already existing topics. For example, The topic creation will not try to re-partition existing topics with different number of partitions. Environment variable: Show more |
Map<String,Integer> |
|
Timeout for admin client calls used in topic creation. Defaults to 2 seconds. Environment variable: Show more |
|
|
Environment variables that are passed to the container. Environment variable: Show more |
Map<String,String> |
|
Enables transaction support. Also enables the producer idempotence. Find more info about Redpanda transaction support on https://vectorized.io/blog/fast-transactions/. Notice that KIP-447 (producer scalability for exactly once semantic) and KIP-360 (Improve reliability of idempotent/transactional producer) are not supported. Environment variable: Show more |
boolean |
|
Port to access the Redpanda HTTP Proxy (pandaproxy). If not defined, the port will be chosen randomly. Environment variable: Show more |
int |
About the Duration format
To write duration values, use the standard You can also use a simplified format, starting with a number:
In other cases, the simplified format is translated to the
|