Kafka n8n integrationAutomate Kafka with n8n.
Your brokers already carry the events, n8n decides what happens next. The Kafka n8n integration gives you 1 node operation to publish a message on a topic, plus a trigger that consumes a topic under a consumer group ID. Written for teams that already run Kafka.
Verified Trustpilot reviews · AI, automation & growth agency
What does the Kafka n8n integration actually cover?
Kafka is where the events pile up. The n8n Kafka node writes to that pile, the Kafka Trigger reads from it. One operation publishes a message on a topic you name, with an optional key and headers. One trigger subscribes to a topic under a consumer group ID, the identifier several consumers share to split a topic between them, and hands each message to the rest of the workflow.
First scenario, landing a stream somewhere people can read it. The trigger consumes a topic, a Set node keeps the three fields that matter, and a Postgres node writes one row per message. Same shape, different audience: send the cleaned rows to Google Sheets when the people who read them do not have database access.
Second scenario, turning a topic into a human alert. A topic of failed jobs feeds an IF node, and anything above your threshold posts to Slack with the message key in the text. The consumer group ID keeps this workflow independent from whatever else consumes the same topic.
Third scenario, n8n as the producer. Any workflow can finish on default.execute and publish what it built. One public template does exactly that: an HTTP Request node fetches the position of the ISS every minute and sends it to a Kafka topic. Replace the API call with a form submission or a database query and the pattern holds.
One thing to know before you plan around it. The Kafka credential opens a connection to your brokers, it is not an HTTP API, so there is no generic node to fall back on when something is missing. On an HTTP based tool you would reach for HTTP Request and call the endpoint yourself. Here, what the node does not expose simply does not happen from n8n: no topic creation from a workflow, no consumer group inspection, no offset browsing.
The other known limit sits on the trigger. Version 1 consumes uncompressed messages and GZIP messages, and fails with an unsupported compression format error on LZ4, Snappy or Zstd, which is a common default for Confluent and JVM producers. Either the producer switches to gzip or none, or you opt into the version 2 preview. If you want that reviewed against your own setup before you commit, the n8n review and the n8n training pages go deeper on where n8n fits.
How do you connect n8n to your Kafka brokers?
- 01
Create the Kafka credential
In n8n, open Credentials and create a Kafka credential. A credential is the stored connection detail that every node reuses, so you fill this in once for all your Kafka workflows. Start with Client ID, the identifier of the client or consumer group that n8n announces to the cluster. Pick something you will recognise in your broker logs, because that name is how you will tell n8n traffic apart from the rest.
- 02
List your brokers
Brokers takes a comma separated list in the
<broker-service-name>:<port>format. The service name is the one you gave the broker in yourserviceslist, sokafka-1:9092,kafka-2:9092points n8n at two brokers on port 9092. List more than one when you have more than one. If your Kafka environment runs without SSL, turn the SSL toggle off, otherwise leave it on. - 03
Add SASL authentication if you use it
If your cluster authenticates with SASL, turn on the Authentication toggle and fill in Username and Password. Then set SASL Mechanism to whatever the broker is configured for, between
Plain,scram-sha-256andscram-sha-512. Save, then reuse the same credential on both the Kafka node and the Kafka Trigger. Encoding with an authenticated Confluent Schema Registry needs a separate Schema Registry credential.
What does the Kafka Trigger listen to?
Kafka Trigger is the node that starts a workflow when something happens in Kafka. Activate the workflow and every event it receives becomes one execution.
What you see in n8n
Configuration notes
01Set Up the Kafka Trigger
The Kafka Trigger subscribes to one topic and starts the workflow for the messages it receives. Two fields are required, and the workflow has to be active for the trigger to listen at all.
Key parameters
- Topic: the name of the topic to consume from, exactly as it exists on the cluster.
- Group ID: the consumer group this trigger joins. Give each workflow its own ID unless you deliberately want two triggers sharing the partitions of a topic.
- Allow Topic Creation: lets the consumer attach to a topic that does not exist yet, instead of failing on it.
02Choose Where Reading Starts
By default a consumer group picks up where it left off. These options decide what happens on the first run and how often the position is written back.
Key parameters
- Read Messages From Beginning: on, the trigger reads the topic from its oldest retained message instead of only new ones. Useful to backfill once, noisy if you leave it on.
- Auto Commit Threshold: commits the offset after a given number of resolved messages.
- Auto Commit Interval: commits after a given period instead, for example every few seconds.
- Batch Size: how many messages are processed per batch. Set to 1 for message by message processing.
03Shape What Each Message Outputs
Kafka hands over a value and some metadata around it. These options decide what the next node actually sees in its input, and getting them right at the trigger saves a cleanup node later in the workflow.
Key parameters
- JSON Parse Message: tries to parse the message into an object, so the fields become addressable as
{{ $json.field }}instead of one long string. - Only Message: returns just the message property, dropping the surrounding envelope.
- Return Headers: adds the headers received from Kafka to the output, which is where producers usually put a trace or tenant identifier.
- Keep Message as Binary Data: keeps the value binary for downstream processing such as Avro deserialization.
- Use Schema Registry: decodes through a Confluent Schema Registry, with its own credential.
04Keep The Consumer In Its Group
A Kafka consumer proves it is alive on a schedule. These options control that conversation, and they are the ones to look at when a workflow keeps losing its place.
Key parameters
- Heartbeat Interval: how often the consumer tells the broker it is still alive. It has to be lower than Session Timeout, and roughly one third of it is the recommended value.
- Session Timeout: the delay in milliseconds after which the broker treats the consumer as failed.
- Rebalance Timeout: the maximum time a consumer gets to join the group.
- Retry Delay on Error: the wait in milliseconds before retrying a failed offset resolution, which stops a rapid retry loop from hammering the broker.
What can the Kafka node do?
The Kafka node exposes one operation. For each one: the node as you configure it in n8n, the required fields, and our field notes.
Sends messages to a Kafka topic
executeWhat you see in n8n
Notes & use cases
The node publishes one message per incoming item. Kafka returns an acknowledgement, not data, so this is an endpoint in a workflow rather than a lookup.
Key parameters
- Topic: where the message goes, for example
orders-created. - Message: the body to send, often an expression like
{{ $json.id }}. Turn on Send Input Data to push the whole item as JSON instead. - Key: the message key, which is how Kafka keeps related messages on one partition.
- Event Name: the schema written
namespace.name, used when Use Schema Registry is on.
Need help automating Kafka with n8n?
A person reads every message.
Kafka and n8n, common questions
01Is the Kafka n8n integration free?
02What credentials does n8n need for Kafka?
03What are the limits of the Kafka node in n8n?
04Does the Kafka Trigger react in real time?
05n8n or Make for Kafka?
Get our weekly integration tips.
No spam. Unsubscribe anytime.
