se.reactive-messaging.adoc Maven / Gradle / Ivy
The newest version!
///////////////////////////////////////////////////////////////////////////////
Copyright (c) 2020, 2024 Oracle and/or its affiliates.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
///////////////////////////////////////////////////////////////////////////////
= Reactive Messaging
:h1Prefix: SE
:description: Reactive Messaging support in Helidon SE
:keywords: helidon, se, messaging
:feature-name: Reactive Messaging
:rootdir: {docdir}/..
include::{rootdir}/includes/se.adoc[]
== Contents
- <>
- <>
- <>
** <>
** <>
** <>
** <>
*** <>
*** <>
*** <>
- <>
- <>
== Overview
Asynchronous messaging is a commonly used form of communication in the world of microservices.
While it is possible to start building your reactive streams directly by combining operators and
connecting them to reactive APIs, with Helidon SE Reactive Messaging, you can now use prepared
tools for repetitive use case scenarios .
include::{rootdir}/includes/dependencies.adoc[]
[source,xml]
----
io.helidon.messaging
helidon-messaging
----
== Usage
Connecting your streams to external services usually requires a lot of boilerplate
code for configuration handling, backpressure propagation, acknowledgement and more.
In Helidon there is a system of connectors, emitters and means to orchestrate these tasks
called *Reactive Messaging*. It's basically an API for connecting and configuring
connectors and emitters with your reactive streams through <>.
Reactive Messaging relates to
xref:../mp/reactivemessaging/introduction.adoc[MicroProfile Reactive Messaging] as the making of connectors and configuring them can be a repetitive task that ultimately leads to
the same results. Helidon SE Reactive Messaging supports the very same configuration format
for connectors as its MicroProfile counterpart does. Also, MP Connectors are reusable in
Helidon SE Messaging with some limitations such as there is no CDI in Helidon SE.
All <> in Helidon are made to be universally usable by Helidon MP and SE.
=== Channel
A channel is a named pair of `Publisher` and `Subscriber`. Channels can be connected together by
<>. Registering a `Publisher` or `Subscriber` for a channel can be done
by Messaging API, or configured implicitly using registered <>
to generate the `Publisher` or `Subscriber`.
[source,java]
.Example of simple channel:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_1, indent=0]
----
=== Processor
Processor is a typical reactive processor acting as a `Subscriber` to upstream and as a `Publisher`
to downstream. In terms of reactive messaging, it is able to connect two <> to one
reactive stream.
[source,java]
.Example of processor usage:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_2, indent=0]
----
=== Message
Reactive Messaging in Helidon SE uses the same concept of
message wrapping as MicroProfile messaging.
The only notable difference is that SE Messaging does almost no implicit or automatic
acknowledgement due to _no magic_ philosophy of Helidon SE.
The only exception to this are the variants of the methods `Messaging.Builder#listener` and
`Messaging.Builder#processor` configured with consumer or function parameters which will conveniently unwrap the payload
for you. Once the payload is automatically unwrapped, it is not possible to do a manual acknowledgement, therefore
an implicit acknowledgement is executed before the callback.
=== Connectors
Connectors are used to connect <> to external sources.
To make the <>
as easy and versatile as possible, Helidon SE Messaging uses the same API for connectors
that xref:../mp/reactivemessaging/introduction.adoc[MicroProfile Reactive Messaging] does.
This allows connectors to be used in both flavors of Helidon with one limitation which is
that the connector has to be able to work without CDI.
Examples of versatile connectors in Helidon include the following:
* <>
* <>
* <>
==== Messaging Connector
A connector for Reactive Messaging is a factory that produces Publishers and Subscribers for
Channels in Reactive Messaging. Messaging connector is just an implementation of
`IncomingConnectorFactory`, `OutgoingConnectorFactory` or both.
[source,java]
.Example connector `example-connector`:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_3, indent=0]
----
[source,yaml]
.Example of channel to connector mapping config:
----
mp.messaging.outgoing.to-connector-channel.connector: example-connector
mp.messaging.incoming.from-connector-channel.connector: example-connector
----
[source,java]
.Example producing to connector:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_4, indent=0]
----
[source,java]
.Example consuming from connector:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_5, indent=0]
----
===== Configuration for Messaging Connector
A messaging connector in Helidon SE can be configured explicitly by API or implicitly
by config following the notation of link:https://download.eclipse.org/microprofile/microprofile-reactive-messaging-1.0/microprofile-reactive-messaging-spec.html#_configuration[MicroProfile Reactive Messaging].
Configuration that is supplied to connector by the Messaging implementation must include
two mandatory attributes:
* `channel-name` which is the name of the channel that has the connector configured as Publisher or Subscriber, or `Channel.create('name-of-channel')` in case of explicit configuration or `mp.messaging.incoming.name-of-channel.connector: connector-name` in case of implicit config
* `connector` name of the connector `@Connector("connector-name")`
[source,java]
.Example connector accessing configuration:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_6, indent=0]
----
<1> Config context is merged from channel and connector contexts
====== Explicit Config for Messaging Connector
An explicit config for channel's publisher is possible with `Channel.Builder#publisherConfig(Config config)`
and for a subscriber with the `Channel.Builder#subscriberConfig(Config config)`.
The supplied xref:config/introduction.adoc[Helidon Config] is merged with
the mandatory attributes and any implicit configuration found. The resulting configuration is then served to the Connector.
[source,java]
.Example consuming from Kafka connector with explicit config:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_7, indent=0]
----
<1> Prepare channel for connecting kafka connector with specific publisher configuration -> listener,
<2> Channel -> connector mapping is automatic when using `KafkaConnector.configBuilder()`
<3> Prepare Kafka connector, can be used by any channel
====== Implicit Config for Messaging Connector
Implicit config without any hard-coding is possible with xref:config/introduction.adoc[Helidon Config] following notation of link:https://download.eclipse.org/microprofile/microprofile-reactive-messaging-1.0/microprofile-reactive-messaging-spec.html#_configuration[MicroProfile Reactive Messaging].
[source,yaml]
.Example of channel to connector mapping config with custom properties:
----
mp.messaging.incoming.from-connector-channel.connector: example-connector<1>
mp.messaging.incoming.from-connector-channel.first-test-prop: foo<2>
mp.messaging.connector.example-connector.second-test-prop: bar<3>
----
<1> Channel -> Connector mapping
<2> Channel configuration properties
<3> Connector configuration properties
[source,java]
.Example consuming from connector:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_8, indent=0]
----
==== Re-usability in MP Messaging
As the API is the same for xref:../mp/reactivemessaging/introduction.adoc[MicroProfile Reactive Messaging]
connectors, all that is needed to make connector work in both ways is annotating it with
`@ApplicationScoped`. Such connector is treated as a bean in Helidon MP.
For specific information about creating messaging connectors for Helidon MP visit
xref:../mp/reactivemessaging/introduction.adoc[MicroProfile Reactive Messaging].
==== Kafka Connector
[source,xml]
.Maven dependency
----
io.helidon.messaging.kafka
helidon-messaging-kafka
----
===== Reactive Kafka Connector
Connecting streams to Kafka with Reactive Messaging couldn't be easier.
===== Explicit Config with Config Builder for Kafka Connector
[source,java]
.Example of consuming from Kafka:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_9, indent=0]
----
<1> Prepare a channel for connecting kafka connector with specific publisher configuration -> listener
<2> Channel -> connector mapping is automatic when using KafkaConnector.configBuilder()
<3> Prepare Kafka connector, can be used by any channel
[source,java]
.Example of producing to Kafka:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_10, indent=0]
----
<1> Prepare a channel for connecting kafka connector with specific publisher configuration -> listener
<2> Channel -> connector mapping is automatic when using KafkaConnector.configBuilder()
<3> Prepare Kafka connector, can be used by any channel
===== Implicit Helidon Config for Kafka Connector
[source,yaml]
.Example of connector config:
----
mp.messaging:
incoming.from-kafka:
connector: helidon-kafka
topic: messaging-test-topic-1
auto.offset.reset: latest # <1>
enable.auto.commit: true
group.id: example-group-id
outgoing.to-kafka:
connector: helidon-kafka
topic: messaging-test-topic-1
connector:
helidon-kafka:
bootstrap.servers: localhost:9092 # <2>
key.serializer: org.apache.kafka.common.serialization.StringSerializer
value.serializer: org.apache.kafka.common.serialization.StringSerializer
key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
value.deserializer: org.apache.kafka.common.serialization.StringDeserializer
----
<1> Kafka client consumer's property auto.offset.reset configuration for `from-kafka` channel only
<2> Kafka client's property link:{kafka-client-base-url}#consumerconfigs_bootstrap.servers[bootstrap.servers] configuration for all channels using the connector
[source,java]
.Example of consuming from Kafka:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_11, indent=0]
----
<1> Prepare Kafka connector, can be used by any channel
[source,java]
.Example of producing to Kafka:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_12, indent=0]
----
<1> Prepare Kafka connector, can be used by any channel
Don't forget to check out the examples with pre-configured Kafka docker image, for easy testing:
* {helidon-github-examples-url}/messaging
==== JMS Connector
[source,xml]
.Maven dependency
----
io.helidon.messaging.jms
helidon-messaging-jms
----
===== Reactive JMS Connector
Connecting streams to JMS with Reactive Messaging couldn't be easier.
===== Explicit Config with Config Builder for JMS Connector
[source,java]
.Example of consuming from JMS:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_13, indent=0]
----
<1> Prepare a channel for connecting jms connector with specific publisher configuration -> listener
<2> Channel -> connector mapping is automatic when using JmsConnector.configBuilder()
<3> Prepare JMS connector, can be used by any channel
[source,java]
.Example of producing to JMS:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_14, indent=0]
----
<1> Prepare a channel for connecting jms connector with specific publisher configuration -> listener
<2> Channel -> connector mapping is automatic when using JmsConnector.configBuilder()
<3> Prepare JMS connector, can be used by any channel
===== Implicit Helidon Config for JMS Connector
[source,yaml]
.Example of connector config:
----
mp.messaging:
incoming.from-jms:
connector: helidon-jms
destination: se-example-queue-1
session-group-id: session-group-1
type: queue
outgoing.to-jms:
connector: helidon-jms
destination: se-example-queue-1
type: queue
connector:
helidon-jms:
jndi:
jms-factory: ConnectionFactory
env-properties:
java.naming.factory.initial: org.apache.activemq.jndi.ActiveMQInitialContextFactory
java.naming.provider.url: tcp://127.0.0.1:61616
----
[source,java]
.Example of consuming from JMS:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_15, indent=0]
----
<1> Prepare JMS connector, can be used by any channel
[source,java]
.Example of producing to JMS:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_16, indent=0]
----
<1> Prepare JMS connector, can be used by any channel
Don't forget to check out the examples with pre-configured ActiveMQ docker image, for easy testing:
* link:{helidon-github-examples-url}/messaging[Helidon Messaging Examples]
==== AQ Connector
[source,xml]
.Maven dependency
----
io.helidon.messaging.aq
helidon-messaging-aq
----
===== Reactive Oracle AQ Connector
===== Sending and Receiving
[source,java]
.Example of producing to and consuming from Oracle AQ:
----
include::{sourcedir}/se/ReactiveMessagingSnippets.java[tag=snippet_17, indent=0]
----
<1> Prepare Oracle UCP
<2> Setup AQ connector and provide datasource with an identifier `test-ds`
<3> Setup channel for sending messages to queue `example_queue_1` with datasource `test-ds`
<4> Setup channel for receiving messages from queue `example_queue_1` with datasource `test-ds`
<5> Register connector and channels
<6> Add a publisher for several test messages to publish them to `example_queue_1` immediately
<7> Subscribe callback for any message coming from `example_queue_1`
== Configuration
* <>
* <>
* <>
* <>
* <>
== Reference
* link:{microprofile-reactive-messaging-spec-url}[MicroProfile Reactive Messaging Specification]
* link:https://github.com/eclipse/microprofile-reactive-messaging[MicroProfile Reactive Messaging on GitHub]
* link:{helidon-github-examples-url}/messaging[Helidon Messaging Examples]