Message Routing
Kafka gives routing capabilities via Kafka Join and Kafka Streams, together with content-based routing, message transformation, and message enrichment.
Memphis message routing is just like the implementation of RabbitMQ utilizing routing keys, wildcards, content-based routing, and extra. Much like RabbitMQ, it’s also embedded throughout the dealer and doesn’t require exterior libraries or instruments.
Log Compaction
Compaction has been created to help a long-term, probably infinite report retailer based mostly on particular keys.
Kafka helps native matter compaction, which runs on all brokers. This runs mechanically for compacted subjects, condensing the log right down to the newest model of messages sharing the identical key.
In the intervening time, Memphis doesn’t help compaction, however it would sooner or later.
Message Replay
The flexibility to re-consume dedicated messages.
Kafka does help replay by looking for particular offsets because the shoppers have management over resetting the offset.
Memphis doesn’t help replay but however will within the close to future (2023).
Stream Enrichment
Kafka, with its Kafka Streams library, permits builders to implement elastic and scalable shopper functions that may leverage important stream processing options similar to tables, joins, and aggregations of a number of subjects and export to a number of sources through Kafka join.
Memphis gives an identical habits and extra. Embedded contained in the dealer, Memphis customers can create serverless-type features or full containerized functions that combination a number of stations and streams, beautify and enrich messages from totally different sources, write advanced features that can’t be achieved through SQL, and manipulate the schema. As well as, Memphis embedded connectors frameworks will assist to push the outcomes on to an outlined sink.
Pull Retry Mechanism
In case of a failure or lack of skill to acknowledge consumed messages, there needs to be a retry mechanism that can retry to tug the identical offset or batch of messages.
In Kafka, it’s the shopper’s duty to implement one. Some key elements have to be thought of to implement such a mechanism, like blocking vs. non-blocking, offset monitoring, idempotency, and extra.
In Memphis, the retry mechanism is built-in and turned on by default throughout the SDK and dealer. Throughout client creation, the parameter maxMsgDeliveries will decide the variety of retries the station will ship a message if an acknowledgment doesn’t arrive until maxAckTimeMs . The dealer itself information the offsets given and can expose solely the unacknowledged ones to the retry request.
