Learn how to implement leader election and failover using Zookeeper, .NET Core, and Docker. This article demonstrates building a distributed system with automatic leader election, handling failures gracefully to ensure high availability and fault tolerance. …
Category: Distributed Systems
This article provides a practical guideline for unit and integration testing in Apache Flink. Using a financial fraud detection application as an example, we demonstrate how to write effective tests to ensure the correctness of your Flink jobs. …
Discover a practical solution for the Dual Write Problem in distributed systems using the Outbox Pattern. This guide presents a C# demo app with MongoDB, Kafka, and Debezium, illustrating reliable data propagation between services. …
This article explores producing and consuming Kafka messages in Protobuf format, adhering to Cloud Events spec, and covers polymorphic data type serialization. …
In this article, we explore Kafka with Protobuf & C#, including a demo for message production & consumption. See how to manage your schema using Confluent Schema Registry and Control Center. …
A blog post about using C# to produce and consume Kafka messages in CloudEvents format. We’ll use JSON for serialization for now, but I’ll also explore Avro and Protobuf in a subsequent article. …
Exploring Kafka Steams Partitioning, Scaling, and Fault Tolerance – we’ll build a simple streaming app and inspect the contents of the repartition topic, RocksDB state store, and the Kafka internal changelog topic. Also, we’ll see how the workload gets distributed between the running app instances. …
An article about integration tests for your Kafka workflow. Using C# and the “Testcontainers” framework, we first run Kafka in Docker, call an API that will push a message to Kafka, consume the produced Kafka message ensuring it’s correct, and finally drop the created containers. …
A blog post where we use Wireshark to trace the Kafka networking protocol. The client is a simple C# app interacting with a multi-broker Kafka cluster. …
In this article, I’ll introduce Kafka listeners, the client-broker “handshake” and pub/sub interactions. …