This article illustrates monitoring Kafka on Kubernetes using Prometheus and Grafana. We’ll deploy Kafka and Zookeeper, add Kafka Exporter, configure Prometheus to scrape metrics, and set up Grafana dashboards for visualization. …
Author: Vasil Kosturski
Learn how to implement leader election and failover using Zookeeper, .NET Core, and Docker. This article demonstrates building a distributed system with automatic leader election, handling failures gracefully to ensure high availability and fault tolerance. …
This article provides a practical guideline for unit and integration testing in Apache Flink. Using a financial fraud detection application as an example, we demonstrate how to write effective tests to ensure the correctness of your Flink jobs. …
This guide walks you through building a sample Data Lake in AWS by pushing a News API data stream into S3 using a Firehose C# client, analyzing the data with Athena, and setting up an ETL process via Glue to load the data into MySQL RDS. …
This blog post offers a detailed tutorial on configuring strongSwan for an AWS Site-to-Site VPN. It includes step-by-step instructions, security practices, and troubleshooting tips for robust connectivity between local networks and AWS. …
In this demo, we set up primary and replica MySQL RDS instances in two regions. We then insert some data into the primary and read from the replica, accessing the databases through EC2 hosts. We also configure a CloudWatch alarm for high replication lag and demonstrate promoting the replica. …
Discover a practical solution for the Dual Write Problem in distributed systems using the Outbox Pattern. This guide presents a C# demo app with MongoDB, Kafka, and Debezium, illustrating reliable data propagation between services. …
This article explores producing and consuming Kafka messages in Protobuf format, adhering to Cloud Events spec, and covers polymorphic data type serialization. …
In this article, we explore Kafka with Protobuf & C#, including a demo for message production & consumption. See how to manage your schema using Confluent Schema Registry and Control Center. …
A blog post about using C# to produce and consume Kafka messages in CloudEvents format. We’ll use JSON for serialization for now, but I’ll also explore Avro and Protobuf in a subsequent article. …