Posts

Showing posts with the label Kafka

Observability Done Right: Best Practices and Anti-Patterns for Effective System Monitoring

-
Image
  WHAT Observability is a concept that refers to the ability to gain insights into the behavior and performance of complex systems. In the context of software engineering, observability involves the collection, analysis, and visualization of data from software applications, infrastructure, and other components of a system. In the animal kingdom, observability plays a critical role in survival, allowing animals to monitor their surroundings, detect threats, and find food. Dolphins use echolocation to observe their surroundings. They emit high-frequency sounds that bounce off objects, allowing them to create a 3D map of their environment. Thanks for reading Knowledge Cafe! Subscribe for free to receive new posts and support my work. Subscribed WHY In today's era, architectures are becoming increasingly large, complex, and fast-paced due to the faster development and deployment of software by distributed teams with the help of DevOps, continuous delivery, and agile development methodo...
Explore more »

Building Scalable and Resilient Systems

-
Image
What is Event Driven Architecture (EDA) In an event-driven architecture, events are the primary unit of communication between different components of a system. Event producers generate events, which are then sent to an event router or message broker. The event router filters and routes the events to the appropriate event consumers, which take appropriate action based on the events they receive. This decoupling of event producers and consumers allows for a loosely coupled and asynchronous communication between different parts of the system, making it more scalable, flexible, and resilient. The event-driven architecture is well-suited for real-time systems such as IoT, online gaming, financial trading, and real-time analytics. Ways to Implement EDA Event-driven architecture (EDA) can be implemented in a variety of ways, depending on the specific requirements of the system. Here are a few common ways to implement EDA: Using a message broker:  A message broker, such as Apache Kafka or ...
Explore more »

Apache Kafka Use cases

-
Image
Kafka was created first in the tech labs of LinkedIn – the world’s biggest network of professionals. Apache Kafka is the most popular open-source stream-processing software for collecting, processing, storing, and analyzing data at scale. Most known for its excellent performance, low latency, fault tolerance, and high throughput, it's capable of handling thousands of messages per second. Website activity tracking(E commerce) Website activity (product views, product searches) is published to customer activity topics and becomes sourced to real-time processing, offline analytics to tool like Google’s BigQuery, Azure CosmosDB, RedShift. Real time Fraud detection Juniper Research estimates online FDP (fraud detection and prevention) spending is all set to touch $9.3 billion by 2022. Real-time data streaming helps to spot anomalies – that is peculiar and abnormal instances that deviate from the usual way of operation. These anomalies can be classified as fraud or errors. Apache Kafka he...
Explore more »