Kafka Tutorial for beginners -- cont

Configurations


Hardware selection

Disk Throughput

Disk capacity

Memory

    1. Partitions count, replication factor
    2. Patitions
    3. Replication: should be at least 2, usually 3, maximum 4
    4. Configure topic
    5. Retention and clean up policies(Compaction)
    6. Paritions and Segments

Configure producer

Kafka broker discovery

Options for producer configuration

Message compression for high-throughpput producer

Producer batching

Idempotent producer

Configure consumer

How kafka handle consumers exit/enter groups?

Controlling consumer liveness?

Consumer offset

Consumer offset reset behaviors

Delivery semantics for consumers

Offset management

Consumer offset commits strategies

Schema registry

Case study

Video analytics - MovieFlix

GetTaxi

Campaign compare

Mysocial media

Finance application - MyBank

Big data ingestion

Kafka internal

Request processing

Physical storage

Partition Allocation

File Management

 

Next Post Previous Post
No Comment
Add Comment
comment url