diff options
Diffstat (limited to 'ancillary/kafka_notes.md')
| -rw-r--r-- | ancillary/kafka_notes.md | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/ancillary/kafka_notes.md b/ancillary/kafka_notes.md new file mode 100644 index 0000000..e0880b0 --- /dev/null +++ b/ancillary/kafka_notes.md @@ -0,0 +1,18 @@ +# Kafka + +``` +Msg_Size * Msg_Freq/second * (Replication_factor + 1) = inbound_rate + +inbound_rate must be < 50% sustained network throughput +(c5.2xlarge = 2.5gbit sustained, so no more than 1.3gbit => 162.5megabyte/sec) + +D_days retention => inbound_rate * 86400s/day = total_storage + +1 gigabit == 125,000 kbyte + +Process 5000 msg/sec uncompressed, 4500 msgs/sec with compression turned on +(expect 10% perf penalty) — leader partitions only + +-> Ratios: 1 CPU core, 2GB Memory, ~300mbit network, ~1250 IOPS, ~35 MB/s disk throughput. +``` + |