there is no way to control partition strategy

[daniel-garcia]

The kafka client used in this project uses kafka.PartitionAny. This strategy doesn't let the user control how the data is spread onto the partitions on the topic. As a result, writing downstream processes that are partition-based is hampered. For example, if I was writing an application that computed gauge from a counter or an alert based on the duration of some condition, the application would need to have a centralized state so that partition consumers could consult the historical state to evaluate the result. However, if the metrics were reliably hashed onto a partition, the consumers could be operated with consumer-specific state (or in-memory state).

[kd7lxl]

In addition to the use case mentioned, sending samples of a metric to multiple partitions can introduce out-of-order sample errors. With multiple partitions, there is no guarantee that messages will be consumed in order (this is only guaranteed for a partition, not across multiple partitions). To guarantee ordering of samples, all samples for a metric name+label set should go to the same partition. This is not the behavior of the current code with kafka.PartitionAny.