Wow, I'm surprised to see a Kafka hater! My experience of it has been mostly positive, and it seems to solve a lot of problems for people, if they use it as it should be used and not as a source of truth database.
And thats exactly why i hate it -my old company used it as a source of truth db. It was such a mess and id get calls in the middle of the night because data was being moved or held correctly.
Look at mr fancy, with his interface that can search through topics - we're just flying by the top of our head >_>
At least we were smart enough to just generate a CloudEvent and jam it into a KafkaMessage before sending it, so that's pretty standard.
We just use it as a messaging bus - we're not sending "big data" through Kafka. We tried, failed, and immediately gave up, lmao. Now we just save data on HDFS and send where we stored said file (effectively a "claim-check pattern", if you're into design patterns)