kafka 模拟idle现象

刘超 2月前 ⋅ 321 阅读   编辑

  通过自定义Partitioner来模拟idle现象,有如下数据

{"id":"S001","cnt":1,"ts":"2020-06-13T09:58:00Z"}
{"id":"S001","cnt":1,"ts":"2020-06-13T09:58:01Z"}
{"id":"S001","cnt":2,"ts":"2020-06-13T09:58:02Z"}
{"id":"S001","cnt":3,"ts":"2020-06-13T09:58:03Z"}
{"id":"S001","cnt":4,"ts":"2020-06-13T09:58:04Z"}
{"id":"S001","cnt":5,"ts":"2020-06-13T09:58:05Z"}
{"id":"S001","cnt":6,"ts":"2020-06-13T09:58:06Z"}
{"id":"S001","cnt":7,"ts":"2020-06-13T09:58:07Z"}
{"id":"S001","cnt":8,"ts":"2020-06-13T09:58:08Z"}
{"id":"S001","cnt":9,"ts":"2020-06-13T09:58:09Z"}
{"id":"S001","cnt":10,"ts":"2020-06-13T09:58:10Z"}
{"id":"S001","cnt":11,"ts":"2020-06-13T09:58:11Z"}
{"id":"S001","cnt":12,"ts":"2020-06-13T09:58:12Z"}
{"id":"S001","cnt":13,"ts":"2020-06-13T09:58:13Z"}
{"id":"S001","cnt":14,"ts":"2020-06-13T09:58:14Z"}
{"id":"S001","cnt":15,"ts":"2020-06-13T09:58:15Z"}
{"id":"S001","cnt":16,"ts":"2020-06-13T09:58:16Z"}
{"id":"S001","cnt":17,"ts":"2020-06-13T09:58:17Z"}
{"id":"S001","cnt":18,"ts":"2020-06-13T09:58:18Z"}
{"id":"S001","cnt":19,"ts":"2020-06-13T09:58:19Z"}
{"id":"S001","cnt":20,"ts":"2020-06-13T09:58:20Z"} // 这条数据在第一个分区,其他数据在第二个分区
{"id":"S001","cnt":21,"ts":"2020-06-13T09:58:21Z"}
{"id":"S001","cnt":22,"ts":"2020-06-13T09:58:22Z"}
{"id":"S001","cnt":23,"ts":"2020-06-13T09:58:23Z"}
{"id":"S001","cnt":24,"ts":"2020-06-13T09:58:24Z"}
{"id":"S001","cnt":25,"ts":"2020-06-13T09:58:25Z"}
{"id":"S001","cnt":26,"ts":"2020-06-13T09:58:26Z"}
{"id":"S001","cnt":27,"ts":"2020-06-13T09:58:27Z"}
{"id":"S001","cnt":28,"ts":"2020-06-13T09:58:28Z"}
{"id":"S001","cnt":29,"ts":"2020-06-13T09:58:29Z"}
{"id":"S001","cnt":30,"ts":"2020-06-13T09:58:30Z"}
{"id":"S001","cnt":31,"ts":"2020-06-13T09:58:31Z"}
{"id":"S001","cnt":32,"ts":"2020-06-13T09:58:32Z"}
{"id":"S001","cnt":33,"ts":"2020-06-13T09:58:33Z"}
{"id":"S001","cnt":34,"ts":"2020-06-13T09:58:34Z"}
{"id":"S001","cnt":35,"ts":"2020-06-13T09:58:35Z"}
{"id":"S001","cnt":36,"ts":"2020-06-13T09:58:36Z"}
{"id":"S001","cnt":37,"ts":"2020-06-13T09:58:37Z"}
{"id":"S001","cnt":38,"ts":"2020-06-13T09:58:38Z"}
{"id":"S001","cnt":39,"ts":"2020-06-13T09:58:39Z"}

  我们让第21条数据到第一个分区,其他的在第二个分区,实现代码这里

  代码执行后,我们看看效果,如下

# 查看第一分区的数据
zhao@zhao:~/liujichao$ /web/kafka/bin/kafka-console-consumer.sh --bootstrap-server server-1:9092 --topic checkIdle --partition 0
{"id":"S001","cnt":20,"ts":"2020-06-13T09:58:20Z"}

# 查看第二分区的数据
zhao@zhao:~/liujichao$ /web/kafka/bin/kafka-console-consumer.sh --bootstrap-server server-1:9092 --topic checkIdle --partition 1
{"id":"S001","cnt":1,"ts":"2020-06-13T09:58:00Z"}
{"id":"S001","cnt":1,"ts":"2020-06-13T09:58:01Z"}
{"id":"S001","cnt":2,"ts":"2020-06-13T09:58:02Z"}
{"id":"S001","cnt":3,"ts":"2020-06-13T09:58:03Z"}
{"id":"S001","cnt":4,"ts":"2020-06-13T09:58:04Z"}
{"id":"S001","cnt":5,"ts":"2020-06-13T09:58:05Z"}
{"id":"S001","cnt":6,"ts":"2020-06-13T09:58:06Z"}
{"id":"S001","cnt":7,"ts":"2020-06-13T09:58:07Z"}
{"id":"S001","cnt":8,"ts":"2020-06-13T09:58:08Z"}
{"id":"S001","cnt":9,"ts":"2020-06-13T09:58:09Z"}
{"id":"S001","cnt":10,"ts":"2020-06-13T09:58:10Z"}
{"id":"S001","cnt":11,"ts":"2020-06-13T09:58:11Z"}
{"id":"S001","cnt":12,"ts":"2020-06-13T09:58:12Z"}
{"id":"S001","cnt":13,"ts":"2020-06-13T09:58:13Z"}
{"id":"S001","cnt":14,"ts":"2020-06-13T09:58:14Z"}
{"id":"S001","cnt":15,"ts":"2020-06-13T09:58:15Z"}
{"id":"S001","cnt":16,"ts":"2020-06-13T09:58:16Z"}
{"id":"S001","cnt":17,"ts":"2020-06-13T09:58:17Z"}
{"id":"S001","cnt":18,"ts":"2020-06-13T09:58:18Z"}
{"id":"S001","cnt":19,"ts":"2020-06-13T09:58:19Z"}
{"id":"S001","cnt":21,"ts":"2020-06-13T09:58:21Z"}
{"id":"S001","cnt":22,"ts":"2020-06-13T09:58:22Z"}
{"id":"S001","cnt":23,"ts":"2020-06-13T09:58:23Z"}
{"id":"S001","cnt":24,"ts":"2020-06-13T09:58:24Z"}
{"id":"S001","cnt":25,"ts":"2020-06-13T09:58:25Z"}
{"id":"S001","cnt":26,"ts":"2020-06-13T09:58:26Z"}
{"id":"S001","cnt":27,"ts":"2020-06-13T09:58:27Z"}
{"id":"S001","cnt":28,"ts":"2020-06-13T09:58:28Z"}
{"id":"S001","cnt":29,"ts":"2020-06-13T09:58:29Z"}
{"id":"S001","cnt":30,"ts":"2020-06-13T09:58:30Z"}
{"id":"S001","cnt":31,"ts":"2020-06-13T09:58:31Z"}
{"id":"S001","cnt":32,"ts":"2020-06-13T09:58:32Z"}
{"id":"S001","cnt":33,"ts":"2020-06-13T09:58:33Z"}
{"id":"S001","cnt":34,"ts":"2020-06-13T09:58:34Z"}
{"id":"S001","cnt":35,"ts":"2020-06-13T09:58:35Z"}
{"id":"S001","cnt":36,"ts":"2020-06-13T09:58:36Z"}
{"id":"S001","cnt":37,"ts":"2020-06-13T09:58:37Z"}
{"id":"S001","cnt":38,"ts":"2020-06-13T09:58:38Z"}
{"id":"S001","cnt":39,"ts":"2020-06-13T09:58:39Z"}


注意:本文归作者所有,未经作者允许,不得转载

全部评论: 0

    我有话说: