How to analyze trillions data with ClickHouse 虎牙信息 HUYA 李本旺 sundy-li Github: http://github.com/sundy-li Mail: libenw1992@gmail.com APM 覆盖全公司上报数据 o 主播端带宽，卡顿，PCU o 客户端卡顿，API请求，信令通道 o TAF调用 o 基础监控 Open-Falcon o CDN,P2P,UDB … 400+ 业务指标 3000亿 + 日均增量 500w + 每秒新增 实时监控 秒级响应 覆盖海内外服务 3个集群，近百节点部署 Previous architecture Features: • Real-time pre-calculation • Opentsdb Seriously dependent on hbase • Opentsdb poor aggregation ability • Influxdb is not clustered • Not friendly with OLAP • Poor performance • Not flexible • … Why ClickHouse Column Oriented Super Fast OLAP SQL supported Linearly Scalable Simple ClickHouse as time-series database Get the average value group by line, interval 20s Current architecture Best practice start Realtime ingestion • Kafka Engine(×) Not Recommended o Error handling o Kafka partition && consumer auto-rebalance o ClickHouse is shared-nothing architecture o Bad control of data sharding Because Realtime
