Skip to main content

StatsD Emitter

To use this Apache Druid extension, include statsd-emitter in the extensions load list.

Introduction

This extension emits druid metrics to a StatsD server. (https://github.com/etsy/statsd) (https://github.com/armon/statsite)

Configuration

All the configuration parameters for the StatsD emitter are under druid.emitter.statsd.

propertydescriptionrequired?default
druid.emitter.statsd.hostnameThe hostname of the StatsD server.yesnone
druid.emitter.statsd.portThe port of the StatsD server.yesnone
druid.emitter.statsd.prefixOptional metric name prefix.no""
druid.emitter.statsd.separatorMetric name separatorno.
druid.emitter.statsd.includeHostFlag to include the hostname as part of the metric name.nofalse
druid.emitter.statsd.dimensionMapPathJSON file defining the StatsD type, and desired dimensions for every Druid metricnoDefault mapping provided. See below.
druid.emitter.statsd.blankHolderThe blank character replacement as StatsD does not support path with blank characterno"-"
druid.emitter.statsd.queueSizeMaximum number of unprocessed messages in the message queue.noDefault value of StatsD Client(4096)
druid.emitter.statsd.poolSizeNetwork packet buffer pool size.noDefault value of StatsD Client(512)
druid.emitter.statsd.processorWorkersThe number of processor worker threads assembling buffers for submission.noDefault value of StatsD Client(1)
druid.emitter.statsd.senderWorkersThe number of sender worker threads submitting buffers to the socket.noDefault value of StatsD Client(1)
druid.emitter.statsd.dogstatsdFlag to enable DogStatsD support. Causes dimensions to be included as tags, not as a part of the metric name. convertRange fields will be ignored.nofalse
druid.emitter.statsd.dogstatsdConstantTagsIf druid.emitter.statsd.dogstatsd is true, the tags in the JSON list of strings will be sent with every event.no[]
druid.emitter.statsd.dogstatsdServiceAsTagIf druid.emitter.statsd.dogstatsd and druid.emitter.statsd.dogstatsdServiceAsTag are true, druid service (e.g. druid/broker, druid/coordinator, etc) is reported as a tag (e.g. druid_service:druid/broker) instead of being included in metric name (e.g. druid.broker.query.time) and druid is used as metric prefix (e.g. druid.query.time).nofalse
druid.emitter.statsd.dogstatsdEventsIf druid.emitter.statsd.dogstatsd and druid.emitter.statsd.dogstatsdEvents are true, Alert events are reported to DogStatsD.nofalse

Druid to StatsD Event Converter

Each metric sent to StatsD must specify a type, one of [timer, counter, guage]. StatsD Emitter expects this mapping to be provided as a JSON file. Additionally, this mapping specifies which dimensions should be included for each metric. StatsD expects that metric values be integers. Druid emits some metrics with values between the range 0 and 1. To accommodate these metrics they are converted into the range 0 to 100. This conversion can be enabled by setting the optional "convertRange" field true in the JSON mapping file. If the user does not specify their own JSON file, a default mapping is used. All metrics are expected to be mapped. Metrics which are not mapped will log an error. StatsD metric path is organized using the following schema: <druid metric name> : { "dimensions" : <dimension list>, "type" : <StatsD type>, "convertRange" : true/false} e.g. query/time" : { "dimensions" : ["dataSource", "type"], "type" : "timer"}

For metrics which are emitted from multiple services with different dimensions, the metric name is prefixed with the service name. e.g. "druid/coordinator-segment/count" : { "dimensions" : ["dataSource"], "type" : "gauge" }, "druid/historical-segment/count" : { "dimensions" : ["dataSource", "tier", "priority"], "type" : "gauge" }

For most use-cases, the default mapping is sufficient.