prometheus-exporter

Version, currently 0.1.01 version
  • 0.1.0latestFeb 14, 2022

github.com/marmaxev/prometheus-exporter

Crystal prometheus exporter

7 stars
0 dependents
License: MIT

Installation

# Add this to your shard.yml
dependencies:
  prometheus-exporter:
    github: marmaxev/prometheus-exporter
    version: ~> 0.1.0

Then run:

shards install

shard.yml

Crystal
1.1.0
License
MIT
Authors
marmaxev, SShekotihin

Dependencies

Development Dependencies

  • kemal~> 1.0.0github: kemalcr/kemaldev
  • lucky<= 0.29.0github: luckyframework/luckydev
  • sidekiq*github: mperham/sidekiq.cr, branch: masterdev

README

Crystal CI

prometheus-exporter

Prometheus metrics for your applications.

This is a client library based on ruby prometheus exporter, adapted for applications written in the Crystal programming language.

Allows you to collect and send Prometheus metrics to the server for further use. Supports several sets of basic metrics for some popular Crystal shards (Lucky, Kemal and Sidekiq) and custom metrics.

There is no server application in the project yet, so please use the server from the original project.

Installation

  1. Add the dependency to your shard.yml:

    dependencies:
      prometheus-exporter:
        github: marmaxev/prometheus-exporter
  2. Run shards install

  3. Require prometheus-exporter

    require "prometheus-exporter"

Usage

Basic example

Define the default client, register metrics and assign values to them.

The default client has localhost:9394 as the default server address.

# use a default client
client = PrometheusExporter::Client.default

# register a new count metric
counter = client.register(
  :counter,
  :my_counter_name,
  "My counter description"
)

# register a new gauge metric
gauge = client.register(
  :gauge,
  :my_gauge_name,
  "My gauge description"
)

# increment counter
counter.observe(1.0)

# observe gauge metric
gauge.observe(
  value: 123.0,
  keys: { :foo => bar }
)

You can also observe metrics through the Client. For example:

client.observe(
  name: :my_gauge_name,
  value: 123.0,
  keys: { :foo => :bar }
)

This will allow you to get these metrics on server:

# HELP my_gauge_name My gauge description
# TYPE my_gauge_name gauge
my_gauge_name{foo="bar"} 123.0

# HELP my_counter_name My counter description
# TYPE my_counter_name counter
my_counter_name 1.0

Lucky metrics

If you use Lucky you can add a special handler to your server's middleware. Learn more about HTTP handlers in Lucky here.

# src/app_server.cr
# ...
class App < Lucky::BaseAppServer
  def middleware
    [
      Lucky::HttpMethodOverrideHandler.new,
      Lucky::LogHandler.new,

      PrometheusExporter::Middleware::LuckyHttpRequestCollector.new,  # add this line

      Lucky::ErrorHandler.new(action: Errors::Show),
      Lucky::RouteHandler.new,
      Lucky::StaticFileHandler.new("./public", false),
      Lucky::RouteNotFoundHandler.new,
    ]
  end
end

This handler will process incoming requests and save data about their number and duration.

PrometheusExporter::Middleware::LuckyHttpRequestCollector

All metrics have a status, method and path labels.

TypeNameDescription
Counterhttp_request_countCount of http requests
Gaugehttp_request_duration_secondsDuration of http requests

Example of metrics for each request:

http_request_count{status="200",method="GET",path="/foo"} 1.0

http_request_duration_seconds{status="200",method="GET",path="/foo"} 0.05

Kemal metrics

If you use Kemal you can add a KemalHttpRequestCollector to server's middleware. Learn more about HTTP handlers in Kemal here.

add_handler KemalHttpRequestCollector.new

This handler will process incoming requests and save data about their number and duration.

PrometheusExporter::Middleware::KemalHttpRequestCollector

All metrics have a status, method and path labels.

TypeNameDescription
Counterhttp_request_countCount of http requests
Gaugehttp_request_duration_secondsDuration of http requests

Example of metrics for each request:

http_request_count{status="200",method="GET",path="/bar"} 1.0

http_request_duration_seconds{status="200",method="GET",path="/bar"} 0.1

Sidekiq metrics

There are two sets of metrics being collected for Sidekiq.

  1. Sidekiq
  2. SidekiqStats

When you configure your sidekiq server add PrometheusExporter::Instrumentation::Sidekiq to server's middleware and start PrometheusExporter::Instrumentation::SidekiqStats.

# sidekiq.cr
# ...

cli = Sidekiq::CLI.new
server = cli.configure do |config|
  config.server_middleware.add ::PrometheusExporter::Instrumentation::Sidekiq.new # add this
end

PrometheusExporter::Instrumentation::SidekiqStats.start(frequency: 10) # and this

cli.run(server)

PrometheusExporter::Instrumentation::Sidekiq

All metrics have a job_name label and a queue label.

TypeNameDescription
Summarysidekiq_job_duration_secondsTime spent in sidekiq jobs
Countersidekiq_jobs_totalTotal number of sidekiq jobs executed
Countersidekiq_restarted_jobs_totalTotal number of sidekiq jobs that we restarted because of a sidekiq shutdown
Countersidekiq_failed_jobs_totalTotal number of failed sidekiq jobs

PrometheusExporter::Instrumentation::SidekiqStats

TypeNameDescription
Gaugesidekiq_stats_dead_sizeSize of the dead queue
Gaugesidekiq_stats_enqueuedNumber of enqueued jobs
Gaugesidekiq_stats_failedNumber of failed jobs
Gaugesidekiq_stats_processedTotal number of processed jobs
Gaugesidekiq_stats_processes_sizeNumber of processes
Gaugesidekiq_stats_retry_sizeSize of the retries queue
Gaugesidekiq_stats_scheduled_sizeSize of the scheduled queue
Gaugesidekiq_stats_workers_sizeNumber of jobs actively being processed

Process metrics

You can also get metrics for each process. To do this, start the PrometheusExporter::Instrumentation::Process.

PrometheusExporter::Instrumentation::Process.start(
  type: "my_process",
  frequency: 10
)

PrometheusExporter::Instrumentation::Process

All metrics have a type and pid labels.

TypeNameDescription
Gaugerss_bytesRSS in bytes
Gaugecpu_secondsTotal CPU seconds
Gaugebytes_since_gcBytes since GC
Gaugegc_unmapped_bytesGC unmapped bytes
Gaugegc_total_bytesGC total bytes
Gaugegc_free_bytesGC free bytes
Gaugegc_heap_bytesGC heap bytes

Configure client

By default all of Instrumentations use PrometheusExporter::Client.default.

You can configure the server address and port, assign custom labels for all metrics, and specify whether the client is available for use.

PrometheusExporter::Client.default = PrometheusExporter::Client.new(
  host: "localhost",
  port: 8000,
  custom_labels: {
    :foo => "bar"
  },
  enabled: true
)

Contributing

  1. Fork it (https://github.com/marmaxev/prometheus-exporter/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors