MELIG-1 M: State of InfluxDB
Today was the first instance of a M.E.L.I.G. MeetUp. The topic was about metrics in general (broad general) and what the new version of InfluxDB is going to introduce; mainly in terms of the storage backend.
We were kindly hosted by InnoQ (they have a podcast). Thanks for that...
The MeetUp was a combination of a InfluxDB one and the first M.E.L.I.G. MeetUp. M.E.L.I.G. is about addressing all things related to infrastructure services around:
- Metrics: Think graphite, InfluxDB, Cyanite, OpenTSDB and a like. Including middleware such as graphite-api, carbon-relay-ng
- Events & Logs where events are business related logs you do not want to miss and generic log information that might be used for debugging purposes
- Inventory services to get to know how the infrastructure looks like and looked like in the past
- Glue like Docker, heka, fluentd, runC, storage backend and so on; basically stuff that does not fit in the other parts. :)
I would like to establish a MeetUp which addresses the ever increasing stack around infrastructure - one aspect (use-case) at a time.
To keep things reproducible I will try to provide docker stacks for all topics covered, as to allow the attendees (and others) to recreate the experience. :)
InfluxDB MeetUp today
Today I introduced the meeting by showing of some (somehow random) slides around the topic 'metrics'. I promise to improve my slides next time. Time was precious this time around. Paul Dix, the CEO of InfluxDB was kind enough to connect via Skype for insights into what the new version of InfluxDB will bring and to give an extensive Q&A.
Click here for the complete playlist...
Metrics in General
The metrics stack I showed today can be recreated as follows. Just install
docker-compose and kick of the stack (OK, you need to instal
$ git clone https://github.com/ChristianKniep/orchestra.git melig1 Cloning into 'melig1'... remote: Counting objects: 557, done. remote: Total 557 (delta 1), reused 1 (delta 1), pack-reused 555 Receiving objects: 100% (557/557), 1.53 MiB | 1.01 MiB/s, done. Resolving deltas: 100% (229/229), done. Checking connectivity... done. $ cd melig1/ $ git checkout melig1 # a tag for this MeetUp, in case master does not do the trick $ carbon-relay-ng/ $ compose pull *snip* $ compose up -d Creating carbonrelayng_consul_1... Creating carbonrelayng_carbon_1... Creating carbonrelayng_metrics_1... Creating carbonrelayng_gapi_1... Creating carbonrelayng_grafana_1... $
Open the consul WebUI by pointing our browser to the
DOCKER_HOST:8500 (e.g. http://192.168.99.100:8500).
It should look somehow like this (after a moment to settle the services):
If you then hit the port
8070 you can have a look at the
carbon backed metrics flowing in.
As a middleware relay
carbon-relay-ng is used. It sucks in the metrics and forwards them to whatever backend is discovered by consul. For now it's only carbon.
By adding a second backend, carbon-relay-ng got a new configuration (provided by
consul-template, but that's for another MeetUp) and is restarted.
$ cd add_influxdb/ $ compose up -d Pulling influxdb (qnib/influxdb:latest)... latest: Pulling from qnib/influxdb Status: Downloaded newer image for qnib/influxdb:latest Creating addinfluxdb_influxdb_1... $
Add Grafana 2.x
Since Grafana 1. didn't quite work in my setup I introduced a second version of it.
$ cd ../add_grafana2/ $ compose up -d Pulling grafana2 (qnib/grafana2:latest)... Status: Downloaded newer image for qnib/grafana2:latest Creating addgrafana2_grafana2_1... $
Grafana2 is accessible under
:3000 and provides a dashboard which relies on the
carbon and the