Kibana – Setup log rotation


:information_source: In this document we will see how to properly setup a log rotation for kibana. The e.g. is based on CentOS7 but can be easily adapt for ubuntu or any other linux distribution.


  • Kibana doesn’t handle log rotation, but it is built to work with an external process that rotates logs, such as logrotate.
  • The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.
  • Logrotate can be installed with the logrotate package. It is installed by default and runs daily.
  • The primary configuration file for logrotate is /etc/logrotate.conf; additional configuration files are included from the /etc/logrotate.d directory.



  • Have the following elements (pid.file AND logging.dest) setup in the kibana.yaml configuration file
    server.port: 5601 "${KIBANA_SRV}"
    elasticsearch.url: "http://${ES_SRV}:9200"
    kibana.index: ".kibana"
    pid.file: /var/run/kibana/
    logging.dest: /var/log/kibana/kibana.log
  • :warning: verify if kibana is well authorised to create /var/log/kibana.log file.
    $ mkdir -p /var/log/kibana/ && chown -R kibana:kibana /var/log/kibana/
    $ mkdir -p /var/run/kibana/ && chown -R kibana:kibana /var/run/kibana/


verify that logrotate is properly installed

$ logrotate --version
    logrotate 3.8.6

verify that logrotate configuration include the logrotate.d directory

$ cat /etc/logrotate.conf
  . . .
  include /etc/logrotate.d


logrotate file

We will create a new logrotate configuration for kibana

$ cat << EOF > /etc/logrotate.d/elk-kibana
/var/log/kibana/*.log {
  size 10M
  create 0644 kibana kibana
  rotate 7
    /bin/kill -HUP $(cat /var/run/kibana/ 2>/dev/null) 2>/dev/null

Verify your file syntax with the following command

logrotate -vd /etc/logrotate.d/elk-kibana

If you didn’t get any error, you can manually start the first rotation with

logrotate -vf /etc/logrotate.d/elk-kibana

Crontab file

If the ligne include /etc/logrotate.d is well present in /etc/logrotate.conf and logrotate.conf present in /etc/cron.daily/logrotate you don’t need to do any more setup.

grep "logrotate.conf" /etc/cron.daily/logrotate
    /usr/sbin/logrotate -s /var/lib/logrotate/logrotate.status /etc/logrotate.conf


Happy logs rotation! 

Setup a secured redis-cluster on centos7

Setup a secured redis-cluster on centos7

What is redis exactly?

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

Redis Sentinel provides high availability for Redis. In practical terms this means that using Sentinel you can create a Redis deployment that resists without human intervention to certain kind of failures.

⚠️ You need at least three Sentinel instances for a robust deployment.

Sentinel constantly checks if your master and slave instances are working as expected.

If a master is not working as expected, Sentinel can start a failover process where a slave is promoted to master, the other additional slaves are reconfigured to use the new master, and the applications using the Redis server informed about the new address to use when connecting.

ℹ️ Sentinel manages the failover, it doesn’t configure Redis for HA. It is an important distinction.

Read more

Usefulness of an MQ layer in an ELK stack

Usefulness of an MQ layer in an ELK stack

First things first, disambiguation; let’s talk about these strange words.

What does ELK means?

An ELK stack is composed of Elasticsearch, Logstash, and Kibana; these 3 components are owned by the company and are particulary useful to handle Data. (ship, enrich, index and search your Data)

What does MQ means?

In computer science, The message queue paradigm is a sibling of the publisher/subscriber pattern.
You can imagine an MQ componant works like several mailboxes; meaning that publisher(s)/subscriber(s) do not interact with the message at the same time, many publishers can post a message for one subsriber and vice versa.

➡ Redis or Kafka or RabbitMQ can be used as buffer in the ELK stack.

Read more

Contact Us