Installing, configuring Skedler Reports as Kibana Plugin with Elasticsearch and Kibana Environment using Docker Compose

Introduction

If you are using ELK stack, you can now install Skedler as a Kibana plugin. Skedler Reports plugin is available for Kibana versions from 6.5.x to 7.6.x.

Let’s take a look at the steps to Install Skedler Reports as a Kibana plugin.

Prerequisites:

  1. A Linux machine
  2. Docker Installed
  3. Docker Compose Installed

Let’s get started!

Login to your Linux machine and update the repository and install Docker and Docker Compose. Then follow the below steps to update the Repository:

Setting Up Skedler Reports

Create a Directory, say skedlerplugin

ubuntu@guidanz:~$ mkdir skedlerplugin

ubuntu@guidanz:~$ cd skedlerplugin/

ubuntu@guidanz:~$ vim docker-compose.yml

Now, create a Docker Compose file for Skedler Reports. You also need to create a Skedler Reports configuration file, reporting.yml, and a Docker Compose file for Skedler as below,

version: “2.4”

services:

#  Skedler Reports container

  reports:

    image: skedler/reports:latest

    container_name: reports

    privileged: true

    cap_add:

      – SYS_ADMIN

    volumes:

      – /sys/fs/cgroup:/sys/fs/cgroup:ro

      – reportdata:/var/lib/skedler

      – ./reporting.yml:/opt/skedler/config/reporting.yml

    command: /opt/skedler/bin/skedler

    depends_on:

      elasticsearch: { condition: service_healthy }

    ports:

      – 3000:3000

    healthcheck:

      test: [“CMD”, “curl”, “-s”, “-f”, “http://localhost:3000”]

    networks: [‘stack’]

volumes:

  reportdata:

    driver: local

networks: {stack: {}}

Create an Elasticsearch configuration file – reporting.yml and paste the config as below.

ubuntu@guidanz:~$ mkdir skedlerplugin

ubuntu@guidanz:~$ cd skedlerplugin/

ubuntu@guidanz:~$ vim reporting.yml

Download the reporting.yml file found here

Setting Up Elasticsearch

You also need to create an Elasticsearch configuration file, elasticsearch.yml. Docker Compose file for Elasticsearch is below,

#Elasticsearch container

  elasticsearch:

    container_name: elasticsearch

    hostname: elasticsearch

    image: “docker.elastic.co/elasticsearch/elasticsearch:7.6.0”

    logging:

      options:

        max-file: “3”

        max-size: “50m”

    environment:

      – http.host=0.0.0.0

      – transport.host=127.0.0.1

      – bootstrap.memory_lock=true

      – “ES_JAVA_OPTS=-Xms${ES_JVM_HEAP} -Xmx${ES_JVM_HEAP}”

    mem_limit: 1g

    ulimits:

      memlock:

        soft: -1

        hard: -1

    volumes:

      – ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml

      – esdata:/usr/share/elasticsearch/data

    ports: [‘9200:9200’]

    healthcheck:

      test: [“CMD”, “curl”,”-s” ,”-f”, “http://localhost:9200/_cat/health”]

    networks: [‘stack’]

volumes:

  esdata:

    driver: local

networks: guidanz

Create an Elasticsearch configuration file elasticsearch.yml and paste the config as below.

cluster.name: guidanz-stack-cluster

node.name: node-1

network.host: 0.0.0.0

path.data: /usr/share/elasticsearch/data

http.port: 9200

xpack.monitoring.enabled: true

http.cors.enabled: true

http.cors.allow-origin: “*”

http.max_header_size: 16kb

Setting Up Skedler Reports as Kibana Plugin

Create a Directory inside skedlerplugin, say kibanaconfig

ubuntu@guidanz:~$ mkdir kibanaconfig

ubuntu@guidanz:~$ cd kibanaconfig/

ubuntu@guidanz:~$ vim Dockerfile

Now, create a Docker file for Kibana and check the Docker file for Kibana as below,

FROM docker.elastic.co/kibana/kibana:7.6.0

RUN ./bin/kibana-plugin install https://www.skedler.com/plugins/skedler-reports-plugin/4.10.0/skedler-reports-kibana-plugin-7.6.0-4.10.0.zip

Then, copy the URL of the Skedler Reports plugin matching your exact Kibana version from here.

You also need to create a Docker Compose file for Kibana is below,

#Kibana container

  kibana:

    container_name: kibana

    hostname: kibana

    build:

      context: ./kibanaconfig

      dockerfile: Dockerfile

    image: kibanaconfig

    logging:

      options:

        max-file: “3”

        max-size: “50m”

    volumes:

      – ./kibanaconfig/kibana.yml:/usr/share/kibana/config/kibana.yml

      – ./kibanaconfig/skedler_reports.yml:/usr/share/kibana/plugins/skedler/config/skedler_reports.yml

    ports: [‘5601:5601’]

    networks: [‘stack’]

    depends_on:

      elasticsearch: { condition: service_healthy }

    restart: on-failure

    healthcheck:

      test: [“CMD”, “curl”, “-s”, “-f”, “http://localhost:5601/”]

      retries: 6

Create a Kibana configuration file kibana.yml inside the kibanaconfig folder and paste the config as below.

ubuntu@guidanz:~$ cd kibanaconfig/

ubuntu@guidanz:~$ vim kibana.yml

server.port: 127.0.0.1:5601

elasticsearch.url: “http://elasticsearch:9200”

server.name: “full-stack-example”

xpack.monitoring.enabled: true

Create a Skedler Reports as Kibana Plugin configuration file skedler_reports.yml inside the kibanaconfig folder and paste the config as below.

ubuntu@guidanz:~$ cd kibanaconfig/

ubuntu@guidanz:~$ vim skedler_reports.yml

#/*********** Skedler Access URL *************************/

skedler_reports_url: “http://ip_address:3000”

#/*********************** Basic Authentication *********************/

# If Skedler Reports uses any username and password

#skedler_username: user

#skedler_password: password

Configure the Skedler Reports server URL in the skedler_reports_url variable. By default, the variable is set as shown below,

If the Skedler Reports server URL requires basic authentication, for example, Nginx, uncomment and configure the skedler_username and skedler_password with the basic authentication credentials as shown below: Now run the docker-compose.

ubuntu@guidanz:~/skedlerplugin$ docker-compose up -d

Access Skedler Reports the IP and Port and you will see the Skedler Reports UI.

| http://ip_address:3000

Access Elasticsearch the IP and Port and you will see the Elasticsearch UI.

| http://ip_address:9200

Access Kibana using the IP and Port and you will see the Kibana UI.

| http://ip_address:5601

So now the Composite docker-compose file will look like below,

You can Simply do compose up and down.

version: “2.4”

services:

#  Skedler Reports container

  reports:

    image: skedler/reports:latest

    container_name: reports

    privileged: true

    cap_add:

      – SYS_ADMIN

    volumes:

      – /sys/fs/cgroup:/sys/fs/cgroup:ro

      – reportdata:/var/lib/skedler

      – ./reporting.yml:/opt/skedler/config/reporting.yml

    command: /opt/skedler/bin/skedler

    depends_on:

      elasticsearch: { condition: service_healthy }

    ports:

      – 3000:3000

    healthcheck:

      test: [“CMD”, “curl”, “-s”, “-f”, “http://localhost:3000”]

    networks: [‘stack’]

#  Elasticsearch container

  elasticsearch:

    container_name: elasticsearch

    hostname: elasticsearch

    image: “docker.elastic.co/elasticsearch/elasticsearch:7.1.1”

    logging:

      options:

        max-file: “3”

        max-size: “50m”

    environment:

      – http.host=0.0.0.0

      – transport.host=127.0.0.1

      – bootstrap.memory_lock=true

      – “ES_JAVA_OPTS=-Xms${ES_JVM_HEAP} -Xmx${ES_JVM_HEAP}”

    mem_limit: ${ES_MEM_LIMIT}

    ulimits:

      memlock:

        soft: -1

        hard: -1

    volumes:

      – ./config/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml

      – esdata:/usr/share/elasticsearch/data

    ports: [‘9200:9200’]

    healthcheck:

      test: [“CMD”, “curl”,”-s” ,”-f”, “http://localhost:9200/_cat/health”]

    networks: [‘stack’]

 #Kibana container

  kibana:

    container_name: kibana

    hostname: kibana

    build:

      context: ./kibanaconfig

      dockerfile: Dockerfile

    image: kibanaconfig

    logging:

      options:

        max-file: “3”

        max-size: “50m”

    volumes:

      – ./kibanaconfig/kibana.yml:/usr/share/kibana/config/kibana.yml

      – ./kibanaconfig/skedler_reports.yml:/usr/share/kibana/plugins/skedler/config/skedler_reports.yml

    ports: [‘5601:5601’]

    networks: [‘stack’]

    depends_on:

      elasticsearch: { condition: service_healthy }

    restart: on-failure

    healthcheck:

      test: [“CMD”, “curl”, “-s”, “-f”, “http://localhost:5601/”]

      retries: 6

volumes:

  esdata:

    driver: local

  reportdata:

    driver: local

networks: {stack: {}}

You can Simply do compose up and down.

ubuntu@guidanz:~/skedlerplugin$ docker-compose down 

ubuntu@guidanz:~/skedlerplugin$ docker-compose up -d

Summary

Docker compose is a useful tool to manage container stacks for your client. And manage all related containers with one single command.

Skedler Update: Version 3.9 Released

Skedler Update: Version 3.9 Released

Here’s everything you need to know about the new Skedler v3.9. Download the update now to take advantage of its new features for both Skedler Reports and Alerts.

What’s New With Skedler Reports v3.9

  • Support for:
    • ReadOnlyRest Elasticsearch/Kibana Security Plugin.
    • Chromium web browser for Skedler report generation.
    • Report bursting in Grafana reports if the Grafana dashboard is set with Template Variables.
    • Elasticsearch version 6.4.0 and Kibana version 6.4.0.
  • Ability to install Skedler Reports through Debian and RPM packages.
  • Simplified installation levels of Skedler Reports here.
  • Upgraded license module
    • NOTE: License reactivation is required when you upgrade Skedler Reports from the older version to the latest v3.8. Refer to this URL to reactivate the Skedler Reports license key.
    • Deactivation of Skedler license key in UI

What’s New With Skedler Alerts v3.9

  • Support for:
    • Installing Skedler Alerts via Debian and RPM packages.
    • GET method type in Webhook.
    • Elasticsearch 6.4.0.
  • Simplified installation levels of Skedler. Refer to this URL for installation guides.
  • Upgraded license module:
    • NOTE: License reactivation is required when you upgrade Skedler Alerts from the older version to the latest v3.8. Refer to this URL to reactivate the Skedler Alerts license key.
  • Deactivation of Skedler Alerts license key in UI

 

Get Skedler Reports

Download Skedler Reports

Get Skedler Alerts

Download Skedler Alerts

 

How to Use Skedler with Amazon’s Elastic Search Tools

As the majority of IT-savvy businesses synchronize with the cloud, capital infrastructure charges are becoming rapidly replaced with more lucrative options which optimize and scale company costs. Rather than IT planning for weeks on end, users are able to instantly discover thousands of servers within minutes for more rapid, effectual results. This starts with Amazon Elasticsearch Service: the cloud’s top trending tool for Elasticsearch-Logstash-Kibana applications.

Skedler

With Amazon Elasticsearch Service, you’re able to effortlessly scale log analytics, text searches and monitoring within a few minutes. It also allows you to exercise Elasticsearch’s APIs and real-time functionalities along with its scalability and security models by production workloads, including Kibana, Logstash, and AWS; enabling you to start to action your data insights effectively and quickly. Simply run the AWS Elasticsearch service, or run your own cluster using the AWS EC2.

Server or Cluster?

If you choose to run the server on AWS, there are a few clear advantages. Primarily, you won’t have to manually replace failed nodes, since you can add it to the cluster as you go. You can add and remove nodes through an API and manage access rights via IAM, which is far easier than setting up a reverse proxy, as well as receive daily insights into daily snapshots to S3, including CloudWatch monitoring for your Elasticsearch cluster.

On the other hand, if you choose to download and run, you’ll have more instance types and sizes available. In this case, you’d be able to use bigger i2 instances than AWS alone, allowing you to scale further and get more insights into logs and metrics. You’re also able to alter the index settings with more detail than just analysis and replicas, such as with delayed allocation, which often carries a lot of data per node. Additionally, you can modify more cluster-wide settings than in AWS alone, as well as gain access to all other APIs, which is particularly useful when debugging. And whilst Cloudwatch collects a reasonable amount of metrics, with EC2 you can utilize a more comprehensive Elasticsearch monitoring solution, with clusters of more than 20 nodes at a time.

Amazon Elasticsearch Service within Skedler

Regardless of which route you take, the ultimate challenge is learning how to add reporting for your ELK application within the Amazon Elasticsearch Service. In this instance, Skedler accounts for proven reporting solutions specifically with ELK applications, with many of our customers running ELK and Skedler together on AWS to account for both log and IOT analytics, as well as several other features. Skedler provides helpful deployment options to add reporting to your existing AWS ELK application, including Skedler as a service within EC2, or Skedler AWS Containers service; both of which support Amazon Simple Email Service (SES) for emailing reports reliably and economically.
Ready to try Amazon Elasticsearch Service with and through Skedler? Try it free.  After the free trial period, you can purchase Skedler license and instantly convert your evaluation environment into a production environment. 

Email XLS Reports from ELK to data analysts

Introducing XLS Email Reports in Skedler 1.1

Skedler PDF reports  are an easy way to send actionable visual  information from your Elasticsearch-Kibana application to non-technical users.   But how do you meet the requirement from your customers, data analysts and users  who are looking for raw data from your Kibana application?   What if they would like to receive data periodically for analyzing with Microsoft Excel or Tableau?

We are proud to announce the beta release of Skedler 1.1 that allows you to schedule XLS reports in addition to PDF, PNG and in-line PNG reports.

Skedler - Select report format in schedule a report window

Schedule XLS reports in Skedler

Scheduled XLS reports makes it simple to schedule, generate, and send hourly/daily/weekly/monthly data snapshots to data analysts.   Skedler will automatically generate an XLS worksheet for each chart in your Kibana dashboard, combine all the worksheets into a single XLS file, and email the report to target recipients as per the schedule.

Consider the following Kibana dashboard with several charts providing information regarding Crime statistics in Alameda County for the past 90 days.

Skedler - Kibana® dashboard - Alameda County Past 90 days crime statistics

Alameda county – Past 90 days crime statistics

Skedler will automatically convert this dashboard into a single XLS report with multiple worksheets, one worksheet for each chart.

Skedler - Kibana® excel (XLS) report

Daily XLS Report screenshot

You can notice that the XLS report contains six(6) worksheets, each corresponding to the chart in the Kibana dashboard.   Skedler automatically uses the chart title for the worksheet name so that it is easy to identify the worksheets and the data.  You can download the sample email report from Daily XLS Report of All Crimes Dashboard.

In addition to emailing XLS report to customers, data analysts and others who prefer to use their own tools,  you can also generate and store the XLS reports in server side file folders which can then be used for various analysis or as a data feed for other applications.

Test drive Skedler v1.1 Beta with XLS reports

If you are interested in trying out the Beta release of Skedler v1.1,  contact us and we will help you get started with Skedler.  Skedler is easy to set up and get started.  We will provide you with the technical support required to get past common configuration issues.

Copyright © 2023 Guidanz Inc
Translate »