Skedler Review: The Report Scheduler Solution for Kibana

 Matteo Zuccon is a software developer with a passion for web development (RESTFull services, JS Frameworks), Elasticsearch, Spark, MongoDB, and agile processes. He runs whiletrue.run. Follow him on Twitter @matteo_zuccon. 

With Kibana you can create intuitive charts and dashboards. Since Aug 2016 you can export your dashboards in a PDF format thanks to Reporting. With Elastic version, 5 Reporting has been integrated into X-Pack for the Premium and Enterprise subscriptions.

Recently I tried Skedler, an easy to use report scheduling and distribution application for Kibana that allows you to centrally schedule and distribute Kibana Dashboards and Saved Searches as hourly/daily/weekly/monthly PDF, XLS or PNG reports to various stakeholders.

Skedler is a standalone app that allows you to utilize a new dashboard where you can manage Kibana reporting tasks (schedule, dashboards and saved search). Right now there are four different price plans (from free to premium edition).

In this post I am going to show you how to install Skedler (on Ubuntu) and how export/schedule a Kibana dashboard.

Install Pre-requisites

sudo apt-get -y update

sudo apt-get install -y libfontconfig1 libxcomposite1 libxdamage1 libcups2 libasound2 libxrandr2 libxfixes3 libnss3 libnss3-dev libxkbcommon-dev libgbm-dev libxshmfence-dev libatk1.0-0 libatk-bridge2.0-0 libgtk-3-0 gcc make

Install .deb package

Download the latest skedler-xg.deb file and extract it.  If you have previously installed the .deb package, remove it before installing the latest version.

curl -O https://skedler-v5-releases.s3.amazonaws.com/downloads/latest/skedler-xg.deb

sudo dpkg -i skedler-xg.deb

Install .tar.gz package

Download the latest skedler-xg.tar.gz file and extract it.

curl -O https://skedler-v5-releases.s3.amazonaws.com/downloads/latest/skedler-xg.tar.gz

sudo tar xzf skedler-xg.tar.gz

cd skedler-xg

sudo chmod -R 777 *

Configure your options for Skedler v5

Skedler Reports has a number of configuring options that can be defined in its reporting.yml file (located in the skedler folder).  In the reporting.yml file, you can configure options to run Skedler in an air-gapped environment, change the port number, define the hostname, change the location for the Skedler database, and log files.

Read more about the reporting.yml configuration options.

Start Skedler for .deb

To start Skedler, the command is:

sudo service skedler start

To check status, the command is:

sudo service skedler status

To stop Skedler. the command is:

sudo service skedler stop

Start Skedler for .tar.gz

To run Skedler manually, the command is:

sudo bin/skedler

To run Skedler as a service, the commands are:

sudo ./install_as_service.sh

To start Skedler, the command is:

sudo service skedler start

To check status, the command is:

sudo service skedler status

To stop Skedler. the command is:

sudo service skedler stop

Access Skedler Reports

The default URL for accessing Skedler Reports v5 is:

http://localhost:3005/

If you had made configuration changes in the reporting.yml, then the Skedler URL is of the following format:

http://<hostname or your domainurl>:3005

or

http://<hostname or your domain url>:<port number>

Login to Skedler Reports

By default, you will see the Create an account UI.  Enter your email to create an administrator account in Skedler Reports. Click on Continue.

Note: If you have configured an email address and password in reporting.yml, then you can skip the create account step and proceed to Login.

An account will be created and you will be redirected to the Login page.

Sign in using the following credentials:

Username: <your email address>   (or the email address you configured in reporting.yml)Password: admin   (or the password you configured in reporting.yml)

Click Sign in.

You will see the Reports Dashboard after logging in to the skedler account.   

In this post, I demonstrated how to install and configure Skedler and how to create a simple schedule for our Kibana dashboard. My overall impression of Skedler is that it is a powerful application to use side-by-side with Kibana that allows you to deliver reports directly to your stakeholders.

These are the main benefits that Skedler offers:

• It’s easy to install
• Linux, Windows  and Mac OS support (it runs on Node.js server)
• Reports are generated locally (your data isn’t sent to the cloud or Skedler servers)
• Competitive price plans
• Supports Kibana and Grafana.
• Automatically discovers your existing Kibana Dashboards and Saved Searches (so you can easily use Skedler in any environment with no new stack installation needed)
• It lets you centrally schedule and manage who gets which reports and when they get them
• Allows for hourly, weekly, monthly, and yearly schedules
• Generates XLS and PNG reports besides PDF as opposed to Elastic Reporting that only supports PDF.
• I strongly recommend that you try Skedler because it can help you to automatically deliver reports to your stakeholders and it integrates within your ELK environment without any modification to your stack.

Click here for free trial option.

You can find more resources about Skedler here:

• Skedler – How it works
• Skedler Licensing
• Skedler – Blog

Kibana Reporting in Action: A Kane LPI Case Study

Every company carries valuable data, whether it’s relevant to a specific client’s private information, statistics, or finances. Alongside the fear that you might lose precious data or incur a security breach with faulty programs, data needs to be filed and exported accurately in order to meet certain deadlines and practical standards; which is why Kibana reporting has proven to be an imperative tool for loss prevention within any given company.

The Company

Kane LPI Solutions is a prime example of how Kibana reporting achieved higher marketability within the cognoscente program Skedler. A trusted provider of Third Party Administration services for more than 15 years, Kane LPI has issued over US$11 billion of offshore annuity and investment products for an extensive global client base. They needed a robust solution to help prevent financial and reputation losses as well as boost KPIs. As it stood, competition was fierce — there were many much larger players out there on the field.

The Challenge

The company’s practical processes specifically involved sending sensitive and accurate post-trading files on a timely basis. Kane LPI’s clients had strict operational requirements for control and compliance: if the time window for sending files was missed, financial loss was borne both for them and the company. Delays like this could incur strict penalties by settlement and clearing corporations such as The Depository Trust and Clearing Corporation (DTCC). With so much at stake, our challenge was to fundamentally satisfy operational concerns to prevent those losses, as well as improve pragmatic business flow.

Kibana Reporting: A Means of Automation

Kibana reporting allowed Kane LPI to send out daily scheduled reports to both clients and internal users from thousands of lines of log entries within multiple systems, serving as a key monitoring tool and satisfying auditing requirements in the process. As a result, Skedler became a critical go-to ELK stack tool for this company and its clients, allowing Kane LPI to send out reports to their clients and internal users, which the company couldn’t do before.

Developers were able to receive automated error reports at the beginning of the day, allowing them to ensure information is sent to regulatory and settlement organizations to meet deadlines. Simultaneously, managers received daily and weekly reports on SLA performance and non-compliance, enabling them to take remedial steps to prevent recurrence and to protect time-sensitive transactions. Vendors also received daily and weekly reports on SLA performance and non-compliance of their product in KPI solution, which fundamentally reused the investment of energy and time in an ELK stack based solution.

As a precautionary measure, Skedler introduced a critical line of defence to errors by inaugurating manager and software vendors with automated reporting sent at the beginning of the day, which described potential errors during transfer of files, allowing future errors to be minimized. We then added another layer of security by administering any issues during batch runs from the previous 24 hours. 

Statistically, all of these enhancements enabled Kane LPI’s clients to avoid up to $5 million per month in trade and exchange losses, as well as protect their well-nurtured reputation as a high-quality provider.

Ready to start saving time by creating, scheduling and distributing Kibana reports automatically? Try Skedler for free.

The Top 3 ELK Stack Tools Every Business Intelligence Analyst Needs in 2017

A version of this post, updated for 2018, can be found here: The Top 5 ELK Stack+ Tools Every Business Intelligence Analyst Needs.

The world’s most popular log management platform, ELK Stack, has ultimately reflected its nifty, modernized capabilities with this recent statistic: each month, it is downloaded 500,000 times. So what makes ELK Stack and ELK Stack Tools just so attractive? In many cases, it fulfills what’s really been needed in the log analytics space within SaaS: IT companies are favoring open source products more and more. Since it’s based on the Lucene search engine, Elasticsearch is a NoSQL database which forms as a log pipeline tool; accepting inputs from various sources, executing transformations, then exporting data to designated targets. It also carries enhanced customizability, which is a key preference nowadays, since program tweaking is more lucrative and stimulating for many engineers. This is coupled with ELK’s increased interoperability, which is now a practically indispensable feature, since most businesses don’t want to be limited by proprietary data formats.

ELK Stack tools which simply higher-tier those impressive elements will elevate data analysis just that little bit further; depending on what you want to do with it, of course.

Logstash

Elite tool Logstash is well-known for its intake, processing and output capabilities. It’s mainly intended for organizing and searching for log files, but works effectively for cleaning and streaming big data from all sorts of sources into a comprehensive database, including metrics, web applications, data stores, and various AWS services. Logstash also carries impressive input plugins such as cloudwatch and graphite, allowing you to sculpt your intelligence to be as easy to work with as possible. And, as data travels from source to store, those filters identify named fields to accelerate your analysis; deciphering geo coordinates from IP addresses, and anonymizing PII data. It even derives structure from seemingly unstructured data.

Kibana 5

Analysis program Kibana 5.0 boasts a wealth of new refurbishments for pioneering intelligence surveying. Apart from amplified functionalities such as increased rendering, less CPU usage, and elevated data and index handling, Kibana 5.0 has enriched visualisations with interactive platforms, leveraging the aggregation capabilities of Elasticsearch. Space and time auditing are a crucial part of Kibana’s make up: the map service empowers you to foresee geospatial data with custom location data on a schematic of your selection, whilst the time series allows you to perform advanced generation analysis by describing queries and transformations.

Skedler

ELK Stack reporting tool, Skedler, combines all the automated processes you’d never dream you could have within one unit. Fundamentally, it ups your speed-to-market auditing with cutting-edge scheduling, which Kibana alone does not offer; serving as a single system for both interactive analysis and reporting. Skedler methodically picks up your existing dashboards in the server for cataloging, whilst also enabling you to create filters, refine specific recipients, and filter file folders to use whilst scheduling. Additionally, Skedler automatically applies prerequisite filters with generate reports, preserving them as defined; and encompasses high-resolution PDF and PNG options to incorporate in reporting, which sequentially eliminates the need for redundant reporting systems.

There you have it, the top ELK stack tools no business intelligence analyst should ever be without!

Ready to start streamlining your analysis and start reporting with more stability? Right now, we’re offering a free trial.

Are You Wasting Time Manually Sending Kibana Reports?

Automated processes are, invariably, becoming more and more integral to our everyday lives, both in and out of the office. They’ve replaced much of the manual workforce and have improved systematic procedures, which otherwise would be at the mercy of various human error elements as well as higher risks of data breaches. This, as well as recognizing manual reporting as time-consuming labour, are some key issues we don’t need to worry about any more by virtue of processing automation; Kibana being one of those favorable products.

Focus on What Matters

As a result of businesses adopting bots as part of our everyday processes, we’re left with the far more creative aspects of information science (which automation hasn’t quite caught up with yet). Naturally, Elasticsearch’s aesthetically enhanced data delivery is one of its chief selling points: users are able to explore unchartered data with clear-cut digital graphics at their very disposal. This significant upgrade in data technology has allowed us to possess more varied and complex insights; it’s more exciting now than it has ever been before.

In contrast, however, tedious tasks such as email deliveries of reports to customers, compliance managers and other stakeholders remain arduous and time-consuming; deterring attention from more stimulating in-depth data analysis. What we know to be necessary is for analysts to have the time available to devote themselves to exploring Tableau’s analytics, instead of undergoing mundane processes such as manual spreadsheet creation, generating, email exporting, and distributing.

Automate Kibana Reports

Perhaps it’s possible that you’ve already started utilizing Kibana without realizing the perks of automated scheduling. Luckily, Skedler can completely undertake those prosaic tasks, at an affordable price. As an automated scheme which meets full compliance and operations requirements, Skedler allows your peers, customers and other stakeholders to be kept informed in a virtually effortless and secure way. Comprehensive exporting preferences such as PDF, XLS and PNG are also serviceable; allowing you the luxury of consigning instant or scheduled report generation in the format you desire.

Additionally, Skedler’s reporting motions are facilitated through its prestigious dashboard system, which automatically discovers your existing Kibana dashboards and saved searches to make them available for reporting – again, saving you time creating, scheduling and sending Kibana reports. All your filtered reporting and data chartering is available on a single, versatile platform; meaning you won’t spend extensive amounts of time searching through your outgoing email reports for a specific item.

Skedler simply allows you to examine all of your criteria through one umbrella server with clear functionalities to separate the stunning data visualization deliveries, and the slightly less exciting archive of manual spreadsheet generation and handling for other departments, which it can totally manage by itself.

Ready to start saving time by creating, scheduling and distributing Kibana reports automatically? Try Skedler for free.

3 Apps to Get the Most Out of Kibana 5.0

A new financial quarter starts, full-scale data appraisals are once again at the forefront for every business’ sales agenda. Luckily, Elasticsearch’s open source tool Kibana 5.0 is the talk of the town – and for good reason.

Improvements since version 4.0 are unequivocally noticeable. Its new and far more sleek user interface display not only wows in terms of visuals (note the subsidiary menu that minimizes when not in use), but demonstrates impressive UI capabilities that allows you to reach data far more effectively. The new CSV upload, for example, has the potential to catch a much wider data spread, transforming it to index mapping that’s effortlessly navigable. Its new management tab allows you to view the history of the files with associated data, as well as Elasticsearch indexes where you actively send log files.

This version’s huge boost in code architecture grants the potential for more augmentations than ever, especially with split code self-contained plugins with open-end code tweaking, resulting in several lucrative alpha and beta versions. And it’s essentially allowed us the privilege to now ask: what kind of data insight does my company really need, and which app is best to harness it?

1. Logz.io

Logz.io has fundamentally enriched Kibana with two major touches: increased data security, and more serviceable enterprise sequences as a result. Take their access user tokens, for example, which enable share visualizations and dashboard with those who aren’t necessarily Logz.io users, rather than the URL share function. You can pretty much be as selective with your data as you so please; specific and cross-referenced filter searches are an added function to the tokens. This makes it easy to attach pre-saved filters when back in Kibana.

2. Skedler

Skedler has specifically focused developed reporting capabilities with actionables to perform on data, effectively meaning you can do more with it all in a proactive way. Scheduling is an integral part of this program’s faculty, as it works with your existing database searches and dashboards; allowing you to organize dispatches daily, weekly, monthly and so on. Again, you’re able to action specific filters as and when you’re scheduling, making your reports as customized as needed when sending for peer review.

3. Predix

Predix has established itself as a strong contender for effective data trend sweeps, such as HTTP responses, latencies and visitors – and you’re able to debug apps at the sam e time. Combining this with Kibana’s exhaustive data visualizations and pragmatic dashboard, controlling and managing your log data not only highly secure, but it allows you to become more prognostic when forecasting future data.

Ready to save hours generating, scheduling and distributing PDF and XLS reports from your
Elasticsearch Kibana (ELK) application to your team, customers and other stakeholders? Try Skedler for free.

Skedler v1.3 is here, loaded with features!

We are excited to announce the early availability of Skedler v1.3 with amazing features that have been on the wish list of our customers.

Schedule High Quality Customized PDF Reports

Customize your PDF in Skedler

Whether you email PDF reports to internal users, partners, or customers, quality and customization are important aspects to consider.  Skedler v1.3 is designed to make it incredibly easy to schedule and email customized high quality PDF reports based on your Kibana dashboards.   You can now customize the logo, report titles, page size, layout, and font for your scheduled PDF reports.   We have made significant enhancements to the PDF report quality.  You need to see it to believe it.

Attach Excel Report to PDF Reports

Attach Excel Reports

What if your stakeholders prefer playing with the raw data using Excel?  We got your covered.  With one click, you can attach excel export to scheduled PDF reports of your Kibana dashboard.   Check out a sample excel report here.

Schedule PDF Reports from Kibana Searches

Schedule Report for Kibana Search

Reports based on Kibana Discover (Search) tab has been a frequent request from our customers and today, we are happy to say that Skedler v1.3 has taken the first big step in fulfilling this popular request. You can now schedule PDF reports in Skedler for your saved searches in Kibana Discover tab.  No need to create dashboards! Combine it with the PDF customization capability to schedule high quality reports for your stakeholders.  You can’t generate Excel reports yet, but it is coming soon!

Easy to Install + Amazing Support

Skedler is designed such that most customers can get it running within an hour.  However, if your environment is unique (OS/network/Shield/Nginx/Mail server configuration issues) and if you need help with the configuration, we have an amazingly responsive and talented Skedler team to support you through your evaluation and beyond.

Free Trial

You can try Skedler free to experience the value of Skedler and decide if you would like to include it in your solution.  Contact us (use your official email id only please) to get started.  We will  send you an electronic (Docusign) Evaluation License Agreement.  Upon signing it, we will send you the link to download the latest version.

Email XLS Reports from ELK to data analysts

Introducing XLS Email Reports in Skedler 1.1

Skedler PDF reports  are an easy way to send actionable visual  information from your Elasticsearch-Kibana application to non-technical users.   But how do you meet the requirement from your customers, data analysts and users  who are looking for raw data from your Kibana application?   What if they would like to receive data periodically for analyzing with Microsoft Excel or Tableau?

We are proud to announce the beta release of Skedler 1.1 that allows you to schedule XLS reports in addition to PDF, PNG and in-line PNG reports.

Schedule XLS reports in Skedler

Scheduled XLS reports makes it simple to schedule, generate, and send hourly/daily/weekly/monthly data snapshots to data analysts.   Skedler will automatically generate an XLS worksheet for each chart in your Kibana dashboard, combine all the worksheets into a single XLS file, and email the report to target recipients as per the schedule.

Consider the following Kibana dashboard with several charts providing information regarding Crime statistics in Alameda County for the past 90 days.

Alameda county – Past 90 days crime statistics

Skedler will automatically convert this dashboard into a single XLS report with multiple worksheets, one worksheet for each chart.

Daily XLS Report screenshot

You can notice that the XLS report contains six(6) worksheets, each corresponding to the chart in the Kibana dashboard.   Skedler automatically uses the chart title for the worksheet name so that it is easy to identify the worksheets and the data.  You can download the sample email report from Daily XLS Report of All Crimes Dashboard.

In addition to emailing XLS report to customers, data analysts and others who prefer to use their own tools,  you can also generate and store the XLS reports in server side file folders which can then be used for various analysis or as a data feed for other applications.

Test drive Skedler v1.1 Beta with XLS reports

If you are interested in trying out the Beta release of Skedler v1.1,  contact us and we will help you get started with Skedler.  Skedler is easy to set up and get started.  We will provide you with the technical support required to get past common configuration issues.

Introducing Skedler – Powerful Report Scheduling for Kibana

Kibana is a nice tool for quickly creating real-time dashboards from Elasticsearch data.   But, what if your managers, business users and other stakeholders need reports delivered to them periodically, e.g., weekly status report on business/technical metrics? Wouldn’t it be great to send the right information to the right people at the right time automatically instead of asking them to check the dashboard periodically?

Possible Solutions

Unfortunately, Kibana doesn’t offer this reporting functionality which is a common feature in traditional BI reporting platforms.  So, the possible solutions are:

1.  Use a BI report scheduling tool that has a plug-in for Elasticsearch and build the reports from scratch –>  For most customers, this is not practical approach for two reasons:  1.  You need to recreate the reports from scratch and maintain them in two systems (Kibana and the BI platform).  2.  BI platforms are large deployments requiring dedicated resources.  So, unless you already have a BI platform that works with Kibana / Elasticsearch, this approach is an expensive affair.

2.  Build your own custom report scheduler combining a bunch of tools –>  This is a mammoth effort.  Scheduling and generating consistent reports from Kibana and Elasticsearch is a very complex technical exercise which requires deep expertise on several fronts (UI engineering, Kibana, Elasticsearch, Back end scheduler expertise, systems expertise to handle Kibana/Elasticsearch configurations etc).  Plus, you need to allocate dedicated resources to keep pace with the frequent changes in Elasticsearch/Kibana.

For most organizations, both #1 and #2 are not viable options.  They can neither afford a new BI reporting platform nor divert their scarce engineering resources to build and manage this software.

Introducing Skedler – Report Scheduler for Kibana

Skedler is a powerful report scheduling solution that allows organizations to schedule, generate, and distribute Kibana dashboards periodically to various stakeholders.  It is simple to install and is designed to work seamlessly with your existing Kibana/Elasticsearch installation.

Skedler enables organizations to create PDF and PNG snapshots (more format to come soon) quickly using Kibana dashboards.  It provides the flexibility and features to ensure that reports are sent to the right people when they need them through centralized scheduling and distribution.  With Report Scheduler, Kibana can serve as a single system for both interactive analysis and reporting, eliminating redundant systems and saving significant costs.

See how Skedler works in this video below:

https://www.youtube.com/watch?v=try-KdXm11o&utm_source=skedler_site&utm_medium=blog_article&utm_campaign=howitworks_video

Easy Installation

Skedler can be installed and ready to go in just a few seconds.  Simply unzip the skedler package in a server, configure your Kibana and Elasticsearch URLs in the config file, start the skedler engine and you are all set to start generating reports.

Auto-discovery of Kibana dashboards

Skedler automatically discovers the dashboards in your Kibana server and makes them available for scheduling.

High Resolution PDF Reports

Skedler generates high resolution PDF reports from your Kibana dashboards and auto-optimizes the report regardless of how your Kibana dashboard is set up.  You can also customize the resolution of the reports as needed to suit your requirements.

Dynamic filters, recipients, and folders

What if you want to generate customer specific reports from a single dashboard and email the reports to targeted email recipients or save them in custom folders?  Skedler has a solution for that.   You can create filters, filter specific recipients, and filter specific file folders in Skedler and use them while scheduling reports. Skedler will automatically apply the filters on the fly to dashboards, generate reports, and email/preserve the reports as defined.

Eliminate Redundant Reporting Systems

Report Scheduler allows you to drive both analysis and reporting from the Kibana platform, eliminating the need for redundant reporting systems.

See a Live Demo and Test Drive Skedler

Skedler is a powerful solution that will allow you to add report scheduling capability to your Kibana-Elasticsearch system within a few minutes.  See a live demo of the Skedler solution and get set up for a free test drive.  Request a live demo and test drive  today.

Save

Save

Save