How To Install And Configure Kibana In Linux

Hello Everyone

Welcome to CloudAffaire and this is Debjeet.

In this series, we will explore one of the most popular log management tools in DevOps better known as ELK (E=Elasticserach, L=Logstash, K=Kibana) stack.

What Is Kibana?

Kibana is an open-source analytics and visualization platform designed to work with Elasticsearch. You use Kibana to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualize your data in a variety of charts, tables, and maps.

Kibana makes it easy to understand large volumes of data. Its simple, browser-based interface enables you to quickly create and share dynamic dashboards that display changes to Elasticsearch queries in real-time.

Configuring Kibana:

The Kibana server reads properties from the kibana.yml file on startup. The location of this file differs depending on how you installed Kibana. For example, if you installed Kibana from an archive distribution (.tar.gz or .zip), by default it is in $KIBANA_HOME/config. By default, with package distributions (Debian or RPM), it is in /etc/kibana.

The default host and port settings configure Kibana to run on localhost:5601. To change this behavior and allow remote users to connect, you’ll need to update your kibana.yml file. You can also enable SSL and set a variety of other options. Finally, environment variables can be injected into configuration using ${MY_ENV_VAR} syntax.

Kibana Configuration Options:

• console.enabled: Default: true Set to false to disable Console.
• cpu.cgroup.path.override: Override for cgroup cpu path when mounted in a manner that is inconsistent with /proc/self/cgroup
• cpuacct.cgroup.path.override: Override for cgroup cpuacct path when mounted in a manner that is inconsistent with /proc/self/cgroup
• csp.rules: A template content-security-policy that disables certain unnecessary and potentially insecure capabilities in the browser.
• csp.strict: Default: false Blocks access to Kibana to any browser that does not enforce even rudimentary CSP rules.
• csp.warnLegacyBrowsers: Default: true Shows a warning message after loading Kibana to any browser that does not enforce even rudimentary CSP rules.
• elasticsearch.customHeaders: Default: {} Header names and values to send to Elasticsearch.
• elasticsearch.hosts: Default: [ “http://localhost:9200” ] The URLs of the Elasticsearch instances to use for all your queries.
• elasticsearch.logQueries: Default: false Logs queries sent to Elasticsearch.
• elasticsearch.pingTimeout: Default: the value of the elasticsearch.requestTimeout setting Time in milliseconds to wait for Elasticsearch to respond to pings.
• elasticsearch.preserveHost: Default: true When this setting’s value is true, Kibana uses the hostname specified in the server.host setting.
• elasticsearch.requestHeadersWhitelist: Default: [ ‘authorization’ ] List of Kibana client-side headers to send to Elasticsearch. T
• elasticsearch.requestTimeout: Default: 30000 Time in milliseconds to wait for responses from the back end or Elasticsearch. This value must be a positive integer.
• elasticsearch.shardTimeout: Default: 30000 Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
• elasticsearch.sniffInterval: Default: false Time in milliseconds between requests to check Elasticsearch for an updated list of nodes.
• elasticsearch.sniffOnConnectionFault: Default: false Update the list of Elasticsearch nodes immediately following a connection fault.
• elasticsearch.sniffOnStart: Default: false Attempt to find other Elasticsearch nodes on startup.
• elasticsearch.ssl.alwaysPresentCertificate: Default: false Controls Kibana’s behavior in regard to presenting a client certificate when requested by Elasticsearch.
• elasticsearch.ssl.certificate: and elasticsearch.ssl.key: Paths to a PEM-encoded X.509 client certificate and its corresponding private key.
• elasticsearch.ssl.certificateAuthorities: Paths to one or more PEM-encoded X.509 certificate authority (CA) certificates which make up a trusted certificate chain for Elasticsearch.
• elasticsearch.ssl.keyPassphrase: The password that will be used to decrypt the private key that is specified via elasticsearch.ssl.key.
• elasticsearch.ssl.keystore.path: Path to a PKCS#12 keystore that contains an X.509 client certificate and its corresponding private key.
• elasticsearch.ssl.keystore.password: The password that will be used to decrypt the keystore that is specified via elasticsearch.ssl.keystore.path.
• elasticsearch.ssl.truststore.path: Path to a PKCS#12 trust store that contains one or more X.509 certificate authority (CA) certificates.
• elasticsearch.ssl.truststore.password: The password that will be used to decrypt the trust store specified via elasticsearch.ssl.truststore.path.
• elasticsearch.ssl.verificationMode: Default: “full” Controls the verification of the server certificate that Kibana receives when making an outbound SSL/TLS connection to Elasticsearch.
• elasticsearch.startupTimeout: Default: 5000 Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
• elasticsearch.username: and elasticsearch.password: Provide the username and password that the Kibana server uses to perform maintenance on the Kibana index at startup.
• interpreter.enableInVisualize: Default: true Enables use of interpreter in Visualize.
• kibana.defaultAppId: Default: “home” The default application to load.
• kibana.index: Default: “.kibana” Kibana uses an index in Elasticsearch to store saved searches, visualizations, and dashboards.
• kibana.autocompleteTimeout: Default: “1000” Time in milliseconds to wait for autocomplete suggestions from Elasticsearch.
• kibana.autocompleteTerminateAfter: Default: “100000” Maximum number of documents loaded by each shard to generate autocomplete suggestions.
• logging.dest: Default: stdout Enables you specify a file where Kibana stores log output.
• logging.json: Default: false Logs output as JSON.
• logging.quiet: Default: false Set the value of this setting to true to suppress all logging output other than error messages.
• logging.rotate: [experimental] Specifies the options for the logging rotate feature.
• logging.rotate.enabled: [experimental] Default: false Set the value of this setting to true to enable log rotation.
• logging.rotate.everyBytes: [experimental] Default: 10485760 The maximum size of a log file (that is not an exact limit).
• logging.rotate.keepFiles: [experimental] Default: 7 The number of most recent rotated log files to keep on disk.
• logging.rotate.pollingInterval: [experimental] Default: 10000 The number of milliseconds for the polling strategy in case the logging.rotate.usePolling is enabled.
• logging.rotate.usePolling: [experimental] Default: false By default we try to understand the best way to monitoring the log file.
• logging.silent: Default: false Set the value of this setting to true to suppress all logging output.
• logging.timezone: Default: UTC Set to the canonical timezone id (for example, America/Los_Angeles) to log events using that timezone.
• logging.verbose: Default: false Set the value of this setting to true to log all events, including system usage information and all requests.
• map.proxyElasticMapsServiceInMaps: Default: false Set to true to proxy all Maps application Elastic Maps Service requests through the Kibana server.
• map.regionmap: Specifies additional vector layers for use in Region Map visualizations.
• map.regionmap.includeElasticMapsService: Turns on or off whether layers from the Elastic Maps Service should be included in the vector layer option list.
• map.regionmap.layers[].attribution: Optional. References the originating source of the geojson file.
• map.regionmap.layers[].fields[]: Mandatory. Each layer can contain multiple fields to indicate what properties from the geojson features you wish to expose.
• map.regionmap.layers[].fields[].description: Mandatory. The human readable text that is shown under the Options tab when building the Region Map visualization.
• map.regionmap.layers[].fields[].name: Mandatory. This value is used to do an inner-join between the document stored in Elasticsearch and the geojson file.
• map.regionmap.layers[].name: Mandatory. A description of the map being provided.
• map.regionmap.layers[].url: Mandatory. The location of the geojson file as provided by a webserver.
• map.tilemap.options.attribution: Default: “© [Elastic Maps Service](https://www.elastic.co/elastic-maps-service)” The map attribution string.
• map.tilemap.options.maxZoom: Default: 10 The maximum zoom level.
• map.tilemap.options.minZoom: Default: 1 The minimum zoom level.
• map.tilemap.options.subdomains: An array of subdomains used by the tile service. Specify the position of the subdomain the URL with the token {s}.
• map.tilemap.url: The URL to the tileservice that Kibana uses to display map tiles in tilemap visualizations.
• ops.interval: Default: 5000 Set the interval in milliseconds to sample system and process performance metrics. The minimum value is 100.
• newsfeed.enabled: Default: true Controls whether to enable the newsfeed system for the Kibana UI notification center.
• path.data: Default: data The path where Kibana stores persistent data not saved in Elasticsearch.
• pid.file: Specifies the path where Kibana creates the process ID file.
• server.basePath: Enables you to specify a path to mount Kibana at if you are running behind a proxy. .
• server.compression.enabled: Default: true Set to false to disable HTTP compression for all responses.
• server.compression.referrerWhitelist: Default: none Specifies an array of trusted hostnames, such as the Kibana host, or a reverse proxy sitting in front of it.
• server.cors: Default: false Set to true to enable CORS support. This setting is required to configure server.cors.origin.
• server.cors.origin: Default: none Specifies origins. “origin” must be an array.
• server.customResponseHeaders: Default: {} Header names and values to send on all responses to the client from the Kibana server.
• server.host: Default: “localhost” This setting specifies the host of the back end server. To allow remote users to connect, set the value to the IP address or DNS name of the Kibana server.
• server.keepaliveTimeout: Default: “120000” The number of milliseconds to wait for additional data before restarting the server.socketTimeout counter.
• server.maxPayloadBytes: Default: 1048576 The maximum payload size in bytes for incoming server requests.
• server.name: Default: “your-hostname” A human-readable display name that identifies this Kibana instance.
• server.port: Default: 5601 Kibana is served by a back end server. This setting specifies the port to use.
• server.socketTimeout: Default: “120000” The number of milliseconds to wait before closing an inactive socket.
• server.ssl.certificate: and server.ssl.key: Paths to a PEM-encoded X.509 server certificate and its corresponding private key.
• server.ssl.certificateAuthorities: Paths to one or more PEM-encoded X.509 certificate authority (CA) certificates which make up a trusted certificate chain for Kibana.
• server.ssl.clientAuthentication: Default: “none” Controls Kibana’s behavior in regard to requesting a certificate from client connections.
• server.ssl.enabled: Default: false Enables SSL/TLS for inbound connections to Kibana.
• server.ssl.keyPassphrase: The password that will be used to decrypt the private key that is specified via server.ssl.key.
• server.ssl.keystore.path: Path to a PKCS#12 keystore that contains an X.509 server certificate and its corresponding private key.
• server.ssl.keystore.password: The password that will be used to decrypt the keystore specified via server.ssl.keystore.path.
• server.ssl.truststore.path: Path to a PKCS#12 trust store that contains one or more X.509 certificate authority (CA) certificates which make up a trusted certificate chain for Kibana.
• server.ssl.truststore.password: The password that will be used to decrypt the trust store specified via server.ssl.truststore.path.
• server.ssl.redirectHttpFromPort: Kibana will bind to this port and redirect all http requests to https over the port configured as server.port.
• server.ssl.supportedProtocols: Default: TLSv1.1, TLSv1.2 An array of supported protocols with versions. Valid protocols: TLSv1, TLSv1.1, TLSv1.2
• server.xsrf.whitelist: It is not recommended to disable protections for arbitrary API endpoints.
• status.allowAnonymous: Default: false If authentication is enabled, setting this to true enables unauthenticated users to access the Kibana server status API and status page.
• telemetry.allowChangingOptInStatus: Default: true. If true, users are able to change the telemetry setting at a later time in Advanced Settings. If false, Kibana looks at the value of telemetry.optIn to determine whether to send telemetry data or not. telemetry.allowChangingOptInStatus and telemetry.optIn cannot be false at the same time.
• telemetry.optIn: Default: true If true, telemetry data is sent to Elastic. If false, collection of telemetry data is disabled. To enable telemetry and prevent users from disabling it, set telemetry.allowChangingOptInStatus to false and telemetry.optIn to true.
• telemetry.enabled: Default: true Reporting your cluster statistics helps us improve your user experience.
• vega.enableExternalUrls: Default: false Set this value to true to allow Vega to use any URL to access external data sources and images.
• xpack.license_management.enabled: Default: true Set this value to false to disable the License Management user interface.
• xpack.rollup.enabled: Default: true Set this value to false to disable the Rollup user interface.
• i18n.locale: Default: en Set this value to change the Kibana interface language. Valid locales are: en, zh-CN, ja-JP.
• xpack.actions.enabledActionTypes: Default: [ * ] Set this value to an array of action types that are enabled.
• xpack.actions.whitelistedHosts: Default: [ * ] Set this value to an array of host names which actions such as email, slack, pagerduty, and webhook can connect to.
• xpack.apm.enabled: Set to false to disabled the APM plugin Kibana. Defaults to true.
• xpack.apm.ui.enabled: Set to false to hide the APM plugin Kibana from the menu. Defaults to true.
• xpack.apm.ui.transactionGroupBucketSize: Number of top transaction groups displayed in APM plugin in Kibana. Defaults to 100.
• xpack.apm.ui.maxTraceItems: Max number of child items displayed when viewing trace details. Defaults to 1000.
• apm_oss.indexPattern: Index pattern is used for integrations with Machine Learning and Kuery Bar. It must match all apm indices. Defaults to apm-*.
• apm_oss.errorIndices: Matcher for indices containing error documents. Defaults to apm-*.
• apm_oss.onboardingIndices: Matcher for indices containing onboarding documents. Defaults to apm-*.
• apm_oss.spanIndices: Matcher for indices containing span documents. Defaults to apm-*.
• apm_oss.transactionIndices: Matcher for indices containing transaction documents. Defaults to apm-*.
• apm_oss.metricsIndices: Matcher for indices containing metric documents. Defaults to apm-*.
• apm_oss.sourcemapIndices: Matcher for indices containing sourcemap documents. Defaults to apm-*.
• xpack.grokdebugger.enabled: Set to true (default) to enable the Grok Debugger.
• xpack.searchprofiler.enabled: Set to true (default) to enable the Query Profiler.
• xpack.graph.enabled: Set to false to disable the graph analytics features.
• xpack.infra.enabled: Set to false to disable the Logs and Metrics app plugin Kibana. Defaults to true.
• xpack.infra.sources.default.logAlias: Index pattern for matching indices that contain log data. Defaults to filebeat-*,kibana_sample_data_logs*.
• xpack.infra.sources.default.metricAlias: Index pattern for matching indices that contain Metricbeat data. Defaults to metricbeat-*.
• xpack.infra.sources.default.fields.timestamp: Timestamp used to sort log entries. Defaults to @timestamp.
• xpack.infra.sources.default.fields.message: Fields used to display messages in the Logs app. Defaults to [‘message’, ‘@message’].
• xpack.infra.sources.default.fields.tiebreaker: Field used to break ties between two entries with the same timestamp. Defaults to _doc.
• xpack.infra.sources.default.fields.host: Field used to identify hosts. Defaults to host.name.
• xpack.infra.sources.default.fields.container: Field used to identify Docker containers. Defaults to container.id.
• xpack.infra.sources.default.fields.pod: Field used to identify Kubernetes pods. Defaults to kubernetes.pod.uid.
• i18n.locale: Kibana currently supports the following locales: English – en (default), Chinese – zh-CN, Japanese – ja-JP.
• xpack.ml.enabled: Set to true (default) to enable Kibana machine learning features.
• xpack.monitoring.enabled: Set to true (default) to enable the monitoring features in Kibana.
• xpack.monitoring.elasticsearch.hosts: Specifies the location of the Elasticsearch cluster where your monitoring data is stored. By default, this is the same as elasticsearch.hosts.
• xpack.monitoring.elasticsearch.username: Specifies the username used by Kibana monitoring to establish a persistent connection in Kibana to the Elasticsearch monitoring cluster.
• xpack.monitoring.elasticsearch.password: Specifies the password used by Kibana monitoring to establish a persistent connection in Kibana to the Elasticsearch monitoring cluster.
• xpack.monitoring.elasticsearch.pingTimeout: Specifies the time in milliseconds to wait for Elasticsearch to respond to internal health checks, has a default value of 30000.
• xpack.monitoring.kibana.collection.enabled: Set to true (default) to enable data collection from the Kibana NodeJS server for Kibana Dashboards to be featured in the Monitoring.
• xpack.monitoring.kibana.collection.interval: Number of milliseconds to wait in between data sampling on the Kibana NodeJS server for the metrics that are displayed in the Kibana dashboards.
• xpack.monitoring.elasticsearch.logFetchCount: Specifies the number of log entries to display in the Monitoring UI. Defaults to 10. The maximum value is 50.
• xpack.monitoring.max_bucket_size: Specifies the number of term buckets to return out of the overall terms list when performing terms aggregations to retrieve index and node metrics.
• xpack.monitoring.min_interval_seconds: Specifies the minimum number of seconds that a time bucket in a chart can represent. Defaults to 10.
• xpack.monitoring.ui.enabled: Set to false to hide the Monitoring UI in Kibana.
• xpack.reporting.enabled: Set to false to disable the reporting features.
• xpack.reporting.encryptionKey: Set to any text string. By default, Kibana will generate a random key when it starts, which will cause pending reports to fail after restart.
• xpack.reporting.kibanaServer.port: The port for accessing Kibana, if different from the server.port value.
• xpack.reporting.kibanaServer.protocol: The protocol for accessing Kibana, typically http or https.
• xpack.reporting.kibanaServer.hostname: The hostname for accessing Kibana, if different from the server.host value.
• xpack.reporting.queue.indexInterval: How often the index that stores reporting jobs rolls over to a new index. Defaults to week.
• xpack.reporting.queue.pollEnabled: Set to true (default) to enable the Kibana instance to to poll the index for pending jobs and claim them for execution.
• xpack.reporting.queue.pollInterval: Specifies the number of milliseconds that the reporting poller waits between polling the index for any pending Reporting jobs. Defaults to 3000.
• xpack.reporting.queue.timeout: How long each worker has to produce a report. Specified in milliseconds. Defaults to 120000
• xpack.reporting.capture.maxAttempts: If capturing a report fails for any reason, Kibana will re-attempt othe reporting job, as many times as this setting. Defaults to 3.
• xpack.reporting.capture.loadDelay: When visualizations are not evented, this is the amount of time before taking a screenshot.
• xpack.reporting.capture.browser.type: Specifies the browser to use to capture screenshots. This setting exists for backward compatibility. The only valid option is chromium.
• xpack.reporting.capture.browser.chromium.disableSandbox: Defaults to false for all operating systems except Debian, Red Hat Linux, and CentOS which use true
• xpack.reporting.capture.browser.chromium.proxy.enabled: Enables the proxy for Chromium to use. Defaults to false
• xpack.reporting.capture.browser.chromium.proxy.server: The uri for the proxy server. Providing the username and password for the proxy server via the uri is not supported.
• xpack.reporting.capture.browser.chromium.proxy.bypass: An array of hosts that should not go through the proxy server and should use a direct connection instead.
• xpack.reporting.csv.maxSizeBytes: The maximum size of a CSV file before being truncated.
• xpack.reporting.index: Reporting uses a weekly index in Elasticsearch to store the reporting job and the report content. Defaults to .reporting.
• xpack.reporting.roles.allow: Specifies the roles in addition to superusers that can use reporting. Defaults to [ “reporting_user” ]
• xpack.security.cookieName: Sets the name of the cookie used for the session. The default value is “sid”.
• xpack.security.encryptionKey: An arbitrary string of 32 characters or more that is used to encrypt credentials in a cookie.
• xpack.security.secureCookies: Sets the secure flag of the session cookie. The default value is false.
• xpack.security.session.idleTimeout: Sets the session duration. The format is a string of <count>[ms|s|m|h|d|w|M|Y] (e.g. 70ms, 5s, 3d, 1Y).
• xpack.security.session.lifespan: Sets the maximum duration, also known as “absolute timeout”.
• xpack.security.loginAssistanceMessage: Adds a message to the login screen. Useful for displaying information about maintenance windows, links to corporate sign up pages etc.
• xpack.spaces.enabled: Set to true (default) to enable Spaces in Kibana.
• xpack.spaces.maxSpaces: The maximum amount of Spaces that can be used with this instance of Kibana. Defaults to 1000.

Installing Kibana in Linux:

Step 1: Create a yum repository for Kibana.

Step 2: Install Kibana.

To get more details on ELK, please refer below documentation.

https://www.elastic.co/guide/index.html