Elasticsearch: Secure Settings

Created on 6 Jan 2017 · 3Comments · Source: elastic/elasticsearch

This is a meta issue to track work on making sensitive settings secure in elasticsearch. The core infrastructure for this feature is added in #22335, which provides the elasticsearch-keystore tool. The following work is still necessary to consider the feature complete:

[x] Convert existing settings (eg aws keys) to new infrastructure
[x] Add password support to the keystore
[x] Add reading password for start scripts (init.d and systemd)
[x] Investigate best location for keystore file. Currently in config dir, which has read permissions for the life of the node, but could place somewhere with no SM permissions, since reading is done before SM is setup.

The following would be nice to haves:

[ ] Investigate only allowing reading secure settings registered by plugins (eg not allowing reading other plugin's secure settings)
[x ] ~~Investigate setting explicit algorithm for PBE~~
[ ] Add support for private keys
[ ] Add support for certificates
[ ] Add support to read secure settings from Vault instead of keystore

:CorInfrSettings Meta CorInfra team-discuss

Source

rjernst

Most helpful comment

Dears,
I faced an issue with elasticsearch.keystore location when using Elastic on Kubernetes clusters and configuration are mounted via ConfigMaps. Due to this I have "read-only" fs of /etc/elasticsearch and because of that , ES is going to crash.

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.FileSystemException: /etc/elasticsearch/elasticsearch.keystore.tmp: Read-only file system
--
  | Likely root cause: java.nio.file.FileSystemException: /etc/elasticsearch/elasticsearch.keystore.tmp: Read-only file system
  | at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
  | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
  | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
  | at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
  | at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
  | at java.nio.file.Files.newOutputStream(Files.java:216)
  | at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:411)
  | at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:407)
  | at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:255)
  | at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:467)
  | at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:238)
  | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:295)
  | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
  | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
  | at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
  | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
  | at org.elasticsearch.cli.Command.main(Command.java:90)
  | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
  | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
  | Refer to the log for complete error details.

It will be great to have a settings for keystore location in elasticsearch.yaml to set directory outside of /etc/elasticsearch.

Can you advice in this topic?

Thanks in advance,

pdanysz on 18 Mar 2019

👍5

All 3 comments

I filed an issue to cover the specific feature of password protecting the keystore
https://github.com/elastic/elasticsearch/issues/32691

joshbressers on 7 Aug 2018

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.FileSystemException: /etc/elasticsearch/elasticsearch.keystore.tmp: Read-only file system
--
  | Likely root cause: java.nio.file.FileSystemException: /etc/elasticsearch/elasticsearch.keystore.tmp: Read-only file system
  | at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
  | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
  | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
  | at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
  | at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
  | at java.nio.file.Files.newOutputStream(Files.java:216)
  | at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:411)
  | at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:407)
  | at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:255)
  | at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:467)
  | at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:238)
  | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:295)
  | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
  | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
  | at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
  | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
  | at org.elasticsearch.cli.Command.main(Command.java:90)
  | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
  | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
  | Refer to the log for complete error details.

It will be great to have a settings for keystore location in elasticsearch.yaml to set directory outside of /etc/elasticsearch.

Can you advice in this topic?

Thanks in advance,

pdanysz on 18 Mar 2019

👍5

I'm labeling this team-discuss to determine which of the as-yet-incomplete "nice to have" tasks we still intend to complete. Once we do so, they should be promoted to individual issues and this issue closed, as the main thrust of this issue has been long since completed.