Elasticsearch SSO Implementation
Craig Pickles, 23 August 2021
Country: Australia
Akkodis has implemented Amazon Elasticsearch to aggregate logs and provide a centralised location for monitoring and troubleshooting. The use of the managed service reduces operational overheads but also leverages the tight integration with other AWS services.
The Scenario
A state government customer has a number of microservices and applications that are logging to individual Cloud Watch log streams and investigating incidents or tracing bugs across dependencies is challenging and time consuming.
Akkodis has implemented Amazon Elasticsearch to aggregate logs and provide a centralised location for monitoring and troubleshooting. The use of the managed service reduces operational overheads but also leverages the tight integration with other AWS services.
A key requirement is to securely authenticate to Elasticsearch and Kibana using Single Sign On, rather than using basic authentication and the administrative burden of managing a separate user database. This also permissions for accessing indices and tenants in Kibana that can then be mapped to existing roles within the organisation.
Akkodis designed and implemented this solution in early 2021.
The Solution
Akkodis implemented Amazon Elasticsearch managed service in conjunction with a number of other AWS services for log ingestion. To provide a Single Sign On capability, AWS SSO and Fine-grained Access Control with SAML Authentication was implemented in Amazon Elasticsearch. This solution uses existing Active Directory groups and maps those to fine grained permissions in the Elasticsearch domain.
The architecture is illustrated as follows:
In AWS SSO, Applications for Kibana are configured per environment and the SAML metadata for these applications are exported and stored in Secrets Manger for use during deployment. The steps are somewhat manual due to the limited functionality available in the CLI for the AWS SSO service.
The Applications are configured using the Idp-initiated URLs for each Elasticsearch domain, which allows the SSO landing page to be used as the point of entry for users.
| Attribute | Value |
|---|---|
| Application ACS URL | https://domain_endpoint/_plugin/kibana/_opendistro/_security/saml/acs/idpinitiated |
| Application SAML audience | https://domain_endpoint |
A specific set of mappings are defined to enable username, email address and Active Directory groups to be included in the SAML assertion.
A specific set of mappings are defined to enable username, email address and Active Directory groups to be included in the SAML assertion.
The Elasticsearch domain has VPC access enabled and is therefore only accessible from within the VPC and over Direct Connect. As such there is less emphasis on the Access Policy, which is permissive, with the focus instead on the use of Fine-Grained Access control and SAML authentication. SAML authentication imports the metadata from AWS SSO and the backend roles are mapped from an attribute in the SAML assertion:
The roles in AWS SSO are mapped as “backend roles” for the required Elasticsearch roles:
| Backend Role | Elasticsearch Role/s |
|---|---|
| Developer | read_only, kibana_user |
| Administrator | security_manager, kibana_user, admin_access |
| Support | read_only, kibana_user |
| Release | read_only, kibana_user |
Configuration of authentication in the Elasticsearch domain is done using the REST API, as are Tenants, visualizations, dashboards and the roles and permission that enable users to access them. Once of the principles of the solution was to automate as much of deployment as possible and security is no exception.
It is worthwhile noting that a current limitation using AWS SSO is that the groups mapped through as part of the SAML assertion is a UUID, rather than a named group. While this does not impact functionality, the use of ids is not as clean as a named role. As the AWS SSO managed service matures, there is an expectation that this will change, and the solution will be refined.
The Outcome
The Single Sign-On landing page is used to access Kibana in all environments, in addition to AWS accounts. The landing pages provides a centralized location for which to access all SSO integrated workloads, which is likely to be extended in the future.
The application in AWS SSO redirects to the organisation login page and after successful authentication, the user is logged into Kibana.
The roles for the logged in user present the assigned permissions, and associated backend roles.
This solution provides the customer with a managed service that will be maintained over time.