DDC: AWS OpenSearch Centralized Logging System with Terraform

Introduction

This document provides an overview of the DDC implementation of a centralized logging system using AWS OpenSearch. The system is deployed using Terraform code, and it encompasses the collection and storage of various logs, such as application logs, CloudTrail logs, VPC flow logs, and RDS audit logs. Additionally, the OpenSearch cluster is configured in a secure private network accessible only via VPN and integrated with the Roche Single Sign-On (SSO) for authentication.

Deployment of AWS OpenSearch using Terraform

DDC has chosen AWS OpenSearch to centralize and analyze logs from various sources. The deployment of OpenSearch is achieved through Terraform infrastructure-as-code (IAC), ensuring consistency and repeatability in the deployment process.

Log Data Collection

The central objective of this centralized logging system is to aggregate logs from multiple sources, including:

Application Logs: Log data generated by DDC’s applications. CloudTrail Logs: AWS CloudTrail logs for tracking AWS API activity. VPC Flow Logs: Logs that record network traffic within Virtual Private Clouds. RDS Audit Logs: Audit logs from Amazon RDS databases.

Data Ingestion via Lambda Function

To facilitate the movement of log data into the AWS OpenSearch cluster, Lambda functions have been employed. These serverless functions act as data pipelines, ingesting log data and forwarding it to the appropriate OpenSearch index for further analysis and visualization.

Private Network Configuration

To ensure the security of log data and the OpenSearch cluster, the cluster is configured within a private network. This private network is designed to allow access only via a Virtual Private Network (VPN). This strict access control is essential for protecting sensitive log data from unauthorized access.

Integration with Roche SSO

To enhance authentication and access control, the AWS OpenSearch cluster is integrated with the Roche Single Sign-On (SSO) system. This integration ensures that only authorized users with the appropriate credentials can access the OpenSearch domain. Roche SSO provides an additional layer of security for the centralized logging system.

Conclusion

DDC’s implementation of AWS OpenSearch for centralized logging is a robust and secure solution for aggregating and analyzing log data from multiple sources. The use of Terraform for deployment, integration with VPN for access control, and Roche SSO for authentication collectively contribute to a comprehensive and secure centralized logging system. This system empowers DDC to monitor and respond to log data efficiently and safeguard sensitive information.