The most appropriate solution for a bank that is utilizing Amazon MSK to feed real-time data into a data lake and is experiencing an almost full storage capacity is:
- ✅ Proactive Monitoring: Amazon CloudWatch allows you to monitor the KafkaDataLogsDiskUsed metric, which would help identify when storage is nearing its limit.
- ✅ Automated Disk Management: By setting up to automatically flush out the oldest messages when disk usage exceeds 85%, you can manage disk space effectively, preventing the cluster from running out of space.
-
🔴 Now, let's examine why the other options are not the best choice:
-
❌ A. Use the Amazon MSK console to triple the broker storage and restart the cluster: Tripling the broker storage would not address the underlying issue of managing the disk space effectively and would also result in unnecessary costs.
-
❌ C. Create a custom Amazon MSK configuration. Set the log retention hours parameter to 48. Update the cluster with the new configuration file: Reducing the log retention hours to 48 may not suffice for the requirement of data being accessible within 24 hours, and could potentially lead to loss of data.
-
❌ D. Triple the number of consumers to ensure that data is consumed as soon as it is added to a topic: The number of consumers does not directly influence the storage usage in an Amazon MSK cluster. Moreover, tripling the consumers might introduce unnecessary cost and complexity.
