Topic: kinesis-firehose
- Drowning in Logs, Starving for Answers
A production Kubernetes cluster emitted 25 million structured log lines and 62 GB a day, but no one could answer 'which service is failing right now?' with a query. Here is the pipeline that turned that firehose of JSON into columnar Parquet you can hit with plain SQL, and the design decisions behind it.