Guided scenario
Open-source streaming & messaging
Teams building event-driven systems often want to own their event backbone rather than lock into a proprietary cloud bus. This shortlist ranks streaming platforms by open-source licensing, a real self-hosting option, and Kafka compatibility so existing producers and consumers work unchanged. Also weigh throughput and retention guarantees and the operational burden of running it yourself.
Amazon Kinesis
aws.amazon.com/kinesis
Amazon Kinesis is AWS's service for collecting, processing, and analyzing real-time streaming data, including Data Streams and Firehose.
Amazon MSK
aws.amazon.com/msk
Amazon MSK is AWS's managed service for Apache Kafka, handling provisioning, patching, and operations so teams can run Kafka on AWS without managing clusters.
Azure Event Hubs
azure.microsoft.com/en-us/products/event-hubs
Azure Event Hubs is Microsoft's managed event ingestion service for streaming large volumes of events, targeting teams building streaming pipelines on Azure.
Confluent Cloud
www.confluent.io
Confluent Cloud is a managed service for Apache Kafka, built by Kafka's original creators, providing event streaming with added tooling and connectors.
Google Pub/Sub
cloud.google.com/pubsub
Google Pub/Sub is Google Cloud's fully managed, serverless messaging service for ingesting and delivering event streams at scale.
Redpanda
redpanda.com
Redpanda is a Kafka-compatible streaming data platform written in C++ that runs without a JVM or ZooKeeper, aiming for high performance and simpler operations.
Pairs well with
Tools commonly used alongside this setup. Chosen editorially, not ranked.
Amazon DynamoDB
Managed Databases
Amazon DynamoDB is AWS's fully managed, serverless NoSQL key-value and document database designed for high throughput and single-digit-millisecond latency at scale. It targets teams building high-traffic applications on AWS that want predictable performance without managing servers. Capacity can be on-demand or provisioned, making pricing usage based, and an always-free allowance covers 25 GB of storage with limited capacity. The service is proprietary and AWS-only, so it is not open-source, self-hostable in production, or multi-cloud. It holds SOC2, ISO27001, FedRAMP, HIPAA, and PCI-DSS compliance.
Airbyte
ETL / Data Pipelines
Airbyte is an open-source data integration platform with one of the largest connector catalogs, available both self-hosted and as a managed cloud service. It targets data teams that want open, extensible pipelines following an ELT model with change data capture (CDC) support. The platform is open-source and free to self-host, while Airbyte Cloud is a managed option billed by credit and usage, with a trial available. Its broad catalog includes many community-maintained connectors, giving teams flexibility in sources and destinations. Airbyte carries SOC2 and ISO27001 compliance.
Chronosphere
Monitoring & APM
Chronosphere is an observability platform built for cloud-native and Kubernetes environments, with a particular focus on controlling the cost of metrics data at scale. Founded in 2019 and compatible with Prometheus and OpenTelemetry, it covers APM, infrastructure monitoring, and log management for teams operating very large deployments. Pricing is enterprise and quote-based with no free tier, placing it in the premium tier. While its origins trace to the open-source M3 project, the platform itself is proprietary and closed-source, and it holds SOC2 and ISO27001 compliance.