Streams ยท Kafka

One bootstrap endpoint, a real Kafka cluster.

Excloud Streams provisions a managed Apache Kafka cluster in KRaft mode, replication factor 3, min.insync.replicas 2. Clients connect to a single TLS endpoint over SASL_SSL with SCRAM-SHA-512 โ€” no broker addresses to manage, no ZooKeeper.

$ exc stream create --name events-prod --instance_type m1a.large

exc โ€” terminal
$ exc stream create --name events-prod \
    --zone_id 1 --subnet_id 1 \
    --instance_type m1a.large \
    --root_volume_size_gib 100 \
    --allowed_cidrs 203.0.113.0/24 --wait
โœ“ running โ€” events-prod.stream.excloud.dev:9092
$ exc stream bootstrap --id 12
security_protocol: SASL_SSL ยท sasl_mechanism: SCRAM-SHA-512

Pricing

The cluster bills at the same rates as a plain compute instance.

A Streams cluster runs on the m1a instance type you pick, plus a root volume โ€” billed at exactly the compute and block storage rates already on this site. Only m1a sizes are supported, so a cluster always has dedicated cores.

Full compute pricing

InstancevCPURAMRate
m1a.large 2 8 GiB โ‚น1.889/hr
m1a.xlarge 4 16 GiB โ‚น3.778/hr
m1a.2xlarge 8 32 GiB โ‚น7.556/hr

Plus the root volume at โ‚น4/GBยทmo. A 100 GB volume on m1a.large runs โ‚น1.889/hr + โ‚น400/mo.

Topics

Desired state, applied through the Admin API.

Every topic command applies the change to Kafka and records the desired state, so the topic survives broker restarts. Partition counts only go up โ€” Kafka rejects a decrease, so the CLI does too.

  • Default retention is 7 days (604800000 ms), cleanup policy delete
  • cleanup_policy: compact keeps only the latest record per key
  • Replication factor defaults to 3, bounded by the cluster's replica count
exc โ€” terminal
$ exc stream topic create --id 12 --name orders \
    --partitions 12 --retention_ms 604800000
$ exc stream topic patch --id 12 --name orders \
    --partitions 24 --cleanup_policy compact
$ exc stream topic list --id 12
orders  partitions=24  rf=3  cleanup=compact

Users & ACLs

Give each application its own SCRAM user.

Every cluster comes with one admin SCRAM credential, shown once at create time. For applications, create separate SCRAM users and grant only the Kafka ACLs they need โ€” READ on the topics they consume, WRITE on the ones they produce to, nothing more.

  • Principals use the User:<username> form
  • Rotating a user's password requires updating the app's secret before restart
  • Deleting a user immediately fails authentication for clients using it
exc โ€” terminal
$ exc stream user create --id 12 --username orders-app
password (shown once): ...
$ exc stream acl create --id 12 \
    --principal User:orders-app \
    --resource_type TOPIC --resource_name orders \
    --operation READ --pattern_type LITERAL \
    --permission_type ALLOW --host '*'

Get started

The bootstrap endpoint is the only address your clients need.

Create a cluster, create a topic, create a SCRAM user, run the console producer. If the consumer gets the message back, the endpoint, the credentials, and the ACL are all correct.

$ exc stream create --name events-prod --instance_type m1a.large