On this page
An online reservation service stays available after a clinic closes. The clinic’s database may not. At SmartDoctor, each hospital had its own database host and connection, and many databases were shut down at the end of the working day.
That made “write the reservation to the hospital database” an availability-dependent operation. The platform needed to retain the booking and deliver it when the destination could accept work.
A database fleet changes the problem
I worked in an environment with more than 1,000 hospital database hosts, based on my recollection of the operational scope. A shared server could not assume every destination was reachable or keep every connection pool alive indefinitely.
The hospital module resolved hospital identity from request context and selected the corresponding SQL Server datasource. The reviewed implementation created Hikari pools on demand, expired inactive cached pools, closed removed pools and invalidated selected failed connection configurations before looking them up again.
Those mechanisms made connection lifecycle an explicit part of application behavior. They do not, by themselves, prove security isolation or that every resource-exhaustion case was solved.
Keep booking acceptance separate from delivery
The reservation platform retained integration work in an outbox. A Debezium connector captured inserts from that MySQL outbox; the Kafka consumer dispatched reservation messages to SQS or made the specified integration request.
SQS message groups carried hospital identity. The hospital-side consumer established the matching context, applied create or update work, and deleted the message after successful handling.
This was the concrete reason for CDC in this system: propagate recorded reservation integration work across a boundary where the target database could be unavailable. It was not an attempt to stream every hospital SQL Server database into Kafka.
An offline destination needs different handling
In the reviewed consumer, classified database-offline errors extended message visibility by 1,800 seconds. Other failures followed a shorter retry path and could move to a dead-letter queue after repeated receives, with diagnostic notification.
The separation mattered. A closed clinic was an expected availability condition; a malformed or persistently failing operation needed investigation. Treating both as identical failures would obscure what the operator should do next.
The original product requirement was to synchronize reservations promptly when a hospital returned. The retained implementation shows availability-aware deferred delivery, not a database-startup event or guaranteed immediate insertion. Queue visibility, worker availability and processing time all affect how soon a waiting reservation becomes visible at the clinic.
Delivery still needs a correctness contract
The producer used a hospital message group and a create-message deduplication identity. Those are useful controls, but they do not establish permanent exactly-once database effects. A downstream effect may complete before a consumer acknowledgment fails, leaving work eligible for replay.
The architectural result was a retained path from platform-side integration intent to hospital-side execution, with explicit behavior for offline destinations. I do not claim a newly reproduced delivery-latency percentile or zero-loss result from this source review.
Keep the Kafka deployment runnable during an ARM migration
The delivery pipeline also constrained a cluster cost reduction. I changed application builds for Graviton, but could not get the Kafka connector running successfully in the attempted ARM setup. I kept Kafka workloads on a separate x86 node group and moved other workloads to Graviton.
The recorded node configuration changed from seven m6i.xlarge instances at $1.344/hour to three r7g.xlarge and four t3a.medium instances at $0.793/hour, a reduction of about 41%. These are historical node estimates, not verified bills or a company-wide saving. Keeping the connector environment on x86 allowed the rest of that cost reduction to proceed without requiring every workload to move architectures together.
Operations follow the same hospital boundary
I also created scheduled database-fleet checks for fragmentation, expensive queries, CPU and memory, including twice-daily fragmentation inspection. The exact maintenance script and thresholds are not in the reviewed snapshot. I therefore describe the operational responsibility without attributing an unrecovered REBUILD, REORGANIZE or recompilation policy to it.
The central design question was practical: how does a service keep accepting useful work when the system that must eventually receive it has a different working day? The Strimzi and outbox CDC implementation and hospital-group routing with Istio explain the supporting infrastructure.