Effective Kafka Load Testing: Beyond Throughput Metrics

An order API can accept a request, publish to Kafka and return success while the fulfilment worker is still waiting on a database. A producer throughput result cannot tell you when the order became usable. For this workload, define completion as the orders.completed event emitted after the required application work finishes.

LoadStrike brings that completion check into a code-first Kafka load test. Keep the order ID across the input and result, let the test match both sides, and review the outcome as pressure increases. The feature is useful for evaluating event-driven services because it connects the generated demand to an observable business result.

Follow an order to its completion event

For an illustrative workflow, the input record carries trackingId = order-1001 and tenantId = tenant-a. After the application completes the operation, the result record carries the same trackingId and tenantId plus status = completed. These are example payload values, not a recorded customer transaction or an executed benchmark.

Configure TrackingField as json:$.trackingId on both endpoints and, if needed, GatherByField as json:$.tenantId on the destination. JSON property names are exact-case. Give every intended operation a unique ID so a retry or reused test ID cannot be mistaken for a new business operation.

LoadStrike matches the selected ID; the match alone does not validate every property in the payload. Select a topic whose event contract represents successful completion, or add an application-specific check for success. An event emitted when work is merely queued is evidence of acceptance.

• Choose the start boundary: a generated Kafka input, an API request, or an existing event being observed.
• Choose the completion boundary: the result emitted after the operation you want to measure.
• Use a dedicated observer consumer group and verify one smoke completion is visible before increasing load.

Read consumer lag beside matched completion latency

Collect broker records per second and bytes per second, plus the application consumer metrics, alongside the LoadStrike report. The native Kafka endpoint publishes or consumes records; endpoint support does not automatically make consumer lag, broker CPU or every Kafka administration metric a LoadStrike measurement.

Apache Kafka documents records-lag-max as the maximum partition lag in records, based on the current consumer offset rather than the committed offset. Record which consumer and lag definition you used. The LoadStrike observer group and the application worker group have different jobs and can have different lag.

Read the correlation clock boundary as well. In the C# Kafka path, source time is recorded after successful delivery and destination time uses the Kafka record timestamp, with observation time as a fallback. Clock alignment and topic timestamp configuration affect interpretation. This interval can differ from a client stopwatch or the time a worker spends executing code.

• Rising application lag with rising completion latency is a reason to inspect the consumer path, partition distribution and downstream dependencies.
• Low lag with slow completion is a reason to inspect work performed after consumption.
• Fast matched latency with more timeouts can mean the report only describes the work that finished.
• A configured scenario rate is offered demand; confirm achieved invocations and actual Kafka record counts before comparing capacity.

Interpret timeouts, duplicate IDs and grouped outcomes

A timeout says the configured window ended without a matching completion being recorded. It does not establish that Kafka permanently lost the record. Check the selector, observer offsets and coverage, completion contract and downstream delay. An unmatched destination has no matching source in the correlation context; retained traffic or activity outside the observation window can contribute.

Duplicate source and destination counters expose repeated tracking values. Investigate application retries, replayed Kafka history and accidentally reused test IDs. These counters are useful evidence for a load test, but they are not a guarantee about exactly-once database updates or external side effects.

Worked arithmetic example with invented values: if a run had 100 unique tracked operations, 98 matched completions and two timeouts after its correlation windows were resolved, the matched proportion would be 98/100 = 98%. Three extra destination observations with repeated IDs would not raise that to 101 completed operations. This is an explanation of the calculation, not a LoadStrike benchmark result; apply it only when your real denominator and observation coverage are established.

• Report the matched proportion and timeout count beside matched latency percentiles.
• Keep unmatched destinations and duplicate observations separate from unique completed operations.
• Use GatherByField to inspect tenant or event-type groups; retain the sample count for each group.
• A grouped latency summary describes observed outcomes and does not guarantee atomicity or consistency across services.

Make a Kafka performance result reviewable

A useful result states what was completed, where the timestamps came from, how much load actually started and how much source activity had a matching outcome. Keep the application build, SDK version, payload distribution, partitions, consumer groups, offset settings, credentials scope and timeout fixed when comparing runs.

Retain the LoadStrike report artifacts and the Kafka monitoring export for the same test window. Repeat the same workload before claiming a change improved completion. If the generator missed scheduled arrivals, state that limitation and investigate it before assigning the limit to Kafka.

To evaluate LoadStrike, start with one result topic and one business ID in an environment you own or are authorized to test. Use the linked Kafka setup guide to build the smoke check, then compare completion, timeouts and duplicate visibility under a controlled load profile. Native Kafka testing requires a valid runner key on a plan that includes it.

• Include offered and achieved load, unique source operations, matched completions and unresolved or timed-out work.
• Publish the report period, load phases, repeat count and timestamp definition with the result.
• Add measured findings only after a run; this article makes no throughput, latency or customer-result claim.