Skip to content

STORM-4041: fix(topology_lag): Kafka Topology Lag breaking when no offsets are committed (Backport #8589)#8591

Merged
reiabreu merged 1 commit into2.xfrom
backport-8589-2.x
May 9, 2026
Merged

STORM-4041: fix(topology_lag): Kafka Topology Lag breaking when no offsets are committed (Backport #8589)#8591
reiabreu merged 1 commit into2.xfrom
backport-8589-2.x

Conversation

@reiabreu
Copy link
Copy Markdown
Contributor

@reiabreu reiabreu commented May 8, 2026

What is the purpose of the change

This is a backport of #8589 to the 2.x branch.

It fixes a regression where the Kafka spout lag reporting in the Storm UI would fail silently (throwing a ClassCastException in logs) when partitions had no committed offsets.

Brief change log

  • KafkaOffsetLagUtil: Optimized offset retrieval using batch calls and added null-checks to handle partitions without committed offsets (reporting -1 instead of crashing).
  • TopologySpoutLag: Improved JSON parsing logic to safely handle non-JSON error messages from the monitor subprocess, preventing ClassCastException.
  • Integration Test: Added KafkaOffsetLagUtilTest using Testcontainers to verify the fix.

Related Issues

Verifying this change

  • Added KafkaOffsetLagUtilTest which passes locally on the 2.x branch.
  • Manually verified compilation of storm-kafka-monitor.

@reiabreu reiabreu added this to the 2.8.8 milestone May 8, 2026
@reiabreu reiabreu closed this May 8, 2026
@reiabreu reiabreu reopened this May 8, 2026
@reiabreu reiabreu requested a review from rzo1 May 8, 2026 22:54
@reiabreu reiabreu merged commit 5126b49 into 2.x May 9, 2026
16 checks passed
@rzo1 rzo1 deleted the backport-8589-2.x branch May 10, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant