Skip to content

Fix the handling of subscriptions with multiple topics and extra fields#2425

Merged
thjaeckle merged 2 commits into
eclipse-ditto:masterfrom
boschglobal:bugfix/multple-topics-per-target
May 12, 2026
Merged

Fix the handling of subscriptions with multiple topics and extra fields#2425
thjaeckle merged 2 commits into
eclipse-ditto:masterfrom
boschglobal:bugfix/multple-topics-per-target

Conversation

@abalarev
Copy link
Copy Markdown
Contributor

@abalarev abalarev commented Apr 24, 2026

Summary
Resolves incorrect handling of subscriptions for multiple topics, containing at least one extra field.

Issue

  • multiple topics without extraFields works ok - sends to the matching ones
  • having multiple topics with extraFields works for only one of them (when it's matching), but not deterministic for which one
  • having a topic or topics with extraFields and some without extraFields works again only for one of the topics with extraFields (when it's matching and not for each topic), but for none of the topics without extraFields

What's changed
OutboundMaooingProcessorActor flow changed as follows:

  • retrieving of extra fields for all topics (per target) combined in a single request
  • the filter is now applied using already retrieved extra fields as it needs them to filter correctly. Only those extra fields defined in the matching filter are left in the outbound signal
  • splitting of targets with and without extra fields is now removed as it became redundant
  • the new flow is still optimized for targets with no extra fields in the topics.
  • outbound signal ordering is now changed, therefore the order of unit tests expectations is also changed
  • more unit tests are created to verify the handling of the problem cases described above

Added system tests: eclipse-ditto/ditto-testing#33

@abalarev abalarev changed the title Handling of subscriptions with multiple topics with some extraFields [WIP] Handling of subscriptions with multiple topics with some extraFields Apr 27, 2026
@abalarev abalarev force-pushed the bugfix/multple-topics-per-target branch from f26bbaf to 01fd8d8 Compare April 27, 2026 08:52
@abalarev abalarev changed the title [WIP] Handling of subscriptions with multiple topics with some extraFields Handling of subscriptions with multiple topics with some extraFields Apr 27, 2026
@abalarev abalarev changed the title Handling of subscriptions with multiple topics with some extraFields Handling of subscriptions with multiple topics with extraFields Apr 27, 2026
@abalarev abalarev changed the title Handling of subscriptions with multiple topics with extraFields Handling of subscriptions with multiple topics with extra fields Apr 27, 2026
@abalarev abalarev changed the title Handling of subscriptions with multiple topics with extra fields Fix the handling of subscriptions with multiple topics with extra fields Apr 27, 2026
@thjaeckle
Copy link
Copy Markdown
Member

@abalarev will you also provide a system-test which tests this fix - seems important enough to me to cover it with an integration-test

@thjaeckle thjaeckle added this to the 3.9.0 milestone Apr 29, 2026
@thjaeckle thjaeckle added the bug label Apr 29, 2026
@abalarev
Copy link
Copy Markdown
Contributor Author

@thjaeckle Sounds reasonable, I'll add some system test(s)

@abalarev abalarev changed the title Fix the handling of subscriptions with multiple topics with extra fields Fix the handling of subscriptions with multiple topics and extra fields Apr 29, 2026
@thjaeckle
Copy link
Copy Markdown
Member

@abalarev thanks for the added system tests 👍
I just ran them and seems there are some related test failures for MQTT, could you check them please?
https://github.com/eclipse-ditto/ditto/actions/runs/25307537926

@abalarev
Copy link
Copy Markdown
Contributor Author

abalarev commented May 4, 2026

@thjaeckle Please run the system tests again, I've pushed a fix.

@abalarev abalarev force-pushed the bugfix/multple-topics-per-target branch from 01fd8d8 to 340ea56 Compare May 5, 2026 08:55
Copy link
Copy Markdown
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this — the refactor direction looks right and the headline scenarios are well-covered. A few correctness concerns and polish items inline.

One overarching note on test coverage: the four correctness concerns flagged below (shared-root trimming, cross-streaming-type early return, non-deterministic findFirst over a Set, and catch-all / extras precedence) are not yet exercised by the new tests. Once the intended behavior for each is settled, please add regression tests that lock it in.

Signed-off-by: Andrey Balarev <andrey.balarev@bosch.com>
@abalarev abalarev force-pushed the bugfix/multple-topics-per-target branch 4 times, most recently from b6b2921 to a2c6d48 Compare May 11, 2026 14:41
@abalarev
Copy link
Copy Markdown
Contributor Author

@thjaeckle Thank you for the review findings and comments!
I've fixed the issues, added and updated the unit tests to cover the changes.
Please review again!

@abalarev abalarev requested a review from thjaeckle May 11, 2026 15:09
@thjaeckle
Copy link
Copy Markdown
Member

@abalarev one thread is still unresolved - the other changes look good 👍

Signed-off-by: Andrey Balarev <andrey.balarev@bosch.com>
@abalarev abalarev force-pushed the bugfix/multple-topics-per-target branch from a2c6d48 to f0ccc9d Compare May 12, 2026 06:54
Copy link
Copy Markdown
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @abalarev — all the items from the previous round are addressed and the new tests do a good job of locking in the intended behavior.
The refactor reads cleaner than the previous shape.

LGTM.

@thjaeckle thjaeckle merged commit 469bc5d into eclipse-ditto:master May 12, 2026
3 checks passed
@abalarev abalarev deleted the bugfix/multple-topics-per-target branch May 12, 2026 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants