Skip to content

Supporting compacted event types (#154)#158

Closed
ePaul wants to merge 8 commits intomasterfrom
#154-support-compaction-key
Closed

Supporting compacted event types (#154)#158
ePaul wants to merge 8 commits intomasterfrom
#154-support-compaction-key

Conversation

@ePaul
Copy link
Copy Markdown
Member

@ePaul ePaul commented Nov 26, 2022

This is implementing #154.

  • add tests
  • check whether/how this can be supported for snapshot sending
    • → this will work out of the box, if the data objects for the snapshots are the same type as the normal objects.
  • add/update documentation
  • (maybe) bump version number (if this doesn't get into 21.0.0)
  • Do we want a "key extractor as a bean" feature similarly to the snapshot generators? → support compacted event types (#154) via spring beans #164
  • after #142 Update flyway and spring boot #143 is merged, adjust all the imports in the tests (and possibly other code as well).

@ePaul ePaul added enhancement nakadi-submission spring-boot-2 Issues/PRs which only apply to the Spring-Boot 2 versions (Releases 20.*+) persistence everything around DB access labels Nov 26, 2022
@ePaul ePaul force-pushed the #154-support-compaction-key branch from f706ec7 to 0d53fa6 Compare November 30, 2022 18:23
@ePaul ePaul changed the title start of supporting a compaction key (#154) Supporting compacted event types (#154) Nov 30, 2022
@ePaul ePaul marked this pull request as ready for review November 30, 2022 22:21
@ePaul ePaul requested a review from fbrns as a code owner November 30, 2022 22:21
* @param dataType the Java type of the objects for which the extractor is used.
* @param extractor a function which will extract the compaction key from an object.
*/
<X> void registerCompactionKeyExtractor(String eventType, Class<X> dataType, CompactionKeyExtractor<X> extractor);
Copy link
Copy Markdown
Member Author

@ePaul ePaul Nov 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the snapshot triggering mechanism, we have the SnapshotEventGenerator interface, which an application needs to implement and provide as spring beans.

Here instead, the CompactionKeyExtractor (+ event-Type + Class) has to be passed to the registerCompactionKeyExtractor method on the EventLogWriter.

This can be seen as somewhat inconsistent. Should we try to do the same here?

  • We get a wrapper class for these three things.
  • The EventLogWriterImpl would get a list of these as constructor parameter.
  • The Autoconfiguration collects beans of this type to inject them into the ELWImpl.

Then we can get rid of this register-method completely, and the interface of EventLogWriter would not change at all.

* @see EventLogWriter#registerCompactionKeyExtractor(String, Class, CompactionKeyExtractor)
*/
@FunctionalInterface
public interface CompactionKeyExtractor<X> {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a glorified Function<X, String>.

- change new files to JUnit 5
- replace @parameterized by subclasses
@ePaul ePaul force-pushed the #154-support-compaction-key branch from bf56602 to 5b03d6c Compare December 6, 2022 21:27
Comment on lines +40 to +42
@Captor
private ArgumentCaptor<EventLog> eventLogCapture;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused variable.

Suggested change
@Captor
private ArgumentCaptor<EventLog> eventLogCapture;

@ePaul
Copy link
Copy Markdown
Member Author

ePaul commented Dec 16, 2022

Replaced by #164.

@ePaul ePaul closed this Dec 16, 2022
@ePaul ePaul deleted the #154-support-compaction-key branch August 10, 2023 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement nakadi-submission persistence everything around DB access spring-boot-2 Issues/PRs which only apply to the Spring-Boot 2 versions (Releases 20.*+)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant