Skip to content

NC | lifecycle | add expire delete marker rule #8961

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

nadavMiz
Copy link
Contributor

@nadavMiz nadavMiz commented Apr 9, 2025

Describe the Problem

added expire delete marker rule. see https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html#lifecycle-config-conceptual-ex7

Explain the Changes

  1. add expire delete marker rule
  2. in case last item is a delete marker, change the state to start from this delete marker in the next run

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

  1. sudo npx jest test_nc_lifecycle_cli
  • Doc added/updated
  • Tests added

@nadavMiz nadavMiz requested a review from romayalon April 9, 2025 14:15
@nadavMiz nadavMiz self-assigned this Apr 9, 2025
@nadavMiz nadavMiz force-pushed the lifecycle-delete-marker-expiration1 branch 2 times, most recently from b000007 to ec6cc47 Compare April 10, 2025 06:50
versions_list = await this.load_versions_list(object_sdk, bucket_json, rule_state);
}
const candidates = [];
const filter_func = this._build_lifecycle_filter({filter: lifecycle_rule.filter, expiration: 0});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when having an expired delete marker we don't care if there is also expiration by days/date?

Copy link
Contributor Author

@nadavMiz nadavMiz Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its a separate rule than both of them. also I think you can't even put expired delete marker and days/date together

Copy link
Contributor

@romayalon romayalon Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm IIRC if days is set then automatically expired delete marker = true, right?
does it mean that the expired delete marker is on objects that passed the expiration days?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to aws:

You can't specify both a Days and an ExpiredObjectDeleteMarker tag on the same rule. When you specify the Days tag, Amazon S3 automatically performs ExpiredObjectDeleteMarker cleanup when the delete markers are old enough to satisfy the age criteria. To clean up delete markers as soon as they become the only version, create a separate rule with only the ExpiredObjectDeleteMarker tag.

so I guess you are correct. I'll change this

Copy link
Contributor

@romayalon romayalon Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and expire date + expire delete marker based on aws?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to put date and expire delete marker together returns MalformedXML error. there is nothing on AWS documentation about about implicit ExpiredObjectDeleteMarker when having date rule. so I guess we can't have both of them together

@nadavMiz nadavMiz force-pushed the lifecycle-delete-marker-expiration1 branch 3 times, most recently from 7f30b55 to f0d565d Compare April 10, 2025 12:16
@nadavMiz nadavMiz force-pushed the lifecycle-delete-marker-expiration1 branch from f0d565d to fcfff6c Compare April 22, 2025 08:11
@nadavMiz nadavMiz force-pushed the lifecycle-delete-marker-expiration1 branch from fcfff6c to 2cb1c4d Compare April 22, 2025 08:52
@nadavMiz nadavMiz force-pushed the lifecycle-delete-marker-expiration1 branch from 2cb1c4d to f216a3b Compare April 22, 2025 08:53
@nadavMiz nadavMiz requested a review from romayalon April 22, 2025 08:54
@nadavMiz nadavMiz merged commit be61cd9 into noobaa:master Apr 22, 2025
11 of 12 checks passed
@romayalon romayalon mentioned this pull request May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants