[PL-128135] Add s3 gateway on rgw servers #1701
Conversation
Ma27
force-pushed
the
PL-128135-s3gw
branch
2 times, most recently
from
2300e39 to
b5322e1
Compare
|
|
||
| cephPkgs = fclib.ceph.mkPkgs role.cephRelease; | ||
|
|
||
| radosListenPort = toString 7480; |
There was a problem hiding this comment.
I think we need to leave 80 in here for pre-signed URLs?
There was a problem hiding this comment.
Do you happen to have a reproducer for the old issue? Because I cannot reproduce on dev (and the tests are fine either):
$ cat .aws/credentials
[dev2]
endpoint_url = https://objects.dev.fcio.net
<redacted>
$ aws --profile dev2 s3 ls s3://test/
[...]
2025-07-21 13:21:18 12 testfile
$ aws --profile dev2 s3 presign s3://test/testfile
https://objects.dev.fcio.net/test/testfile?AWSAccessKeyId=03KAF2L1WU7PHIH0H66M&Signature=Oh1zNWA0TnHbdNCPerhlEEG%2FqHU%3D&Expires=1755084588
$ curl "https://objects.dev.fcio.net/test/testfile?AWSAccessKeyId=03KAF2L1WU7PHIH0H66M&Signature=Oh1zNWA0TnHbdNCPerhlEEG%2FqHU%3D&Expires=1755084588"
Hello World
ctheune
left a comment
ctheune
left a comment
There was a problem hiding this comment.
Definitely on the right track, see the comments.
| rgwMimeTypesFile = "${pkgs.mime-types}/etc/mime.types"; | ||
| debugRados = "1 5"; | ||
| rgwFrontends = "beast port=80"; | ||
| rgwFrontends = "beast port=${radosListenPort}"; |
There was a problem hiding this comment.
please bind explicity to srv (but also keep the firewall!)
Sets the listening address in the form address[:port], where the address is an IPv4 address string in dotted decimal form, or an IPv6 address in hexadecimal notation surrounded by square brackets. Specifying a IPv6 endpoint would listen to v6 only. The optional port defaults to 80 for endpoint and 443 for ssl_endpoint. Can be specified multiple times as in endpoint=[::1] endpoint=192.168.0.100:8000.
| sto = fclib.network.sto; | ||
| in | ||
| lib.mkMerge [ | ||
| (lib.mkOrder 700 '' |
There was a problem hiding this comment.
those might be needed if we have to enable the port redirections again
Ma27
force-pushed
the
PL-128135-s3gw
branch
2 times, most recently
from
9ffbaac to
49fd404
Compare
PL-128135 We no longer let radosgw listen on port 80, but on port 7480 instead and let the reverse proxy in front listen on port 80. The haproxy in between uses other rgws belonging to the same cluster as backup servers. TLS termination is done with nginx. The ACME challenge is DNS-01 since we usually have multiple RGW instances.
…& GC old data PL-128135
PL-128135 Upon working on PL-128135, I encountered a case where in the early setup either 64 or 320 pgs were present breaking the test. Since this is timing-related, only check that _at least_ 64 pgs are free.
@flyingcircusio/release-managers
Release process
./changelog.shPR release workflow (internal)
Design notes
onoroff. Example: rate limiting.Security implications