Code of Conduct
Search before asking
Describe the feature
Add a configuration to control whether the built-in JSON event logger actively manages event log directory and file permissions.
Proposed configuration:
kyuubi.event.json.log.manage.permissions.enabled=true
The default value should be true to keep the existing behavior unchanged.
When set to false, the JSON event logger should still create and write event log paths, but it should not actively set fixed permissions on event log directories or files.
Motivation
In some production deployments, event log directories are not managed only by POSIX-style HDFS permissions. They may be governed by Apache Ranger, inherited HDFS ACLs, default ACL inheritance, or other external authorization systems.
In these environments, the engine user may have enough permission to create and write JSON event logs, but may not be allowed to change permissions on the target path. The current JSON event logger behavior can therefore fail during initialization or log file creation because it actively sets permissions.
This makes multi-tenant Kyuubi deployments harder to manage when a shared JSON event log root is controlled by Ranger or another external permission system.
Making this behavior configurable allows administrators to keep the current default behavior, or disable Kyuubi-managed JSON event log permissions and let the external permission system manage access control.
Describe the solution
Introduce a new boolean configuration:
kyuubi.event.json.log.manage.permissions.enabled=true
When the value is true, the JSON event logger keeps the current behavior:
- create event log directories with the existing fixed directory permission
- set event log files to the existing fixed file permission
When the value is false, the JSON event logger:
- creates event log directories without passing fixed permissions
- does not call setPermission for JSON event log files
- still validates that the configured log root is a directory
- still writes JSON event logs normally
This keeps backward compatibility while allowing deployments with Ranger, HDFS ACLs, or other external permission systems to manage access control outside Kyuubi.
Additional context
This is useful for multi-tenant HDFS deployments where kyuubi.engine.event.json.log.path points to a shared event log root.
Example runtime configuration:
kyuubi.engine.event.loggers=SPARK,JSON
kyuubi.engine.event.json.log.path=hdfs:///path/to/kyuubi/engine-events
kyuubi.event.json.log.manage.permissions.enabled=false
The proposed change includes tests for both behaviors:
- default permission management remains enabled
- JSON event logs can still be written when permission management is disabled
Are you willing to submit PR?
Code of Conduct
Search before asking
Describe the feature
Add a configuration to control whether the built-in JSON event logger actively manages event log directory and file permissions.
Proposed configuration:
kyuubi.event.json.log.manage.permissions.enabled=trueThe default value should be true to keep the existing behavior unchanged.
When set to false, the JSON event logger should still create and write event log paths, but it should not actively set fixed permissions on event log directories or files.
Motivation
In some production deployments, event log directories are not managed only by POSIX-style HDFS permissions. They may be governed by Apache Ranger, inherited HDFS ACLs, default ACL inheritance, or other external authorization systems.
In these environments, the engine user may have enough permission to create and write JSON event logs, but may not be allowed to change permissions on the target path. The current JSON event logger behavior can therefore fail during initialization or log file creation because it actively sets permissions.
This makes multi-tenant Kyuubi deployments harder to manage when a shared JSON event log root is controlled by Ranger or another external permission system.
Making this behavior configurable allows administrators to keep the current default behavior, or disable Kyuubi-managed JSON event log permissions and let the external permission system manage access control.
Describe the solution
Introduce a new boolean configuration:
kyuubi.event.json.log.manage.permissions.enabled=trueWhen the value is true, the JSON event logger keeps the current behavior:
When the value is false, the JSON event logger:
This keeps backward compatibility while allowing deployments with Ranger, HDFS ACLs, or other external permission systems to manage access control outside Kyuubi.
Additional context
This is useful for multi-tenant HDFS deployments where
kyuubi.engine.event.json.log.pathpoints to a shared event log root.Example runtime configuration:
The proposed change includes tests for both behaviors:
Are you willing to submit PR?