Skip to content

Comments

Implement ActorSingleton for DistributedActor#980

Merged
ktoso merged 18 commits intoapple:mainfrom
yim-lee:da-singleton
Jul 14, 2022
Merged

Implement ActorSingleton for DistributedActor#980
ktoso merged 18 commits intoapple:mainfrom
yim-lee:da-singleton

Conversation

@yim-lee
Copy link
Member

@yim-lee yim-lee commented Jul 1, 2022

Resolves #824

⚠️ WIP - haven't updated tests yet ⚠️

@yim-lee yim-lee marked this pull request as draft July 1, 2022 07:47
self.singletonsLock.withLock {
for (_, singleton) in self.singletons {
singleton.stop(system)
public nonisolated func stop(_ system: ClusterSystem) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguably should probably become async instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, though TBH not sure how to handle async in ClusterSystem.shutdown.

@ktoso
Copy link
Member

ktoso commented Jul 3, 2022

Okey I think I have some work to be done here to make it nicer to implement! I'll follow up as soon as I'm out of compiler land!

@ktoso
Copy link
Member

ktoso commented Jul 8, 2022

I'll check this out and see how to improve use of actor context and metadata now 👍

// MARK: Cluster singleton boss

internal protocol ClusterSingletonProtocol {
internal protocol _ClusterSingletonBoss {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If internal no need to underscore it :)

The underscoring is to hide from docc docs, but internal types are not included in those docs anyway so we're good 👍

Love that we're running with the boss name, much more fun 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I will rename it to ClusterSingletonBossProtocol. There was a name conflict. :)

public protocol ClusterSingletonProtocol: DistributedActor where ActorSystem == ClusterSystem {
/// Must be implemented using a `@ActorID.Metadata(\.clusterSingletonID)` annotated property.
var singletonName: String { get }
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup right, we wont be using this after all

private func updateTargetNode(node: UniqueNode?) async throws {
guard self.targetNode != node else {
self.log.debug("Skip updating target node. New node is already the same as current targetNode.", metadata: self.metadata())
let metadata = await self.metadata()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wow this is bad, definitely not async logger metadata access. (Why it's bad: this will always render all the metadata, AND causes suspensions even if we're not even logging -- the logging must be done inside the debug(...) call because it is lazily evaluated (metadata is an auto closure)

It's because the move of the singleton in to the other actor -- if let singleton = await self.singleton {
but why did we do this in a new actor to begin with...? This actor should be able to do everything necessary?

Copy link
Member Author

@yim-lee yim-lee Jul 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

over-engineered it. wanted to group the logic but got carried away. amended 3b882d6.

@yim-lee yim-lee marked this pull request as ready for review July 14, 2022 00:49
@yim-lee yim-lee changed the title [WIP] Implement ActorSingleton for DistributedActor Implement ActorSingleton for DistributedActor Jul 14, 2022
@yim-lee
Copy link
Member Author

yim-lee commented Jul 14, 2022

Make this PR mergeable to prepare for #1001

@yim-lee yim-lee requested a review from ktoso July 14, 2022 00:51
defer { self.memberTimerTasks.removeValue(forKey: member) }

try await Task.sleep(until: .now + delay, clock: .continuous)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah nice thx

await self.settings.plugins.stopAll(self)
pluginsSemaphore.signal()
}
pluginsSemaphore.wait()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another case of "we need send self settings.plugins.stopAll(self)" strikes again huh...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah arguably we could make the shutdown async now 🤔

/// determine the node that the singleton runs on. If the singleton falls on *this* node, `ClusterSingletonBoss`
/// will spawn the actual singleton actor. Otherwise, `ClusterSingletonBoss` will hand over the singleton
/// whenever the node changes.
internal distributed actor ClusterSingletonBoss<Act: ClusterSingletonProtocol>: ClusterSingletonBossProtocol where Act.ActorSystem == ClusterSystem {
Copy link
Member

@ktoso ktoso Jul 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(love the boss naming, makes me smile each time lol) Enough manager objects already! 😆

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

}
} else {
// Run singleton on this node if clustering is not enabled
self.log.debug("Clustering not enabled. Taking over singleton.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, nicely handled as well

case .some(let node) where node == self.actorSystem.cluster.uniqueNode:
()
case .some(let node):
let singleton = try Act.resolve(id: .singleton(Act.self, settings: self.settings, remote: node), using: self.actorSystem)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FYI, That's one of the places I have new impl based on well-known metadata for :)

}

nonisolated func stop() async {
Task {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do without this outer task I think, will try locally

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes, because the method is async now.

enum BufferError: Error {
case full
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Basically our new "async" stash replacement, looking good

reply.shouldStartWith(prefix: "Hello Charlie!")

// singleton.ref (proxy-only)
let proxyRef = try await test.singleton.proxy(of: TheSingleton.self, name: TheSingleton.name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Impl wise I actually realized that we don't handle this well, we might just do the host version for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

Copy link
Member

@ktoso ktoso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looking good! I'll merge and rebase my changes onto this and get to finishing all the tests unlocking 👍

@ktoso ktoso merged commit a8b9e64 into apple:main Jul 14, 2022
@yim-lee yim-lee deleted the da-singleton branch July 14, 2022 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement ActorSingleton for DistributedActor

2 participants