subservers: fail LiT startup when integrated sub-server boot fails#1183
subservers: fail LiT startup when integrated sub-server boot fails#1183
Conversation
Summary of ChangesHello @ffranr, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the reliability of the LiT startup sequence by elevating the status of integrated sub-servers to critical. Previously, LiT could continue operating even if some of its integrated sub-servers failed to launch. Now, any failure in an integrated sub-server's boot process will cause LiT itself to cease its startup, ensuring that the system only reaches an operational state when all essential integrated components are functioning correctly. This change provides a more robust and predictable initialization experience. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request modifies the startup logic to treat integrated sub-server startup failures as fatal. The changes correctly propagate errors from StartIntegratedServers up to the main start function, causing LiT to fail on startup as intended. The documentation has also been updated to reflect this new behavior. The implementation is sound, but I have one suggestion to improve the consistency of error handling.
57ee325 to
86e8ec5
Compare
There was a problem hiding this comment.
Thanks for this @ffranr 🙏!
In addition to the feedback I've commented below, this new behaviour definitely needs itest coverage.
I think we actually started working on this at the same time, and I have local branch with draft code implementing this + itest coverage. If you want to, i can clean that up and push it so that you can cherry-pick that to make it more simple for you. Let me know if that'd be helpful :).
ellemouton
left a comment
There was a problem hiding this comment.
havent actually looked at the diff here yet, just want to make a note in case: we should just make sure that the daemon still runs & ie, that the status server still gets served
1772099 to
0091958
Compare
0091958 to
381d0b1
Compare
ViktorT-11
left a comment
There was a problem hiding this comment.
Thanks for the updates 🚀! Leaving some additional feedback in addition to @ellemouton's feedback.
c910257 to
65e4e0f
Compare
ViktorT-11
left a comment
There was a problem hiding this comment.
Thanks for the updates 🙏! Looking better, but commenting of a few lines I think you should just remove to address some previously mentioned feedback.
- Fail LiT startup if any critical integrated sub-server fails to start. - Log non-critical sub-server startup errors without aborting startup.
- Ensure critical integrated sub-servers initialize first. - Introduce alphabetical sorting for consistent order across startup runs.
65e4e0f to
3368bcf
Compare
ViktorT-11
left a comment
There was a problem hiding this comment.
After testing this PR locally, I’ve realized that this comment isn’t really addressable with how litd is currently designed:
#1183 (comment)
The reason is that shutting down lnd will always trigger the shutdownInterceptor.ShutdownChannel() to be sent over, which in turn also shuts down litd. This behavior stems from the single-interceptor pattern we currently use in litd, which we eventually want to move away from.
As a result, I don’t think it’s possible to keep litd’s status endpoint up and running while still shutting down lnd in response to critical sub-server startup errors.
Given that, I believe the behavior introduced by this PR is effectively the best we can do until the single-interceptor pattern is addressed.
@ellemouton, do you think it’s critical that litd, and therefore the status endpoint, remains up when critical sub-servers fail on startup (which also causes lnd to shut down)? If so, we’ll need to address the single-interceptor issue first. Otherwise, can we ship this as-is for now and tackle the interceptor issue at a later stage?
| case <-time.After(15 * time.Second): | ||
| t.Fatalf("expected tapd startup failure to be reported") | ||
| } | ||
|
|
There was a problem hiding this comment.
Depending on @ellemouton's response to my broader question in the main review comment, we likely want to expand the test code here, to ensure that the status endpoint & lnd node is at the state we expect it to be.
There was a problem hiding this comment.
Issue to track interceptor decoupling: #1201
- Introduce tests for critical and non-critical sub-server startup behavior. - Ensure failures in critical servers stop startup, while non-critical failures are tolerated.
3368bcf to
05b8da3
Compare
I think we need to bring in @jamaljsr & @levmi in on this discussion since the main reason for the status endpoint is to provide context to TW re why things are not starting up. The alternative is: just dont fully start up meaning that LNC sessions wont work & users will just see that they cannot connect to their node |
|
@ellemouton Thanks for the ping on this potential change. I've read through the convo here and am still unclear as to what the behavior change will be versus what we have now in In an ideal world, it would be nice if |
|
@ziggie1984: review reminder |
Closes #1181