Implement core functionalities of the IO component in scaler - YMQ#149
Merged
sharpener6 merged 50 commits intofinos:mainfrom Jul 22, 2025
Merged
Implement core functionalities of the IO component in scaler - YMQ#149sharpener6 merged 50 commits intofinos:mainfrom
sharpener6 merged 50 commits intofinos:mainfrom
Conversation
0e9cd20 to
b65c155
Compare
b346485 to
90a1744
Compare
|
|
885c371 to
b651dd5
Compare
rafa-be
reviewed
Jun 25, 2025
rafa-be
reviewed
Jun 25, 2025
rafa-be
reviewed
Jun 25, 2025
rafa-be
reviewed
Jun 25, 2025
rafa-be
reviewed
Jun 25, 2025
rafa-be
reviewed
Jun 25, 2025
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
Signed-off-by: gxu <georgexu420@163.com>
rafa-be
reviewed
Jul 21, 2025
rafa-be
reviewed
Jul 21, 2025
rafa-be
reviewed
Jul 21, 2025
Signed-off-by: gxu <georgexu420@163.com>
sharpener6
reviewed
Jul 22, 2025
sharpener6
reviewed
Jul 22, 2025
Signed-off-by: gxu <georgexu420@163.com>
sharpener6
approved these changes
Jul 22, 2025
rafa-be
pushed a commit
to rafa-be/scaler
that referenced
this pull request
Jul 30, 2025
…inos#149) * Update to use g++-14 to compile instead of the defaulted g++-13 etc. - update inaccurate wording in the file - make sure scripts are executable Signed-off-by: gxu <georgexu420@163.com> * Use CMAKE_BUILD_TYPE as the source of truth for what build we are on Signed-off-by: gxu <georgexu420@163.com> * Update TODOs and DONEs in readme Signed-off-by: gxu <georgexu420@163.com> * Implement C++ core of YMQ network library Signed-off-by: gxu <georgexu420@163.com> * Update pymod io_context to make sure the code compiles (TMP FIX!) Signed-off-by: gxu <georgexu420@163.com> * Provide basic examples on how to use YMQ in C++ Signed-off-by: gxu <georgexu420@163.com> * Bump version number Signed-off-by: gxu <georgexu420@163.com> * Add qualifiers and attributes to functions (noexcept, nodiscard, etc.) Signed-off-by: gxu <georgexu420@163.com> * Change calls to move_only_functions so functions are not copied Signed-off-by: gxu <georgexu420@163.com> * Extract TCP Operations to their own header Signed-off-by: gxu <georgexu420@163.com> * Implement IOSocketType behavior in YMQ Signed-off-by: gxu <georgexu420@163.com> * Even Unicast and Multicast needs to handle reconnect Signed-off-by: gxu <georgexu420@163.com> * Add pub-sub examples to demonstrate socket types Signed-off-by: gxu <georgexu420@163.com> * upd readme (need rebase later) Signed-off-by: gxu <georgexu420@163.com> * Improve the examples Signed-off-by: gxu <georgexu420@163.com> * Change to new socket types Binder and Connector Signed-off-by: gxu <georgexu420@163.com> * Refactor: MessageConnectionTCP is easier to construct correctly Signed-off-by: gxu <georgexu420@163.com> * upd readme (need rebase later) Signed-off-by: gxu <georgexu420@163.com> * Implement ISO 8601 conforming timestamp output Signed-off-by: gxu <georgexu420@163.com> * Reference implementation of the error type Signed-off-by: gxu <georgexu420@163.com> * Example usage to the error type Signed-off-by: gxu <georgexu420@163.com> * upd readme (need rebase later) Signed-off-by: gxu <georgexu420@163.com> * Make comments better looking Signed-off-by: gxu <georgexu420@163.com> * fixup! Reference implementation of the error type Signed-off-by: gxu <georgexu420@163.com> * fixup! Example usage to the error type Signed-off-by: gxu <georgexu420@163.com> * Use switch on enum to handle error->explanation translation Signed-off-by: gxu <georgexu420@163.com> * use std::string_view instead of const char* Signed-off-by: gxu <georgexu420@163.com> * Fix off by one error when getting format string Signed-off-by: gxu <georgexu420@163.com> * Refinements on Error implementation Signed-off-by: gxu <georgexu420@163.com> * Refactor: Put network utils in their own file Signed-off-by: gxu <georgexu420@163.com> * Refactor: createIOSocket takes a callback and returns nothing Signed-off-by: gxu <georgexu420@163.com> * Remove unneeded comments Signed-off-by: gxu <georgexu420@163.com> * Fix pymod ymq build due to namespace change Signed-off-by: gxu <georgexu420@163.com> * Add namespace scaler::ymq for every file Signed-off-by: gxu <georgexu420@163.com> * Refactor: Extract common part of examples to header file Signed-off-by: gxu <georgexu420@163.com> * check epoll_wait calls return value Signed-off-by: gxu <georgexu420@163.com> * Update automated examples Signed-off-by: gxu <georgexu420@163.com> * Fixup typo found in logging.h Signed-off-by: gxu <georgexu420@163.com> * Fix: send/recv were implemented incorrectly but not shows up in low tp. Signed-off-by: gxu <georgexu420@163.com> * Fix bugs for not adding the correct offset (Thanks magniloquency!) Signed-off-by: gxu <georgexu420@163.com> * Update the exmaple to mimic high throughput scenario Signed-off-by: gxu <georgexu420@163.com> * Make sure onRead returns early when socket is closed and set to 0 Signed-off-by: gxu <georgexu420@163.com> * Update automated example to not use hard coded address in syncBindTo Signed-off-by: gxu <georgexu420@163.com> * Update error's implementation Signed-off-by: gxu <georgexu420@163.com> * Implement error abort for syscalls Signed-off-by: gxu <georgexu420@163.com> * Fix: Did not return when receiving EAGAIN Signed-off-by: gxu <georgexu420@163.com> * Add notice with respect to thread safety of methods Signed-off-by: gxu <georgexu420@163.com> * Remove unneeded IOSocketType(s) Signed-off-by: gxu <georgexu420@163.com> --------- Signed-off-by: gxu <georgexu420@163.com>
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
READY FOR REVIEW.
Implements the C++ core of
YMQ, the network part ofscaler.The initial design and implementation differs by about 10%, with majority comes from the idea of
RecvMessageCallbackshould be placed in theIOSocketlevel.General execution flow behaves like the following graph.
Limitations
Ability
Performance
sendMessageinO(n)time which can be reduced toO(lg n)easily.O(n)time which can be reduced toO(lg n)not so easily.Security