Repository navigation
Decode Incrementally - #517
Merged
Merged
Conversation
merrymercy
reviewed
Jun 8, 2024
merrymercy
left a comment
Collaborator
There was a problem hiding this comment.
LGTM.
- Does it affect the throughput in non-streaming mode?
- Can it improve the throughput for streaming mode?
Collaborator
Author
|
merrymercy
approved these changes
Jun 12, 2024
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
Garrybest
pushed a commit
to Garrybest/sglang
that referenced
this pull request
Jan 9, 2026
* improving code readability for http server
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
… correct the §1.3 model, and make the sgl-project#431 guard non-blocking Two commits: test(sgl-project#483,sgl-project#459): sgl-project#483 ("check_regressions prices prefill on stale rates") is REFUTED -- check_regressions already prices on the RESOLVED rates (landed with sgl-project#475). What was actually missing was the PIN: the only test calling the function is checkpoint-gated and skips in every hermetic run, so a revert would be invisible. test_check_regressions_pricing_483.py is the hermetic falsifier. Also makes the s12 harness honest: an absent measurement is None, never 0.0 (a spec-off tick has no accept length, and 0.0 silently survives every arithmetic it enters); and any procedural property a verdict rests on (warm-up discarded, draws back-to-back, gap between draws) is recorded as a measured number in the artifact, not taken on the harness's word -- the s12 warm-up draw ran for months and was discarded silently, so a warmed-up point and a cold one wrote the same file. perf(sgl-project#517): names the actual decode-loop BAR1 collectives and corrects the catalog's round model -- a decode round is NOT free of host-path collectives (the belief the old model rested on). Per NEXTN round there are 3 replay boundaries (the draft chain is one captured graph, not one per step) and 5 host-path BAR1 broadcasts, all sgl-project#50 speculative rank-agreement syncs, reaching barlink because a barlink boot never constructs pynccl. Measured cost: the sgl-project#431 blocking status-word read costs -9.22% decode TPS against the same-tree NCCL baseline; removing it recovers +2.68%. Fix: the abort-guard status word is now read asynchronously (non-blocking D2H + cudaEventQuery, returning what an earlier check staged), trading reporting LATENCY for zero stream synchronization per check -- detection is unaffected because ctlStatus is sticky. SGLANG_BARLINK_BAR1_ABORT_MAX_LAG (default 4) bounds the deferral with a can-fail proof; ..._ABORT_DEFER=0 restores the exact pre-sgl-project#517 blocking behavior. NOTE_517_bar1_guard_desk.md has the full model, desk-only (no GPU arm run yet -- ticketed). Two real conflicts, both resolved by combining rather than choosing a side: - FEATURE_CATALOG.md §12 (same insertion-point pattern as every merge today): HEAD's "Verified carve-outs (sgl-project#514)" paragraph kept, this branch's two sgl-project#459 paragraphs ("absent measurement is None" / "procedural property must be in the artifact") appended after. Two OTHER hunks (the sgl-project#483-REFUTED note near §2, the "Guard cost" note near §7) landed with no conflict at all -- neither region had been touched since this branch's base. - test/registered/unit/distributed/_bar1_marker_source.py: both HEAD (via sgl-project#514's earlier edits) and this branch carry DIFFERENT LINE_* pins into barlink.py/barlink_bar1.py, because each was written against its own version of those files. Neither side's numbers are correct once both sets of source edits are combined. Resolved by reading the ACTUAL post-merge source: grepped the real logger.info/logger.warning/raise RuntimeError call-start lines in the merged barlink.py and barlink_bar1.py (2236/2250/3207/810) and verified all four renderers execute correctly against them before committing -- not a textual merge of the two guesses. All 17 catalog sections and every other additive paragraph verified present. Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - The five directly touched/new test files (_bar1_marker_source.py's renderers verified live against the real merged source, plus test_barlink_bar1_abort_431.py, test_barlink_bar1_abort_deferred_517.py, test_s12_harness_459.py, test_check_regressions_pricing_483.py): 67 + 7 passed, 0 failed. - test_s12_log_analyse.py specifically (s12_log_analyse.py is touched): same 4 pre-existing failures, unchanged (fixture-data-dependent, not source-code-dependent). - test/registered/unit/distributed/ (full): 24 failed, 2206 passed, 8 skipped, 723 subtests -- same 24 failing IDs as the established baseline, byte-identical. - ruff check --select=F401,F821,UP037: clean. - codespell: 7 hits, all confirmed pre-existing (a German prose sample string already in scripts/gpu_battery/s12_prefill_kurve.py and s14_decode_punkt.py at this branch's base, untouched by its diff).
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 3, 2026
… medians, honest server state Three new server-side computed blocks the Monitor tab's landing poll now returns (all server-side so a curl client sees exactly what the page draws, per the runbook's "curl is a complete client" architecture rule): - server_state.py: a four-state discriminator (not_running | starting | running_no_metrics | running_with_metrics) that keys off an explicit API probe, never a metrics-scrape failure alone -- a connection-refused scrape used to be rendered as "no --enable-metrics", which is a different diagnosis than "not running at all". running_no_metrics is only ever reported with a passing API probe. - rate_medians.py: medians behind the rate tiles, windowed over PROCESSING polls only -- idle polls never enter the window, so an idle server still reports the median of what it did while working, not a zero-diluted average. - spill_tiers.py / tier_occupancy.py: one row per spill/offload tier type x place, with absent tiers PRINTED alongside their reason rather than hidden or shown as zero. Two tiers are deliberately never populated: the sgl-project#407 memtier registry (no production consumer, so reading it would print zeros that look measured) and the sgl-project#286 short-term register (no reachable byte ledger in this build) -- both documented as absent-by- construction, not omissions. webui.py and live_metrics.py wire these into the existing landing poll; metrics_reporter.py and metrics_collector.py get the small hooks the new blocks read from. Clean auto-merge on both touched docs (FEATURE_CATALOG.md and rig-runbook.md) -- neither had a conflicting edit on this line since this branch's base (3e3aca2, the sgl-project#510 merge). rig-runbook.md gains new §8.0 documenting the shell-only verification recipe (curl + three python3 one-liners for state/medians/tiers) and the restart requirement: the dashboard serves from one long-lived process, so a change is live only after that process restarts -- done as the merge step below, not mid-session. All 17 catalog sections and all 21 previously-preserved additive paragraphs verified present (two-axis §1, matrix doctrine §2, all six §12 families from sgl-project#510/sgl-project#513/sgl-project#511/sgl-project#487/qwen38-prep, sgl-project#514's five families + value-pinning, sgl-project#499's resolution-ordering, sgl-project#517's two sgl-project#459 paragraphs, MERGE-DUTY owner-sites + SITREP §12, NVENC §13). Tests (CUDA_VISIBLE_DEVICES=99, PYTHONPATH pinned to this worktree): - The six directly touched/new test files: 517 passed, 0 failed. - test/registered/unit/planner/ (full, HTSGLANG_TEST_MODEL_DIR set): 2337 passed, 1 skipped, 352 subtests -- no failures. - ruff check --select=F401,F821,UP037: 1 finding (live_metrics.py's LiveSnapshot import), confirmed pre-existing -- already unused in the pre-merge base, only mentioned in a docstring, the line just shifted from sgl-project#522's unrelated insertions above it. - codespell: 1 hit (webui.py:1786 "unparseable"), confirmed pre-existing at the identical line in the pre-merge base, in unrelated SVG chess-piece-parsing code this branch never touches.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 5, 2026
…thread, hot path check-free
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 5, 2026
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 6, 2026
A live specimen (2026-08-06 21:08:18, pids 41283/41284/41285) was captured
with all three ranks in the SAME stack and held there for over eleven
minutes, byte-identical across two py-spy samples eleven minutes apart:
Event.synchronize (torch/cuda/streams.py:254)
_read_status_for_check (barlink_bar1.py:4527)
check_aborted (barlink_bar1.py:4718)
all_reduce -> attention_tensor_model_parallel_all_reduce
_gather_hidden_states_and_residual (communicator.py:1287)
Not one abort line was emitted in that window, on any rank. Two independent
defects produced that silence, and this commit closes both.
1. THE GUARD WAITED ON THE FAULT. The staged status read issues its D2H on
the COMPUTE stream, so the event that signals it is ordered behind the
collective that just ran. The `max_lag` bound then forced
`event.synchronize()` on that event -- a wait bounded by the collective,
which is unbounded when the collective is what hung. The guard joined the
wedge instead of reporting it. The wait now has a wall-clock deadline
(SGLANG_BARLINK_BAR1_ABORT_SYNC_DEADLINE_MS, default 2000 ms, 0 restores
the old blocking wait). Expiry leaves the read unresolved and the
accumulated window intact, and logs the op and byte count -- the first
host-visible symptom this path could previously emit at all.
2. THE WATCHDOG SILENTLY SKIPPED THIS TRANSPORT. `poll_status_words` walks
registered transports and calls `poll_status_word` on each, looked up with
`getattr(..., None)`. Only the DEVICE transport ever defined it.
`BarlinkBar1Transport` did not, so the walk reached it, found nothing and
moved on -- meaning the private-stream read, the one reader that still
resolves while the compute stream is wedged, never ran on the BAR1 path.
Ported from barlink_device.py, where the shape has been proven since sgl-project#517
phase 2. The hot path now consults that sticky mirror first.
Escalation: an unbroken run of deadline expiries (default 30, i.e. ~60 s of a
stream that will not retire four bytes; a healthy round is ~46 ms) raises the
new Bar1CollectiveStalled, carrying group/rank/op/nbytes/expiries. It derives
from Bar1KernelAborted so existing handlers keep working, and it states
explicitly that the abort word is CLEAN -- a stall, not a trip. Before this,
the only actor that ever ended the wedge was an external supervisor, minutes
late and with no attribution.
WHAT THE SPECIMEN ALSO SHOWED, and what this commit does NOT yet fix. The
three ranks were stuck in collectives of DIFFERENT sizes: rank 0 at
nbytes=3420160 in decoder layer 6, ranks 1 and 2 at nbytes=12584960 in layer
5. The launch sampler agrees across the whole wedge window and adds a second
disagreeing pair (rank 0 at 4104192, ranks 1/2 at 6040768). A pre-wedge
control over 21:00-21:05 shows the ranks normally run IDENTICAL sizes
(10229760, 10301440, 10444800, 10670080, ... on all three), so the sizes are
not per-rank shards and the disagreement is abnormal. The two sizes ranks 1
and 2 froze on never occur anywhere in the preceding nine hours of their
logs. No Xid/NVRM/IOMMU fault, and the abort word stayed clean throughout, so
no kernel took its abort path: the collectives are waiting on peer
contributions that do not match. Root-causing that divergence needs the
collective history at the moment of the stall, which is exactly the artifact
the two defects above suppressed.
Tests (CUDA_VISIBLE_DEVICES=99, hermetic, no GPU):
test/registered/unit/debug/ 354 passed (327 before, 27 new)
test/registered/unit/distributed/ unchanged; the 4 failures in
test_uniform_decode_mem_603.py are pre-existing, reproduced with this
patch reverted (_FakeScheduler has no attribute 'server_args').
ruff check python/.../device_communicators/ + the new test: clean.
The new suite includes a falsifier that asserts the UNBOUNDED wait does not
return within 500 ms against an event that never completes, so the bounded
case is proven to be the difference rather than assumed.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 6, 2026
…ion 18 reusable building blocks — 33 verified entries, 4 misdescriptions corrected (offload register is two modules; collective history lives in distributed/collective_census.py not debug_utils; byte-identity harness under tests/determinism; sgl-project#517 staged status-read is a pattern not a module), 13 previously undocumented modules incl. runtime_context (~400 consumers); merge-checklist rule; 66-test file:line consistency guard)
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 8, 2026
… probe The SIGUSR1 launch dump must never touch device memory (its documented no-sync constraint), so it could not print the ack barrier's round/ watermark words — and the on-card monotonicity probe had nothing to read. The watchdog abort poll (sgl-project#517) already stages device words through a private stream every 10 ms; it now also stages (round, mesh watermark, a2a watermark) into a pinned mirror, and the dump prints that mirror. Two spaced SIGUSR1 probes under replay load showing strict growth of all three words per rank are the ack barrier's capture-safety proof through the real replay path. 46 test items green (launch-diff + abort suites).
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 16, 2026
…th collective sgl-project#517 named five BAR1 broadcasts on the HOST path per NEXTN decode round. Verified against the current tree, that inventory is now THREE, not five: the three verify-result syncs were fused into one by pack_accept_payload (#616c). The remaining two are the draft-pick sync in _draft_extend_for_decode, which capture_safe_tp_broadcast still issues one collective PER tensor from its loop. This fuses that pair into one buffer. The picks have mixed dtypes (topk_index integer, topk_p float32 at the SAME shape), so unlike the int32 accept payload the fusion is byte-level. Contiguity is required rather than coerced: reshape(-1) on a non-contiguous tensor returns a copy, so an in-place unpack through it would silently drop the broadcast. fuse is opt-in per call site and defaults OFF. _broadcast_draft_picks is also called from inside the draft CUDA-graph capture (draft_forward), and enabling fusion globally would change what a captured region allocates -- a capture change. Only the site that is both per-decode-round and outside any captured region opts in; the prefill, catchup and in-loop sites stay byte-identical. Not claimed here: a ms/round figure. sgl-project#476's 6.64 pp Seam A cost was measured with the pre-sgl-project#517 guard, which synchronised once per collective; sgl-project#517 removed that sync, so the per-collective price must be re-measured on metal rather than carried over. The A/B is window-gated. Tests: test_draft_pick_fusion_524.py, 7 passed (red first: the three symbols did not exist). Regression over test/registered/unit/debug -k "616 or spec_util or draft_pick or eagle or broadcast": 351 passed, 4 failed. Those 4 are pre-existing and unrelated -- verified by reverting both source files and re-running: test_barlink_bar1_abort_poll_616f.py fails identically on the clean tree (SimpleNamespace missing _round_dev, a sgl-project#624-class stub drift). ruff check clean on both changed files (the one F401 in eagle_worker_v2.py is pre-existing on HEAD); ruff format applied.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…bort read BACK The blocker named in bdbbaf4 is cleared, and the stop caller is wired. THE BLINDNESS, restated because the fix is shaped by it. Since sgl-project#517 phase 2 the watchdog is the only reader of a BarlinkDeviceTransport's abort word: _arm_status_poll (barlink_device.py:1440) latches _abort_poll_active ONCE at bring-up (:1478), the latch is one-way, check_aborted (:1540) short-circuits on it, and _abort_code_seen is written only by poll_status_word (:1504) from the watchdog thread. Stopping the watchdog therefore left the word unread by anyone and check_aborted answered "not aborted" forever. STEP 1 -- rearm_inline_reads() (barlink_abort_gate). Stopping the watchdog now clears _abort_poll_active on every registered transport, so check_aborted falls back to the pre-sgl-project#517 in-line device read. This is the degradation should_poll_status already declares intended -- "the guard degrades to the sgl-project#517-phase-1 behaviour rather than to blindness" -- which until now was true only at bring-up, because that is where it is evaluated. The re-arm runs BEFORE the join, so there is no window in which the reader is gone while the latch still claims it reads. Pinned by a test that has the worker record the latch at the moment it observes the stop event. ABORT-WORD SEMANTICS UNTOUCHED: the word stays sticky, nothing writes it, nothing clears a code. Only WHO READS IT changes, back to the reader it had before sgl-project#517 phase 2. The in-line read it restores is the pre-sgl-project#517 per-collective device read, so its cost is paid only AFTER a stop -- during teardown, never during serving. SCOPE CORRECTION to my own bdbbaf4 wording ("every device transport"): the blindness is the barlink_device family only. BarlinkBar1Transport.check_aborted (barlink_bar1.py:5122) never consults the latch -- it always reads through _read_status_for_check (:4889) -- so bar1 was never blind; there the latch only short-circuits the watchdog's own poll. The re-arm covers both anyway, and a test pins the asymmetry so a future edit cannot quietly make bar1 depend on it. STEP 2 -- PeerWatchdog.stop, sibling shape. It used to do `thread, self._thread = self._thread, None` BEFORE the join, so a timed-out join left a live thread with no record and a second stop reported success for a thread it had abandoned. Now: join first, clear the handle only on a real join, keep it and log a loud WARNING on timeout, return "joined"/"detached"/"already stopped". DEADLINE 0.25 s, DERIVED NOT COPIED. _run waits poll_interval_s() (10 ms default) between passes, so a cooperative exit lands within about one tick plus one poll pass. 250 ms is ~25 ticks -- ample here, and 8x tighter than the 2 s the kvso and lane siblings use, which wait on a host tier write and a CUDA kernel launch respectively. STEP 3 -- the caller. barlink_liveness.stop_watchdog() is the counterpart ensure_watchdog never had. scheduler_teardown.release_barlink_watchdog() is ALWAYS-STOP with no gate: the thread leaks on every boot with barlink liveness on, whether or not the destroy is armed, so the destroy's flag must not leak into it. Ordering is enforced BY CONSTRUCTION inside release_distributed, which stops and joins the watchdog before it touches parallel_state; the scheduler also calls it explicitly beforehand, but that call order is one careless edit from inverting. NOT TOUCHED, as instructed: destroy gating, barlink_comm close order, shm unlink, abort-word semantics. No transport close path was edited. Tests, hermetic (CUDA_VISIBLE_DEVICES=""), 24 new: test/registered/unit/distributed/test_barlink_watchdog_rearm_673.py RED-FIRST: 23 failed / 1 passed before the fix. The one that passed is the bar1 precision pin, which was already true -- it documents scope, it does not test the fix. the headline: an abort injected AFTER the stop is still SEEN latch cleared on every registered transport, before the thread is gone, heterogeneous registry left alone, a raising transport does not strand the others, idempotent, semantics unchanged on a clean word stop: timed-out join keeps the handle, detach is logged, a second stop does not claim success, clean join clears and is quiet, idempotent, safe when never started, deadline derived from the poll cadence wiring: graceful stops and re-arms, exception path leaves it alone, ALWAYS-STOP with no gate, no watchdog is a quiet no-op, never raises ordering: release_distributed records stop_watchdog BEFORE destroy_model_parallel; source-order and AST call-site pins can-fail by breakage, FOUR independent neuters: re-arm removed -> 5 red (incl. the blindness test) handle cleared before the join -> 3 red ordering precondition removed -> 1 red (the by-construction test) scheduler call site removed -> 2 red regression, name-level with --color=no against a removed-diff baseline: distributed + managers + debug baseline 31 failed / 5199 passed with fix 29 failed / 5225 passed ZERO new failures. Two baseline failures did not reproduce (test_load_snapshot_backends ZmqRoundTrip, test_phase_flip_mover_streaming_631 staging price) -- both unrelated to barlink teardown, so they are FLAKY, not fixed by this change, and no credit is claimed for them. A harness bug of my own, found by running: the first watchdog stand-in used a worker that returned immediately, so every stop reported "already stopped" and four wiring tests were asserting nothing. The default worker now stays alive until stopped. FILING UPDATE: docs/FINDING_673_barlink_watchdog_stop_blocked.md carries the resolution paragraph and the scope correction. The sgl-project#673 family now stands at 5 of 6 abort-shaped items desk-addressed, 1 filed with no action (the vendored mooncake/mori/nixl workers -- no stop mechanism to wire, PD-only). ON-METAL PROOF REMAINS DEFERRED for the whole family: the abort is intermittent per process, so a single clean boot proves nothing. The fixes ride along on future teardown-armed boots.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…c, cross-ref sgl-project#726 Docs only; no production file touched. Closes the three items my own retro-sweep left open. ## 1. EVERY=32 (former section 19.3): NOT DEAD, but path-dependent Traced the constant to its consumers rather than trusting either ledger entry. check_every() returns 1 when the env var is unset (barlink_abort_gate.py: 189-197). The default is ONE, not 32 -- "=32" is only meaningful if somebody sets it. barlink_bar1.py:5177 reaches it UNCONDITIONALLY, after the abort_check_enabled / pending-launch gates. LIVE on bar1. barlink_device.py:1562 sits in the in-line fallback path BELOW sgl-project#517's _abort_poll_active early return, so on the device transport it is bypassed entirely whenever the watchdog feeds the flag. BOTH LEDGER READINGS WERE PARTIALLY RIGHT AND NEITHER WAS COMPLETE. "Dead since sgl-project#517" is true of the DEVICE path only -- that is exactly what phase 2 made cheap -- while bar1 never got that treatment and still consults the knob. NOTE_517 itself treated ..._EVERY as arm B3, expected to buy nothing on top of the staged read; "buys nothing measurable" is not "the consumer was removed", and that elision is how the two readings drifted apart. The sgl-project#431 utilisation figure (72-84% -> 86-91%) predates sgl-project#517 and was never re-measured after it. THE RECIPE QUESTION, answered: the production boot recipe does NOT set it -- absent from startkommandos-rig.md (grep -c = 0). So there is no dead knob in the recipe misleading the next operator, which was the specific worry. Worth recording: this is the SAME bar1-vs-device asymmetry I found independently in sgl-project#673, where bar1's check_aborted never consults _abort_poll_active while the device path short-circuits on it. Two investigations converged on one structural fact about these two transports. Section 19.3 is now empty and points at the settled 19.1 entry. ## 2. Stale spill-matrix doc (section 19.4): corrected at both sites ANALYSE_spill_matrix_20260804.md claimed kvso x HiCache is mutually exclusive and that "kvso cannot run on the production recipe at all". sgl-project#550 superseded that: the combination is OPT-IN, gated on KVSO_ALLOW_HICACHE (server_args.py:7502, `if os.environ.get("KVSO_ALLOW_HICACHE", "0") != "1": raise`). Per the refusal's own text the two host pools are independent objects with DISJOINT KEY SPACES and their pinned host RAM is summed by ONE JOINT BUDGET GUARD instead of each validating alone; what remains is a measurement (spill-copy vs prefetch contention), not a mechanism. Fixed in section S1 and in the H15 matrix row, with the original text kept inline so the change is legible rather than silently rewritten. ANCHOR CORRECTION: the gate is at server_args.py:7502, NOT the :7385-7395 given in the brief. Cited by symbol per the anchor rule -- the sgl-project#621 sweep found audit line numbers had drifted, two of three into the wrong file. ## 3. sgl-project#726 cross-reference added to section 19.1 The INT8-KV IMMA-QK builder must CREATE the dtype surface, not find it: choices at server_args.py:1026 carry no int8, the fp8 KV scale path is hard-coded per-tensor, and the one per-group-scale precedent (MHATokenToKVPoolFP4) dequantises eagerly before any backend sees it -- copying it buys VRAM savings but NOT the bandwidth savings an int8-KV lane exists for. Budget the plumbing as new work. The anchor is VERIFIED; the FP4-precedent detail is marked UNVERIFIED-LEDGER (relayed from ANALYSE_726, not re-read here). PRIOR-ART GATE honoured: grepped FEATURE_CATALOG, docs/dev and git log --all --grep before each change; both absence claims carry the file:line of the refusing gate (KVSO_ALLOW_HICACHE at :7502, kv_cache_dtype choices at :1026).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?