Repository navigation
Conversation
|
@codecov-ai-reviewer review |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
Walkthrough将图谱抽取从单一同步端点扩展为支持依赖注入的完整 HTTP 服务体系:新增异步作业存储与调度、导入端点、抽取后导入复合端点;引入请求级多图配置透传、并行抽取执行策略、LLM extract 配置回退机制,以及结构化错误响应与统计汇总。 Changes图抽取与导入系统重构
Sequence DiagramssequenceDiagram
participant Client
participant GraphExtractAPI
participant InMemoryGraphExtractJobStore
participant GraphExtractService
participant SchedulerSingleton
rect rgba(70, 130, 180, 0.5)
note over Client,GraphExtractAPI: POST /graph/extract/jobs
Client->>GraphExtractAPI: 提交抽取请求
GraphExtractAPI->>InMemoryGraphExtractJobStore: create(request) → job_id
alt run_jobs_inline=True
GraphExtractAPI->>InMemoryGraphExtractJobStore: run_job(job_id, service)
InMemoryGraphExtractJobStore->>GraphExtractService: extract_sync(request)
GraphExtractService->>SchedulerSingleton: schedule_flow(GRAPH_EXTRACT, ...)
SchedulerSingleton-->>GraphExtractService: raw_result
GraphExtractService-->>InMemoryGraphExtractJobStore: GraphExtractResponse
InMemoryGraphExtractJobStore->>InMemoryGraphExtractJobStore: mark_succeeded
else run_jobs_inline=False
GraphExtractAPI->>InMemoryGraphExtractJobStore: submit_job(job_id, service) → 队列
end
GraphExtractAPI-->>Client: 202 GraphExtractJobCreateResponse
end
rect rgba(100, 149, 237, 0.5)
note over Client,GraphExtractAPI: GET /graph/extract/jobs/{id}/result
Client->>GraphExtractAPI: 轮询结果
GraphExtractAPI->>InMemoryGraphExtractJobStore: expire_jobs() + get(job_id)
InMemoryGraphExtractJobStore-->>GraphExtractAPI: GraphExtractJob
alt status=SUCCEEDED
GraphExtractAPI-->>Client: 200 GraphExtractResponse
else status=PENDING/RUNNING
GraphExtractAPI-->>Client: 202 GRAPH_EXTRACT_JOB_NOT_COMPLETE
else status=EXPIRED
GraphExtractAPI-->>Client: 410 GRAPH_EXTRACT_JOB_EXPIRED
end
end
rect rgba(60, 179, 113, 0.5)
note over Client,GraphExtractAPI: POST /graph/extract-and-import
Client->>GraphExtractAPI: 抽取+导入请求
GraphExtractAPI->>GraphExtractService: extract_sync(req)
GraphExtractService-->>GraphExtractAPI: GraphExtractResponse
GraphExtractAPI->>GraphExtractAPI: 组装 GraphImportRequest
GraphExtractAPI->>GraphExtractService: import_graph(import_req)
GraphExtractService->>SchedulerSingleton: schedule_flow(IMPORT_GRAPH_DATA, ...)
SchedulerSingleton-->>GraphExtractService: import_result
GraphExtractService-->>GraphExtractAPI: GraphImportResponse
GraphExtractAPI-->>Client: 200 GraphExtractAndImportResponse
end
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request introduces job-based asynchronous graph extraction and import APIs, allowing users to submit extraction tasks as background jobs, monitor their status, and cancel them. It also adds support for chunk-based parallel extraction using a thread pool, robust schema validation, detailed import result tracking, and fallback LLM configurations. Key feedback focuses on improving the job queue's reliability by marking jobs as failed when the queue is full, preventing memory leaks by clearing result payloads of expired jobs, reusing thread pools to avoid execution overhead, and refining error handling to prevent client-side validation errors from incorrectly returning 500 Internal Server Errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| try: | ||
| self._queue.put_nowait((job_id, service)) | ||
| except queue.Full as exc: | ||
| raise ValueError("graph extraction job queue is full") from exc |
There was a problem hiding this comment.
When the job queue is full, self._queue.put_nowait raises queue.Full, which is caught and propagated as a ValueError. However, the job remains in the store with a PENDING status indefinitely. This can lead to a resource leak where stale pending jobs count towards self.max_jobs and block new jobs from being created.
Consider marking the job as FAILED with an appropriate error before raising the exception.
try:
self._queue.put_nowait((job_id, service))
except queue.Full as exc:
self.mark_failed(
job_id,
GraphExtractError(
code="GRAPH_EXTRACT_JOB_QUEUE_FULL",
message="graph extraction job queue is full",
phase="job",
job_id=job_id,
),
)
raise ValueError("graph extraction job queue is full") from excThere was a problem hiding this comment.
Fixed in 7850be7. submit_job now rolls back the pending job when queueing fails so capacity is released, with a queue-full regression test.
| if job.status != GraphExtractJobStatus.EXPIRED and (expired_by_result_ttl or expired_by_pending_ttl): | ||
| job.status = GraphExtractJobStatus.EXPIRED | ||
| job.updated_at = now |
There was a problem hiding this comment.
Stale expired jobs (and their potentially large result payloads) are kept in memory indefinitely until a new job is created to trigger cleanup(). This can cause a significant memory leak if the server runs for a long time with occasional bursts of jobs.
To resolve this while preserving the API's ability to return 410 GONE / EXPIRED status, consider clearing the result payload (setting job.result = None) when the job is marked as expired.
| if job.status != GraphExtractJobStatus.EXPIRED and (expired_by_result_ttl or expired_by_pending_ttl): | |
| job.status = GraphExtractJobStatus.EXPIRED | |
| job.updated_at = now | |
| if job.status != GraphExtractJobStatus.EXPIRED and (expired_by_result_ttl or expired_by_pending_ttl): | |
| job.status = GraphExtractJobStatus.EXPIRED | |
| job.result = None | |
| job.updated_at = now |
There was a problem hiding this comment.
Fixed in 7850be7. Expiring a job now clears the result payload while preserving EXPIRED status, with regression coverage.
| with ThreadPoolExecutor(max_workers=worker_count) as executor: | ||
| proceeded_chunks = list( | ||
| executor.map(lambda chunk: self.extract_property_graph_by_llm(schema, chunk), chunks) | ||
| ) |
There was a problem hiding this comment.
Creating and destroying a ThreadPoolExecutor on every call of run() is inefficient because it spawns new OS threads and joins them on context exit, adding significant latency and CPU overhead under load.
Consider using a shared, module-level or class-level ThreadPoolExecutor with a fixed maximum number of workers to reuse idle threads.
There was a problem hiding this comment.
Not changed in this pass. The executor is intentionally request-scoped because max_parallel_chunks is request and config bounded and defaults low, and a shared executor would couple independent API calls and make per-request limits less predictable. The API-level job store controls background job concurrency separately.
| except ValueError as exc: | ||
| log.error("Graph extraction request failed validation: %s", exc) | ||
| raise HTTPException( | ||
| status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, | ||
| detail=_error("GRAPH_EXTRACT_INVALID_FLOW_OUTPUT", str(exc), "extract"), | ||
| ) from exc |
There was a problem hiding this comment.
Catching any ValueError and mapping it to 500 Internal Server Error with GRAPH_EXTRACT_INVALID_FLOW_OUTPUT is problematic. If a ValueError is raised due to client-side input validation (e.g., mismatched graph name and schema in apply_client_config), the API will incorrectly return a 500 error instead of a 400 Bad Request or 422 Unprocessable Entity.
Consider defining a custom exception class (e.g., FlowOutputValidationError) for LLM/flow output validation errors, and catch that specifically, while letting standard ValueError propagate as a client-side error.
There was a problem hiding this comment.
Fixed in 7850be7. Added FlowOutputValidationError for malformed workflow output and mapped plain ValueError to 400 client input errors, with API regression coverage.
There was a problem hiding this comment.
Pull request overview
This PR expands the HugeGraph LLM “graph extraction” API surface from a single synchronous endpoint into a service-backed API set that supports synchronous extraction, async job execution, graph import, and extract-and-import, while adding request-scoped HugeGraph connection configuration, chunk parallelism controls, and more structured validation/error handling.
Changes:
- Introduces service-layer implementations for graph extraction/import (schema normalization, client config handling/redaction, metadata/warnings shaping).
- Adds async extraction jobs (in-memory job store +
/graph/extract/jobsendpoints) and new import/extract-and-import endpoints. - Extends extraction pipeline to support
content_type(textvschunks) and request-bounded chunk-level parallelism, plus request-scoped HugeGraph client configuration propagation.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| hugegraph-llm/src/tests/utils/test_graph_index_utils.py | Adds regression coverage ensuring legacy scheduler invocation remains unchanged for existing extraction path. |
| hugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.py | Adds coverage for schema edge filtering robustness + parallel chunk extraction ordering/paths + malformed JSON failure behavior. |
| hugegraph-llm/src/tests/operators/llm_op/test_info_extract.py | Adds coverage for schema-shape normalization before regex-based extraction. |
| hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py | Adds coverage for import stats, ID mapping, schema id strategies, and normalized extraction compatibility. |
| hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.py | Updates expected behavior to “report and continue” on vertex creation failures. |
| hugegraph-llm/src/tests/operators/document_op/test_chunk_split.py | Adds coverage for paragraph splitting that preserves explicit short paragraph boundaries. |
| hugegraph-llm/src/tests/nodes/test_request_graph_config.py | Adds coverage that nodes pass request-scoped HugeGraph config through to operators/clients. |
| hugegraph-llm/src/tests/nodes/test_extract_node.py | Adds coverage that extraction node uses extract-LLM config and passes max chunk parallelism. |
| hugegraph-llm/src/tests/nodes/test_base_node.py | Adds coverage that unexpected operator exceptions become error statuses (no propagation). |
| hugegraph-llm/src/tests/models/llms/test_init_llm.py | Adds coverage for extract-LLM config fallback behavior across OpenAI/LiteLLM/Ollama. |
| hugegraph-llm/src/tests/flows/test_graph_extract_flow.py | Adds coverage for flow prepare/build behavior around split_type/content_type/max_parallel_chunks and state resets. |
| hugegraph-llm/src/tests/document/test_graph_extract_configurable_split.py | Extends expected extraction response to include max_parallel_chunks metadata. |
| hugegraph-llm/src/tests/api/test_graph_import_api.py | Adds end-to-end API coverage for /graph/import and /graph/extract-and-import, client_config handling, and embedding update option. |
| hugegraph-llm/src/tests/api/test_graph_extract_jobs.py | Adds end-to-end API coverage for job creation/status/result/cancel/expiry/capacity semantics. |
| hugegraph-llm/src/tests/api/test_graph_extract_api.py | Updates API contract tests for new request shapes, structured errors, concurrency isolation, and route registration. |
| hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py | Adds request-scoped HugeGraph client creation via optional graph_config overrides. |
| hugegraph-llm/src/hugegraph_llm/state/ai_state.py | Extends workflow input/state with content_type/max_parallel_chunks and graph_config fields + reset/setup handling. |
| hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py | Adds service-layer sync extraction + import logic (schema normalization, validation, metadata, redaction). |
| hugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.py | Adds in-memory async job store with worker threads, TTL/expiry, and capacity semantics. |
| hugegraph-llm/src/hugegraph_llm/services/init.py | Adds services package initializer. |
| hugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.py | Adds chunk-level parallelism + stricter invalid JSON handling + schema robustness improvements. |
| hugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.py | Adds schema-shape adapter to support both legacy and newer schema formats. |
| hugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.py | Adds request-scoped graph_config support so semantic index targets the request graph. |
| hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.py | Adds graph_config support for request-scoped HugeGraph connection defaults. |
| hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py | Adds request-scoped client config + detailed import_result stats/errors + continues on element creation failures. |
| hugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.py | Refines paragraph splitting to respect explicit paragraph boundaries before recursive splitting. |
| hugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.py | Switches extraction to use extract-LLM config and passes max_parallel_chunks to operator. |
| hugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.py | Passes request-scoped graph_config into semantic index operator. |
| hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.py | Ensures schema fetch uses request-scoped graph_config when available. |
| hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.py | Uses request-scoped graph_config when constructing HugeGraph client. |
| hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.py | Passes request-scoped graph_config into Commit2Graph operator. |
| hugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.py | Adds bypass path for pre-split chunks content_type (no re-splitting). |
| hugegraph-llm/src/hugegraph_llm/nodes/base_node.py | Changes node execution to catch all exceptions and return error status (instead of re-raising unexpected ones). |
| hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py | Adds extract-LLM fallback logic to reuse chat config unless explicit extract config is provided. |
| hugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.py | Adds graph_config propagation into embedding update flow. |
| hugegraph-llm/src/hugegraph_llm/flows/import_graph_data.py | Adds graph_config propagation into import flow. |
| hugegraph-llm/src/hugegraph_llm/flows/graph_extract.py | Adds content_type/max_parallel_chunks support + keeps split_type configurable + returns richer extraction output payload. |
| hugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.py | Adjusts router registration ordering to avoid duplicate route registration. |
| hugegraph-llm/src/hugegraph_llm/config/llm_config.py | Adds defaults/limits for graph extraction chunk parallelism. |
| hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.py | Adds structured error and job/import response models. |
| hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.py | Adds content_type/content request model, schema validation hardening, parallelism validation, and import request models. |
| hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.py | Adds service-backed routes for extract/jobs/import/extract-and-import + structured error handling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| async def custom_route_handler(request: Request): | ||
| try: | ||
| return await original_route_handler(request) | ||
| except RequestValidationError as exc: | ||
| return JSONResponse( | ||
| status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, | ||
| content={ | ||
| "detail": _error( | ||
| "GRAPH_EXTRACT_VALIDATION_ERROR", | ||
| str(exc), | ||
| "request", | ||
| ) | ||
| }, | ||
| ) | ||
|
|
There was a problem hiding this comment.
Fixed in 7850be7. Validation responses now use sanitized loc, msg, and type summaries from exc.errors() and omit raw input values. Added coverage for password and URL not being echoed.
| schema = data.get("schema") | ||
| vertices = data.get("vertices", []) | ||
| edges = data.get("edges", []) | ||
| if not vertices and not edges: | ||
| vertices = data.get("vertices", []) or [] | ||
| edges = data.get("edges", []) or [] | ||
| triples = data.get("triples", []) or [] | ||
| if not vertices and not edges and not triples: |
There was a problem hiding this comment.
Fixed in 7850be7. Schema mode now rejects triples input instead of silently ignoring it, and schema-free mode rejects vertices or edges. Added regression coverage.
| items = [text for text in value if isinstance(text, str) and text.strip()] | ||
| if len(items) != 1: | ||
| raise ValueError("texts alias must contain exactly one non-empty string") | ||
| return items[0].strip() |
There was a problem hiding this comment.
Not changed as multi-text input. The synchronous extract contract is one document per request; the deprecated texts alias is kept only as a single-document compatibility bridge. Multi-document extraction should be modeled as multiple API calls so it does not become hidden batch semantics in the sync endpoint. The PR description has been clarified accordingly.
| try: | ||
| return extract_service.extract_sync(req) | ||
| except ValueError as exc: | ||
| log.error("Graph extraction request failed validation: %s", exc) |
There was a problem hiding this comment.
Fixed in 7850be7. The extraction handler now logs flow-output failures separately from input ValueError paths.
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (1)
hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py (1)
41-49: ⚡ Quick win为
get_hg_client补充类型标注以提升 mypy 覆盖建议给
graph_config和返回值增加显式类型,避免调用方传入非映射对象时静态检查失效。As per coding guidelines, "Type-check Python code with mypy".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py` around lines 41 - 49, Add explicit type annotations to get_hg_client: change the signature to accept graph_config: Optional[Mapping[str, Any]] = None and annotate the return type as -> PyHugeClient; update imports to include typing names (Optional, Mapping, Any) so mypy can validate callers that might pass non-mapping objects, and keep use of huge_settings and PyHugeClient unchanged inside the function.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hugegraph-llm/src/hugegraph_llm/flows/graph_extract.py`:
- Around line 48-60: Validate max_parallel_chunks before assigning it to
prepared_input.max_parallel_chunks: ensure the passed max_parallel_chunks is an
integer and >= 1, otherwise raise a ValueError with a clear message. Add this
check in the same function where max_parallel_chunks is accepted (the
prepare/entry function that currently sets prepared_input.max_parallel_chunks),
and only assign prepared_input.max_parallel_chunks after validation to prevent
downstream runtime errors from 0, negative, or non-integer values.
In `@hugegraph-llm/src/hugegraph_llm/nodes/base_node.py`:
- Around line 78-81: The except block currently logs and returns the full
formatted error (via _format_node_err) which includes stacktrace and node
details; instead, keep the detailed output only in logs by calling
_format_node_err(self, exc) and passing its result to log.error, but return a
sanitized CStatus payload that contains only non-sensitive info (e.g., node
identifier self.name or self.__class__.__name__, the exception type
exc.__class__.__name__ and a short message str(exc) without traceback). Update
the return statement to return CStatus(-1, sanitized_msg) while leaving
log.error(err_msg) unchanged; reference symbols: _format_node_err, log.error,
CStatus, self, exc.
In
`@hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py`:
- Around line 64-67: 当 schema 为空时目前直接调用 schema_free_mode(triples) 会静默丢弃传入的
vertices/edges;请在该分支显式校验输入:如果调用方传入了 vertices 或 edges 而 triples 为空,则抛出/返回错误(例如
ValueError 或自定义异常)并记录明确错误信息;同时在调用 schema_free_mode(triples) 前确保只接受 triples(拒绝包含
vertices/edges 的混合输入)。在修复中请修改 commit_to_hugegraph.py 中处理 schema 分支的逻辑,参考函数/变量名
schema_free_mode、triples、vertices、edges 以定位并更新校验与错误处理。
In `@hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.py`:
- Around line 34-41: When handling the connection branch in schema_manager.py,
validate that connection contains non-empty values for "url", "user", "pwd", and
"graphspace" before passing them to PyHugeClient: if any are missing or None,
either fall back to the corresponding huge_settings values or raise a clear
validation error; update the logic around
connection.get("url"/"user"/"pwd"/"graphspace") (and the code path that
constructs/uses PyHugeClient) to perform this check/fallback so we never pass
None into PyHugeClient and failure points remain explicit.
In `@hugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.py`:
- Around line 158-170: submit_job currently inserts the job into self._jobs in
create() and then calls self._queue.put_nowait; if put_nowait raises queue.Full
the method just raises and the job remains stuck in PENDING (a "zombie"), so
modify submit_job to rollback the job when queue.put_nowait fails: in the except
queue.Full block reacquire self._lock (or use a finally with the existing lock)
and remove the job from self._jobs or set its status to a terminal state (e.g.,
FAILED) and record an error message, ensuring you update any counters related to
max_jobs; reference submit_job, _jobs, _queue, _lock,
GraphExtractJobStatus.PENDING and create() when locating the change.
- Around line 152-156: cleanup() currently calls expire_jobs() (which acquires
the lock) but then iterates and deletes entries from self._jobs outside the
lock, risking concurrent modification; fix by performing the expired_job_ids
computation and deletion while holding the same lock used by expire_jobs()
(i.e., protect access to self._jobs with the same lock guard), so update
cleanup() to acquire the lock, call expire_jobs() or inline its logic under the
lock, then build expired_job_ids from self._jobs and delete them while still
holding the lock to prevent race conditions with workers updating job statuses.
In `@hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py`:
- Around line 306-308: The current flow calls
self.scheduler.schedule_flow(FlowName.UPDATE_VID_EMBEDDINGS,
graph_config=graph_config) when request.options.update_vid_embeddings is true
and if that call fails the whole request returns 500 causing duplicate graph
writes on retry; wrap the schedule_flow call in a try/except around the block
that sets updated_embeddings, catch any exception raised by schedule_flow, log a
warning (including exception details) instead of re-raising, set
updated_embeddings appropriately (e.g., leave True only if scheduling succeeded
or set a separate flag like embedding_scheduled=False), and ensure the handler
returns a successful/import-completed response with a partial/warning status
rather than throwing so the graph write remains durable and retries won’t
reapply writes.
In `@hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py`:
- Around line 44-48: Replace the "x = graph_config.get(key) or
huge_settings.foo" pattern which treats explicit empty strings as missing;
instead, preserve explicit values by checking key presence: use
"graph_config['gs'] if 'gs' in graph_config else huge_settings.graph_space" (and
similarly for "url","graph","user","pwd" using graph_config.__contains__(key) or
'key in graph_config') so an explicit "" for graphspace (gs) isn't overwritten
and request-level config is honored; update the assignment sites where
graph_config and huge_settings are combined accordingly.
In `@hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py`:
- Around line 307-590: The test file has grown too large; split tests into
focused files to keep each under 600 lines. Create new test modules grouping
related cases (e.g., tests for load_into_graph, schema_free_mode,
property_type_validation) and move the corresponding test functions (e.g.,
test_load_into_graph,
test_load_into_graph_returns_actual_created_and_skipped_counts,
test_load_into_graph_with_data_type_validation_success,
test_load_into_graph_with_data_type_validation_failure,
test_load_into_graph_maps_llm_vertex_ids_to_created_vertex_ids,
test_load_into_graph_uses_explicit_customize_string_ids,
test_load_into_graph_accepts_normalized_extraction_without_item_type) into the
appropriate new files; ensure you preserve and adjust imports/fixtures
(Commit2Graph test setup, self.commit2graph, self.schema, PropertyGraphExtract)
and update any patched target strings so tests still reference
hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph; run the test suite to
verify nothing breaks and remove the moved tests from the original file.
In `@hugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.py`:
- Around line 1099-1197: This test module has grown past the 600-line guideline;
split it into focused test files by grouping related tests (e.g., move tests
exercising parallel
behavior—test_run_with_parallel_chunks_preserves_input_order,
test_run_with_single_parallel_chunk_uses_serial_path,
test_run_with_one_chunk_reduces_parallelism_to_direct_extract—into a new
test_run_parallel file), move JSON/error handling tests
(test_run_raises_when_chunk_output_is_malformed_json,
test_run_falls_back_to_default_parallelism_for_invalid_context_value) into a
test_json_error_handling file, and keep shared setup for PropertyGraphExtract
and fixtures (self.schema, self.chunks, self.llm_responses, self.mock_llm) in a
common conftest or a base TestCase to import; update imports and test discovery
accordingly so references to PropertyGraphExtract and mocked methods
(extract_property_graph_by_llm) continue to work.
---
Nitpick comments:
In `@hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py`:
- Around line 41-49: Add explicit type annotations to get_hg_client: change the
signature to accept graph_config: Optional[Mapping[str, Any]] = None and
annotate the return type as -> PyHugeClient; update imports to include typing
names (Optional, Mapping, Any) so mypy can validate callers that might pass
non-mapping objects, and keep use of huge_settings and PyHugeClient unchanged
inside the function.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 05650d2f-ea8e-4787-9b3d-ca654a30841d
📒 Files selected for processing (42)
hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.pyhugegraph-llm/src/hugegraph_llm/config/llm_config.pyhugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.pyhugegraph-llm/src/hugegraph_llm/flows/graph_extract.pyhugegraph-llm/src/hugegraph_llm/flows/import_graph_data.pyhugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.pyhugegraph-llm/src/hugegraph_llm/models/llms/init_llm.pyhugegraph-llm/src/hugegraph_llm/nodes/base_node.pyhugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.pyhugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.pyhugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.pyhugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.pyhugegraph-llm/src/hugegraph_llm/services/__init__.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_service.pyhugegraph-llm/src/hugegraph_llm/state/ai_state.pyhugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.pyhugegraph-llm/src/tests/api/test_graph_extract_api.pyhugegraph-llm/src/tests/api/test_graph_extract_jobs.pyhugegraph-llm/src/tests/api/test_graph_import_api.pyhugegraph-llm/src/tests/document/test_graph_extract_configurable_split.pyhugegraph-llm/src/tests/flows/test_graph_extract_flow.pyhugegraph-llm/src/tests/models/llms/test_init_llm.pyhugegraph-llm/src/tests/nodes/test_base_node.pyhugegraph-llm/src/tests/nodes/test_extract_node.pyhugegraph-llm/src/tests/nodes/test_request_graph_config.pyhugegraph-llm/src/tests/operators/document_op/test_chunk_split.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.pyhugegraph-llm/src/tests/operators/llm_op/test_info_extract.pyhugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.pyhugegraph-llm/src/tests/utils/test_graph_index_utils.py
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._check_property_data_type") | ||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph(self, mock_handle_graph_creation, mock_check_property_data_type): | ||
| """Test load_into_graph method.""" | ||
| # Setup mocks | ||
| mock_handle_graph_creation.return_value = MagicMock(id="vertex_id") | ||
| mock_check_property_data_type.return_value = True | ||
|
|
||
| # Create vertices with proper data types according to schema | ||
| vertices = [ | ||
| {"label": "person", "properties": {"name": "Tom Hanks", "age": 67}}, | ||
| {"label": "movie", "properties": {"title": "Forrest Gump", "year": 1994}}, | ||
| ] | ||
|
|
||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {"role": "Forrest Gump"}, | ||
| "outV": "person:Tom Hanks", # Use the format expected by the implementation | ||
| "inV": "movie:Forrest Gump", # Use the format expected by the implementation | ||
| } | ||
| ] | ||
|
|
||
| # Call the method | ||
| self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| # Verify that _handle_graph_creation was called for each vertex and edge | ||
| self.assertEqual(mock_handle_graph_creation.call_count, 3) # 2 vertices + 1 edge | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._check_property_data_type") | ||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_returns_actual_created_and_skipped_counts( | ||
| self, mock_handle_graph_creation, mock_check_property_data_type | ||
| ): | ||
| """Test import stats reflect actual created and skipped graph elements.""" | ||
| mock_handle_graph_creation.side_effect = [ | ||
| MagicMock(id="person:Tom Hanks"), | ||
| None, | ||
| ] | ||
| mock_check_property_data_type.return_value = True | ||
| vertices = [ | ||
| {"label": "person", "properties": {"name": "Tom Hanks", "age": 67}}, | ||
| {"label": "unknown", "properties": {"name": "Ignored"}}, | ||
| ] | ||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {"role": "Forrest Gump"}, | ||
| "outV": "person:Tom Hanks", | ||
| "inV": "movie:Forrest Gump", | ||
| } | ||
| ] | ||
|
|
||
| result = self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| self.assertEqual(result["vertices_attempted"], 2) | ||
| self.assertEqual(result["vertices_created"], 1) | ||
| self.assertEqual(result["vertices_skipped"], 1) | ||
| self.assertEqual(result["edges_attempted"], 1) | ||
| self.assertEqual(result["edges_created"], 0) | ||
| self.assertEqual(result["edges_skipped"], 1) | ||
| self.assertIn("VertexLabel unknown not found in schema", result["errors"]) | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_with_data_type_validation_success(self, mock_handle_graph_creation): | ||
| """Test load_into_graph method with successful data type validation.""" | ||
| # Setup mocks | ||
| mock_handle_graph_creation.return_value = MagicMock(id="vertex_id") | ||
|
|
||
| # Create vertices with correct data types matching schema expectations | ||
| vertices = [ | ||
| {"label": "person", "properties": {"name": "Tom Hanks", "age": 67}}, # age: INT -> int | ||
| {"label": "movie", "properties": {"title": "Forrest Gump", "year": 1994}}, # year: INT -> int | ||
| ] | ||
|
|
||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {"role": "Forrest Gump"}, # role: TEXT -> str | ||
| "outV": "person:Tom Hanks", | ||
| "inV": "movie:Forrest Gump", | ||
| } | ||
| ] | ||
|
|
||
| # Call the method - should succeed with correct data types | ||
| self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| # Verify that _handle_graph_creation was called for each vertex and edge | ||
| self.assertEqual(mock_handle_graph_creation.call_count, 3) # 2 vertices + 1 edge | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_maps_llm_vertex_ids_to_created_vertex_ids(self, mock_handle_graph_creation): | ||
| """Test edges use server-created vertex ids when LLM ids differ.""" | ||
| mock_handle_graph_creation.side_effect = [ | ||
| MagicMock(id="1:Tom Hanks"), | ||
| MagicMock(id="2:Forrest Gump"), | ||
| MagicMock(id="edge_id"), | ||
| ] | ||
|
|
||
| vertices = [ | ||
| { | ||
| "id": "person:Tom Hanks", | ||
| "label": "person", | ||
| "properties": {"name": "Tom Hanks", "age": 67}, | ||
| }, | ||
| { | ||
| "id": "movie:Forrest Gump", | ||
| "label": "movie", | ||
| "properties": {"title": "Forrest Gump", "year": 1994}, | ||
| }, | ||
| ] | ||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {"role": "Forrest Gump"}, | ||
| "outV": "person:Tom Hanks", | ||
| "inV": "movie:Forrest Gump", | ||
| } | ||
| ] | ||
|
|
||
| self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| self.assertEqual(vertices[0]["id"], "1:Tom Hanks") | ||
| self.assertEqual(vertices[1]["id"], "2:Forrest Gump") | ||
| mock_handle_graph_creation.assert_any_call( | ||
| self.commit2graph.client.graph().addEdge, | ||
| "acted_in", | ||
| "1:Tom Hanks", | ||
| "2:Forrest Gump", | ||
| {"role": "Forrest Gump"}, | ||
| ) | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_uses_explicit_customize_string_ids(self, mock_handle_graph_creation): | ||
| """Test custom string ids are passed to HugeGraph when schema requires them.""" | ||
| mock_handle_graph_creation.side_effect = [ | ||
| MagicMock(id="Tom Hanks"), | ||
| MagicMock(id="Forrest Gump"), | ||
| MagicMock(id="edge_id"), | ||
| ] | ||
| schema = { | ||
| "propertykeys": [ | ||
| {"name": "name", "data_type": "TEXT", "cardinality": "SINGLE"}, | ||
| {"name": "title", "data_type": "TEXT", "cardinality": "SINGLE"}, | ||
| ], | ||
| "vertexlabels": [ | ||
| { | ||
| "id": 7, | ||
| "name": "person", | ||
| "id_strategy": "CUSTOMIZE_STRING", | ||
| "primary_keys": ["name"], | ||
| "properties": ["name"], | ||
| "nullable_keys": [], | ||
| }, | ||
| { | ||
| "id": 8, | ||
| "name": "movie", | ||
| "id_strategy": "CUSTOMIZE_STRING", | ||
| "primary_keys": ["title"], | ||
| "properties": ["title"], | ||
| "nullable_keys": [], | ||
| }, | ||
| ], | ||
| "edgelabels": [{"name": "acted_in", "properties": [], "source_label": "person", "target_label": "movie"}], | ||
| } | ||
| vertices = [ | ||
| {"id": "Tom Hanks", "label": "person", "properties": {"name": "Tom Hanks"}}, | ||
| {"id": "Forrest Gump", "label": "movie", "properties": {"title": "Forrest Gump"}}, | ||
| ] | ||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {}, | ||
| "outV": "Tom Hanks", | ||
| "inV": "Forrest Gump", | ||
| } | ||
| ] | ||
|
|
||
| self.commit2graph.load_into_graph(vertices, edges, schema) | ||
|
|
||
| mock_handle_graph_creation.assert_any_call( | ||
| self.commit2graph.client.graph().addVertex, | ||
| "person", | ||
| {"name": "Tom Hanks"}, | ||
| id="Tom Hanks", | ||
| ) | ||
| mock_handle_graph_creation.assert_any_call( | ||
| self.commit2graph.client.graph().addVertex, | ||
| "movie", | ||
| {"title": "Forrest Gump"}, | ||
| id="Forrest Gump", | ||
| ) | ||
| mock_handle_graph_creation.assert_any_call( | ||
| self.commit2graph.client.graph().addEdge, | ||
| "acted_in", | ||
| "Tom Hanks", | ||
| "Forrest Gump", | ||
| {}, | ||
| ) | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_accepts_normalized_extraction_without_item_type(self, mock_handle_graph_creation): | ||
| """Test normalized LLM output without type fields can be committed.""" | ||
| mock_handle_graph_creation.side_effect = [ | ||
| MagicMock(id="1:Tom Hanks"), | ||
| MagicMock(id="2:Forrest Gump"), | ||
| MagicMock(id="edge_id"), | ||
| ] | ||
| llm_output = """{ | ||
| "vertices": [ | ||
| { | ||
| "id": "person:Tom Hanks", | ||
| "label": "person", | ||
| "properties": { | ||
| "name": "Tom Hanks", | ||
| "age": 67 | ||
| } | ||
| }, | ||
| { | ||
| "id": "movie:Forrest Gump", | ||
| "label": "movie", | ||
| "properties": { | ||
| "title": "Forrest Gump", | ||
| "year": 1994 | ||
| } | ||
| } | ||
| ], | ||
| "edges": [ | ||
| { | ||
| "label": "acted_in", | ||
| "outV": "person:Tom Hanks", | ||
| "outVLabel": "person", | ||
| "inV": "movie:Forrest Gump", | ||
| "inVLabel": "movie", | ||
| "properties": { | ||
| "role": "Forrest Gump" | ||
| } | ||
| } | ||
| ] | ||
| }""" | ||
|
|
||
| items = PropertyGraphExtract(llm=MagicMock())._extract_and_filter_label(self.schema, llm_output) | ||
| vertices = [item for item in items if item["type"] == "vertex"] | ||
| edges = [item for item in items if item["type"] == "edge"] | ||
| self.assertEqual(edges[0]["outV"], "1:Tom Hanks") | ||
| self.assertEqual(edges[0]["inV"], "2:Forrest Gump") | ||
|
|
||
| self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| mock_handle_graph_creation.assert_any_call( | ||
| self.commit2graph.client.graph().addEdge, | ||
| "acted_in", | ||
| "1:Tom Hanks", | ||
| "2:Forrest Gump", | ||
| {"role": "Forrest Gump"}, | ||
| ) | ||
|
|
||
| @patch("hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph.Commit2Graph._handle_graph_creation") | ||
| def test_load_into_graph_with_data_type_validation_failure(self, mock_handle_graph_creation): | ||
| """Test load_into_graph method with data type validation failure.""" | ||
| # Setup mocks | ||
| mock_handle_graph_creation.return_value = MagicMock(id="vertex_id") | ||
|
|
||
| # Create vertices with incorrect data types (strings for INT fields) | ||
| vertices = [ | ||
| {"label": "person", "properties": {"name": "Tom Hanks", "age": "67"}}, # age should be int, not str | ||
| {"label": "movie", "properties": {"title": "Forrest Gump", "year": "1994"}}, # year should be int, not str | ||
| ] | ||
|
|
||
| edges = [ | ||
| { | ||
| "label": "acted_in", | ||
| "properties": {"role": "Forrest Gump"}, | ||
| "outV": "person:Tom Hanks", | ||
| "inV": "movie:Forrest Gump", | ||
| } | ||
| ] | ||
|
|
||
| # Call the method - should skip vertices due to data type validation failure | ||
| self.commit2graph.load_into_graph(vertices, edges, self.schema) | ||
|
|
||
| # Verify that _handle_graph_creation was called only for the edge (vertices were skipped) | ||
| self.assertEqual(mock_handle_graph_creation.call_count, 1) # Only 1 edge, vertices skipped | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy lift
建议拆分该测试文件,避免继续扩大单文件体积。
当前文件已超过 600 行(可见 Line 770 之后仍有内容),本次在 Line 307-590 继续增加大量用例会进一步提高维护和回归定位成本。建议按主题拆分为多个测试文件(如 load_into_graph、schema_free_mode、property_type_validation)。
As per coding guidelines, hugegraph-llm/**/*.py: Keep each Python file under 600 lines for maintainability.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py`
around lines 307 - 590, The test file has grown too large; split tests into
focused files to keep each under 600 lines. Create new test modules grouping
related cases (e.g., tests for load_into_graph, schema_free_mode,
property_type_validation) and move the corresponding test functions (e.g.,
test_load_into_graph,
test_load_into_graph_returns_actual_created_and_skipped_counts,
test_load_into_graph_with_data_type_validation_success,
test_load_into_graph_with_data_type_validation_failure,
test_load_into_graph_maps_llm_vertex_ids_to_created_vertex_ids,
test_load_into_graph_uses_explicit_customize_string_ids,
test_load_into_graph_accepts_normalized_extraction_without_item_type) into the
appropriate new files; ensure you preserve and adjust imports/fixtures
(Commit2Graph test setup, self.commit2graph, self.schema, PropertyGraphExtract)
and update any patched target strings so tests still reference
hugegraph_llm.operators.hugegraph_op.commit_to_hugegraph; run the test suite to
verify nothing breaks and remove the moved tests from the original file.
Source: Coding guidelines
There was a problem hiding this comment.
Not changed in this follow-up. This is a maintainability refactor, not a behavioral defect, and splitting these existing test modules would create a large file-move diff unrelated to the review fixes. I kept this pass scoped to runtime and API correctness and added focused regression tests beside the existing coverage.
There was a problem hiding this comment.
@LRriver, understood — keeping this PR scoped to behavioral correctness is a reasonable call. Would you like me to open a follow-up GitHub issue to track the test file split (load_into_graph, schema_free_mode, property_type_validation groupings) so it doesn't get lost?
(∩˃o˂∩)♡
| def test_run_with_parallel_chunks_preserves_input_order(self): | ||
| """Test parallel chunk extraction keeps deterministic aggregation order.""" | ||
| extractor = PropertyGraphExtract(llm=self.mock_llm) | ||
|
|
||
| def fake_extract(_schema, chunk): | ||
| if chunk == "slow chunk": | ||
| time.sleep(0.02) | ||
| name = "Alice" if chunk == "slow chunk" else "Bob" | ||
| return json.dumps( | ||
| { | ||
| "vertices": [ | ||
| { | ||
| "type": "vertex", | ||
| "label": "person", | ||
| "properties": {"name": name}, | ||
| } | ||
| ], | ||
| "edges": [], | ||
| } | ||
| ) | ||
|
|
||
| extractor.extract_property_graph_by_llm = MagicMock(side_effect=fake_extract) | ||
| context = { | ||
| "schema": self.schema, | ||
| "chunks": ["slow chunk", "fast chunk"], | ||
| "max_parallel_chunks": 8, | ||
| } | ||
|
|
||
| result = extractor.run(context) | ||
|
|
||
| self.assertEqual([vertex["properties"]["name"] for vertex in result["vertices"]], ["Alice", "Bob"]) | ||
| self.assertEqual(result["call_count"], 2) | ||
| self.assertEqual(result["max_parallel_chunks"], 2) | ||
| self.assertEqual(extractor.extract_property_graph_by_llm.call_count, 2) | ||
|
|
||
| @patch("hugegraph_llm.operators.llm_op.property_graph_extract.ThreadPoolExecutor") | ||
| def test_run_with_single_parallel_chunk_uses_serial_path(self, mock_executor): | ||
| """Test max_parallel_chunks=1 keeps the existing serial extraction path.""" | ||
| extractor = PropertyGraphExtract(llm=self.mock_llm) | ||
| extractor.extract_property_graph_by_llm = MagicMock(side_effect=self.llm_responses) | ||
| context = { | ||
| "schema": self.schema, | ||
| "chunks": self.chunks, | ||
| "max_parallel_chunks": 1, | ||
| } | ||
|
|
||
| result = extractor.run(context) | ||
|
|
||
| mock_executor.assert_not_called() | ||
| self.assertEqual(extractor.extract_property_graph_by_llm.call_count, 2) | ||
| self.assertEqual(result["call_count"], 2) | ||
|
|
||
| @patch("hugegraph_llm.operators.llm_op.property_graph_extract.ThreadPoolExecutor") | ||
| def test_run_with_one_chunk_reduces_parallelism_to_direct_extract(self, mock_executor): | ||
| """Test one pre-split chunk is extracted directly even when parallelism is higher.""" | ||
| extractor = PropertyGraphExtract(llm=self.mock_llm) | ||
| extractor.extract_property_graph_by_llm = MagicMock(return_value=self.llm_responses[0]) | ||
| context = { | ||
| "schema": self.schema, | ||
| "chunks": [self.chunks[0]], | ||
| "max_parallel_chunks": 4, | ||
| } | ||
|
|
||
| result = extractor.run(context) | ||
|
|
||
| mock_executor.assert_not_called() | ||
| self.assertEqual(extractor.extract_property_graph_by_llm.call_count, 1) | ||
| self.assertEqual(result["call_count"], 1) | ||
| self.assertEqual(result["max_parallel_chunks"], 1) | ||
|
|
||
| def test_run_raises_when_chunk_output_is_malformed_json(self): | ||
| """Test flow execution fails instead of silently dropping malformed chunk output.""" | ||
| extractor = PropertyGraphExtract(llm=self.mock_llm) | ||
| extractor.extract_property_graph_by_llm = MagicMock(return_value='{"vertices": [], "edges": [}') | ||
| context = { | ||
| "schema": self.schema, | ||
| "chunks": ["malformed output chunk"], | ||
| } | ||
|
|
||
| with self.assertRaisesRegex(ValueError, "Invalid property graph JSON") as error: | ||
| extractor.run(context) | ||
|
|
||
| self.assertIsInstance(error.exception.__cause__, json.JSONDecodeError) | ||
|
|
||
| def test_run_falls_back_to_default_parallelism_for_invalid_context_value(self): | ||
| """Test invalid operator-level parallelism input does not fail before extraction.""" | ||
| extractor = PropertyGraphExtract(llm=self.mock_llm, max_parallel_chunks=2) | ||
| extractor.extract_property_graph_by_llm = MagicMock(return_value=self.llm_responses[0]) | ||
| context = { | ||
| "schema": self.schema, | ||
| "chunks": [self.chunks[0]], | ||
| "max_parallel_chunks": "not-an-int", | ||
| } | ||
|
|
||
| result = extractor.run(context) | ||
|
|
||
| self.assertEqual(result["call_count"], 1) | ||
| self.assertEqual(result["max_parallel_chunks"], 1) | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy lift
建议将该测试模块拆分,避免超长文件继续膨胀。
该文件已明显超过 600 行(Line 1241 附近为文件结尾),本次新增并发与异常路径测试后,后续维护成本会进一步上升。建议按主题拆分(如 run_parallel、json_error_handling、id_normalization)。
As per coding guidelines, hugegraph-llm/**/*.py: Keep each Python file under 600 lines for maintainability.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@hugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.py`
around lines 1099 - 1197, This test module has grown past the 600-line
guideline; split it into focused test files by grouping related tests (e.g.,
move tests exercising parallel
behavior—test_run_with_parallel_chunks_preserves_input_order,
test_run_with_single_parallel_chunk_uses_serial_path,
test_run_with_one_chunk_reduces_parallelism_to_direct_extract—into a new
test_run_parallel file), move JSON/error handling tests
(test_run_raises_when_chunk_output_is_malformed_json,
test_run_falls_back_to_default_parallelism_for_invalid_context_value) into a
test_json_error_handling file, and keep shared setup for PropertyGraphExtract
and fixtures (self.schema, self.chunks, self.llm_responses, self.mock_llm) in a
common conftest or a base TestCase to import; update imports and test discovery
accordingly so references to PropertyGraphExtract and mocked methods
(extract_property_graph_by_llm) continue to work.
Source: Coding guidelines
There was a problem hiding this comment.
Not changed in this follow-up. This is a maintainability refactor, not a behavioral defect, and splitting these existing test modules would create a large file-move diff unrelated to the review fixes. I kept this pass scoped to runtime and API correctness and added focused regression tests beside the existing coverage.
There was a problem hiding this comment.
@LRriver, understood — keeping this PR scoped to runtime and API correctness is a reasonable call, and a file-split would indeed be an unrelated structural change better handled separately. Would you like me to open a follow-up issue to track the refactor (splitting test_property_graph_extract.py into focused sub-modules) so it doesn't get lost?
(´• ω •) (´・_・)
imbajin
left a comment
There was a problem hiding this comment.
Review 结论
不建议当前合并,需要修改后再进入下一轮 review。
总体评分:6.2 / 10
评分拆分:
| 维度 | 评分 | 结论 |
|---|---|---|
| 方案设计 | 6.5 / 10 | 分层方向正确,但写入 API 的目标图安全边界不足,异步 job 方案仍偏 demo/runtime-local。 |
| 逻辑执行 | 5.5 / 10 | 主路径能跑通,但兼容性、queue 失败状态、导入输入边界存在明显问题。 |
| 代码质量 | 6.5 / 10 | service 拆分提升了可测性,但 API handler 过长,异常类型和状态码映射不够清晰。 |
| 测试有效性 | 6.0 / 10 | 测试数量较多、CI 通过,但缺少几个关键回归/安全用例。 |
| 用户易用性 | 5.5 / 10 | 新 API 能力更丰富,但旧协议兼容破坏、写入目标隐式 fallback、job 限制未显式暴露。 |
Review 上下文
该 PR 扩展 graph extraction API:同步抽取、异步任务、graph import、extract-and-import、content_type/content、request-scoped HugeGraph config、chunk 并行抽取、结构化错误响应等。
当前 CI 显示通过,且新增测试覆盖了 API、service、flow、operator、LLM 初始化和部分 HugeGraph 边界场景。但 CI 通过不能覆盖下面几个 public API 兼容性和写入安全问题。
本轮重点检查范围:
- API contract 和旧协议兼容性。
- 抽取、导入、extract-and-import 的执行路径。
- 异步 job 生命周期和异常状态。
- HugeGraph 写入目标和 request-local config 的安全性。
- 测试是否覆盖真实用户会踩到的边界场景。
P0 / 必须修复
1. /graph/extract 破坏了旧协议中 texts: List[str] 的兼容性
PR 描述里说保持之前同步 POST /graph/extract 的 wire shape 兼容,但当前实现并没有做到。
旧模型支持:
{
"texts": ["text A", "text B"],
"schema": { ... }
}新模型把 texts 作为 deprecated alias 处理,并在 _normalize_legacy_texts() 中要求 list 里必须 exactly one non-empty string。也就是说旧客户端只要传多个文本,就会直接 422。
这不是小的行为变化,而是 public API breaking change。尤其这类图抽取接口很自然会把多个段落/chunks 作为 list 传入。
需要二选一:
- 保持兼容:
texts: List[str]继续接受,并映射到等价的 chunk 输入或旧的 multi-text 行为。 - 明确做破坏性变更:加版本化 API、迁移说明、release note,并在测试中显式断言旧协议被拒绝是预期行为。
当前 PR 不能一边声称兼容,一边拒绝旧的合法请求。
建议补测试:
def test_graph_extract_legacy_texts_list_is_still_supported():
response = client.post(
"/graph/extract",
json={"texts": ["a", "b"], "schema": VALID_SCHEMA},
)
assert response.status_code == 200或者如果决定 breaking change,则测试和文档都必须明确说明。
2. /graph/import 和 /graph/extract-and-import 在 inline schema 下会隐式写入默认 HugeGraph graph
当前 GraphImportRequest 允许 inline schema,并且 client_config 是 optional。导入时:
- inline schema 被
normalize_schema()变成 JSON string。 apply_client_config(client_config=None, schema=inline_schema, align_graph_with_schema=True)返回None。Commit2Graph(graph_config=None)fallback 到全局huge_settings.graph_name/user/pwd/space。- 用户只要设置
write_to_graph=true,数据就会写入服务默认 graph。
这个行为对写接口不安全。
问题点:
- 请求体里看不出最终写入哪个 graph。
- 多租户/生产环境里容易误写默认 graph。
write_to_graph=true只能表示“确认写入”,不能替代“明确写入目标”。- inline schema 是用户提供的 schema,不等于目标 graph 标识。
建议修改为:
- 对
/graph/import:如果schema是 inline JSON,则必须要求client_config.graph或独立的target_graph字段。 - 对
/graph/extract-and-import:即使抽取使用 inline schema,也必须显式指定 import target graph。 - response/meta 中返回最终使用的 graph、graphspace、user,密码继续脱敏。
- 不允许写 API 静默 fallback 到全局默认 graph,除非 API contract 明确声明并有显式
use_default_graph=true之类的确认参数。
建议补测试:
def test_graph_import_rejects_inline_schema_without_explicit_target_graph():
response = client.post(
"/graph/import",
json={
"schema": VALID_SCHEMA,
"data": VALID_DATA,
"write_to_graph": True,
},
)
assert response.status_code == 4003. async job queue full 时会留下不会执行的 PENDING job
当前创建异步任务的顺序是:
jobs.create(req)先把 job 放进_jobs,状态为PENDING。jobs.submit_job(job_id, service)再尝试入队。- 如果 queue full,API 返回 429。
- 但刚创建的 job 没有删除,也没有标记 failed/cancelled/expired,仍然留在
_jobs里。
这会导致:
- job store capacity 泄漏。
- 用户可能拿到错误 detail 里的 job_id,但这个 job 永远不会执行。
- 默认
result_ttl_seconds=3600下,pending job 不会按 TTL 过期;当前 pending TTL 只在result_ttl_seconds == 0的特殊测试路径生效。 - 连续 queue full 会把 job store 填满,后续正常请求也受影响。
建议修复方式:
submit_job()失败时回滚删除刚创建的 job。- 或者将其标记为
FAILED,写入明确错误,设置expires_at。 - 更好的方式是先 reserve queue capacity,再创建 job,保证 create 成功的 job 一定可执行或可查询到终态。
建议补测试:
def test_queue_full_does_not_leave_orphan_pending_job():
store = InMemoryGraphExtractJobStore(max_jobs=1)
# 构造 queue full
# 调用 create endpoint 返回 429
# 断言 store 中没有新增的 permanent pending job4. GraphImportRequest.data 校验太浅,public write endpoint 会把非法数据推进运行时
当前 data validator 只检查 vertices、edges、triples 至少有一个 truthy 值。它没有检查:
vertices是否为 list。edges是否为 list。triples是否为 list。- vertex item 是否包含
label和properties。 - edge item 是否包含
label/outV/inV/properties等必要字段。 properties是否为 object。
因此下面这种请求会通过模型层:
{
"schema": { ... },
"data": {
"vertices": "not-a-list"
},
"write_to_graph": true
}然后在 Commit2Graph 内部才因为字符串迭代、字段访问或 schema 逻辑失败。这类问题应该在 HTTP request boundary 返回 4xx,而不是进入图写入流程后变成运行时错误。
建议:
- 在
GraphImportRequest.validate_data()中强校验vertices/edges/triples类型。 - 对 property graph import 增加最小结构校验。
- 对 unsupported triples-only import 保持现有拒绝逻辑,但也要校验 triples 类型。
- 错误返回应包含明确字段路径,便于客户端修复请求。
建议补测试:
@pytest.mark.parametrize("bad_data", [
{"vertices": "not-a-list"},
{"edges": "not-a-list"},
{"vertices": [{"properties": {"name": "marko"}}]},
{"vertices": [{"label": "person", "properties": "not-object"}]},
{"edges": [{"label": "knows", "properties": {}}]},
])
def test_graph_import_rejects_malformed_data_at_request_boundary(bad_data):
response = client.post(
"/graph/import",
json={"schema": VALID_SCHEMA, "data": bad_data, "write_to_graph": True},
)
assert response.status_code == 422P1 / 应尽快修复
5. /graph/extract 的异常状态码和错误分类不够准确
当前 /graph/extract 把 extract_service.extract_sync() 抛出的所有 ValueError 都映射为 HTTP 500,错误码是 GRAPH_EXTRACT_INVALID_FLOW_OUTPUT。
问题:
- 有些
ValueError是用户输入或 schema/config 问题,应该是 400/422。 - 有些是 LLM 输出格式问题,可能是 502/500 或专门的 model-output error。
- 有些才是真正的 internal server error。
/graph/import和/graph/extract-and-import又把ValueError映射为 400,三个 API 不一致。
建议引入更明确的异常类型,例如:
GraphExtractRequestError-> 400/422GraphExtractModelOutputError-> 502 或 500,但错误码明确区分GraphImportDataError-> 400/422- unknown exception -> 500
6. async job 是 process-local 设计,需要在 API contract 中显式说明
当前 job store 是 in-memory + daemon worker thread。这个设计有明显运行时限制:
- 服务重启后 job/result 丢失。
- 多进程/多 worker 部署时 job 不共享。
- running job 无法真正取消,只能拒绝取消。
- result TTL 只存在内存里。
- 没有持久化、没有跨节点调度、没有外部队列背压能力。
这些可以作为 MVP 接受,但必须写进 API 文档或 release note。否则用户会误以为这是生产级 async job 系统。
建议:
- response 或 docs 中说明 job store 是 in-memory。
- 明确取消语义:pending 可取消,running 不可中断。
- 如果目标是 production API,考虑接入持久化 store 或外部 queue。
7. API handler 过长,错误处理和 endpoint 注册逻辑需要拆分
graph_extract_http_api() 现在同时注册 7 个 endpoint,包含 service 初始化、route_class patch、job create/status/result/cancel、import、extract-and-import、错误映射。函数已经过长,后续继续扩展会很难维护。
建议:
- 拆成
register_extract_routes()、register_job_routes()、register_import_routes()。 - 把 error mapping 抽成统一 helper。
- 避免在同一个函数里临时修改并恢复
router.route_class,这种模式对后续 route 注册顺序比较敏感。
8. validation error 的 message=str(exc) 对客户端不够友好
现在自定义 GraphExtractAPIRoute 会把 FastAPI/Pydantic 的 RequestValidationError 包成统一 envelope,这是方向正确的。
但 message 直接使用 str(exc):
- 字段路径不稳定。
- 客户端难以机器解析。
- 错误信息会混入 Pydantic 内部格式。
建议保留 structured errors,例如:
{
"detail": {
"code": "GRAPH_EXTRACT_VALIDATION_ERROR",
"phase": "request",
"message": "request validation failed",
"errors": [
{"loc": ["body", "content"], "msg": "...", "type": "..."}
]
}
}P2 / 可以后续优化
9. extract_type contract 和 service 内部分支不一致
Request model 当前只允许 property_graph,但 service 仍保留 triples 分支。短期不是阻塞项,但会造成维护困惑。
建议:
- 如果本 PR 不支持 triples,就删除或隐藏 service 中 triples 分支。
- 如果准备支持 triples,就把 request model、response model、测试、import 能力一起补齐。
10. import response 的 status 语义需要更细
当前 _build_import_status() 根据 skipped/created 判断 succeeded/partial/failed。这个方向可以,但仍然比较粗:
- attempted 为 0 但 skipped > 0 的情况可能是内部统计异常。
- schema creation 失败、vertex 创建失败、edge 创建失败都混在 warnings/errors 里。
- HTTP status 始终 200,业务 status 可能 failed。
建议后续区分:
- request validation failed -> 4xx
- graph write partial -> 200 + partial
- graph write all failed -> 200/500 需要明确 API contract
- HugeGraph unavailable -> 503/502
测试缺口
当前新增测试数量不少,CI 通过,但缺少以下关键用例:
- legacy
texts: List[str]兼容性。 - inline schema import 不指定 target graph 时的拒绝或明确确认。
- queue full 后不会留下 orphan pending job。
- import data 非 list / malformed item 的请求边界校验。
/graph/extract中不同类型ValueError的状态码映射。- 多 worker/process 下 job store 行为的文档化或限制说明。
这些不是边角 case,而是 public API 和写入安全的核心路径。
合并前最低要求
合并前至少需要完成:
- 修复或明确版本化处理
texts: List[str]breaking change。 - 写 API 禁止隐式 fallback 到默认 graph,或引入显式确认参数并返回最终 target graph。
- queue full 时 job create/submit 必须保持原子语义,不留下不可执行的 pending job。
GraphImportRequest.data增加强类型和结构校验。- 为上述 4 点补单元测试或 API contract 测试。
完成这些后再看是否可以从 Request Changes 改为通过。
|
@imbajin Addressed the review items in
Verification after the follow-up:
|
|
Additional note on the remaining P1/P2 items in the review:
The merge-blocking items from the review are covered by the new commit: legacy |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py (1)
279-333:write_to_graph确认机制已在 HTTP 层拦截,但import_graph仍缺少服务层兜底
hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.py的/graph/import和/graph/extract-and-import在调用graph_import_service.import_graph(...)之前会检查req.write_to_graph(未置位直接返回 400),因此经由现有 HTTP 入口不会绕过确认。不过
hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py的GraphImportService.import_graph本身未校验request.write_to_graph。若未来存在非 HTTP 方式调用该 service(或新增内部调用点复用该方法),可能造成意外写入;建议在import_graph开头加守卫:if not request.write_to_graph: raise ValueError("write_to_graph must be True to confirm graph import")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py` around lines 279 - 333, GraphImportService.import_graph currently doesn't guard against accidental writes when called outside the HTTP layer; add an explicit check at the start of import_graph to verify request.write_to_graph is truthy and raise a ValueError like "write_to_graph must be True to confirm graph import" if not, so any non-HTTP callers cannot accidentally perform imports without explicit confirmation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.py`:
- Around line 227-230: The error message is inconsistent: the validation rejects
triples (if triples: raise ValueError(...)) but the subsequent check still says
"or triple"; update the second ValueError to reflect only supported types
(vertices and edges) by changing its message to something like "data must
contain at least one vertex or edge" so it no longer references triples; adjust
this in the validation block that uses the variables triples, vertices, edges
(the same function/method performing import validation in
graph_extract_requests.py).
---
Nitpick comments:
In `@hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py`:
- Around line 279-333: GraphImportService.import_graph currently doesn't guard
against accidental writes when called outside the HTTP layer; add an explicit
check at the start of import_graph to verify request.write_to_graph is truthy
and raise a ValueError like "write_to_graph must be True to confirm graph
import" if not, so any non-HTTP callers cannot accidentally perform imports
without explicit confirmation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b4f3ba04-c182-458c-9c91-eaad290781e5
📒 Files selected for processing (18)
hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.pyhugegraph-llm/src/hugegraph_llm/flows/graph_extract.pyhugegraph-llm/src/hugegraph_llm/nodes/base_node.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_service.pyhugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.pyhugegraph-llm/src/tests/api/test_graph_extract_api.pyhugegraph-llm/src/tests/api/test_graph_extract_jobs.pyhugegraph-llm/src/tests/api/test_graph_import_api.pyhugegraph-llm/src/tests/nodes/test_base_node.pyhugegraph-llm/src/tests/nodes/test_request_graph_config.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_schema_manager.pyhugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.py
🚧 Files skipped from review as they are similar to previous changes (12)
- hugegraph-llm/src/tests/nodes/test_base_node.py
- hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py
- hugegraph-llm/src/hugegraph_llm/nodes/base_node.py
- hugegraph-llm/src/hugegraph_llm/flows/graph_extract.py
- hugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.py
- hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.py
- hugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py
- hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py
- hugegraph-llm/src/tests/api/test_graph_import_api.py
- hugegraph-llm/src/tests/api/test_graph_extract_jobs.py
- hugegraph-llm/src/tests/api/test_graph_extract_api.py
|
Addressed the remaining CodeRabbit review items in follow-up commits:
Verification run locally:
|
imbajin
left a comment
There was a problem hiding this comment.
整体方向我认可:这个 PR 把 Graph Extract 从单一同步 API 扩展为 service-backed sync extract、async jobs、import、extract-and-import、request-scoped graph config、structured error response,并补了不少 API/job/import 侧测试。整体设计没有引入过重的新框架,基本沿用了现有 Flow / Scheduler / Operator 架构,这个方向是合理的。
不过我目前建议先 Request Changes,不建议直接合并。主要原因不是已有 review 中那些已经修复的 legacy texts、inline schema target graph、queue full rollback、import data validation、write_to_graph service guard 等问题,而是当前组合 API 和生产安全边界仍有几个会影响 public contract 的问题:
/graph/extract-and-import顶层status目前无条件返回"succeeded",但import_graph()可能返回partial或failed。这会让调用方误判整体写入成功,是 API contract 级别的问题。/graph/extract-and-import把 extract 和 import 两个阶段放在同一个 try-catch 里,导致 extract 阶段的异常也会被包装成 import phase 的错误,排障和客户端重试策略会被误导。- 多个 500/runtime error path 仍然把
str(exc)直接返回给客户端,async job failed result 也会暴露原始异常 message。Graph 抽取处理的是用户原文、LLM 输出、schema、graph config,异常内容可能包含敏感文本、provider response 或内部连接信息,建议 500 只返回通用安全信息,详细异常留在 server log。 PropertyGraphExtract的 debug log 会记录完整 chunk input 和 LLM output。即使是 debug 级别,在集中日志或排障环境中也容易泄露原始文档内容和模型输出。- chunk-level 并发目前只限制单请求内
max_parallel_chunks,但没有全局 LLM call backpressure。同步 HTTP 并发和 async job 并发叠加时,实际 LLM 并发可能远超配置预期。这个可以后续完善,但至少需要在文档或配置说明中明确生产部署边界。
建议合并前至少修复前 4 点,并补上对应回归测试:
- import 返回
partial/failed时,extract-and-import 顶层 status 不应固定 succeeded; - extract 阶段异常和 import 阶段异常应返回正确 phase/code;
- runtime 500/job failed result 不应回显敏感异常内容;
- 默认日志路径不应记录 raw input chunk 和 raw LLM output。
当前评分我给 7/10。修掉上述 contract/security 问题后,我认为可以提升到 8/10 左右并考虑合并。这里不建议继续引入复杂抽象,保持小改、清晰、可测试即可。
| return GraphExtractAndImportResponse( | ||
| status="succeeded", | ||
| extract_result=extract_response, | ||
| import_result=import_response, |
There was a problem hiding this comment.
这里不能无条件返回 "succeeded"。graph_import_service.import_graph() 的返回值本身可能是 partial 或 failed,但当前顶层 response 仍固定为 succeeded,会让调用方误判“抽取+写入”整体成功。
建议至少改成:
return GraphExtractAndImportResponse(
status=import_response.status,
extract_result=extract_response,
import_result=import_response,
)如果后续想表达更复杂语义,也可以定义一个很小的状态合成规则:extract succeeded + import succeeded => succeeded;import partial => partial;import failed => failed。这个点建议合并前修复,并补 partial/failed 的 API contract test。
| ) | ||
| try: | ||
| extract_response = extract_service.extract_sync(req) | ||
| import_response = graph_import_service.import_graph( |
There was a problem hiding this comment.
这里建议拆成两个 try-catch。当前这个 try 同时覆盖 extract 和 import 两个阶段,但下面所有异常都被包装成 import phase。这样如果 extract_service.extract_sync(req) 因 LLM 输出非法、schema/config 问题、scheduler/runtime 问题失败,客户端会看到 import 阶段错误,排障和重试策略都会被误导。
建议结构保持简单:
try:
extract_response = extract_service.extract_sync(req)
except FlowOutputValidationError as exc:
... # phase="extract"
except ValueError as exc:
... # phase="request" 或 extract,按现有 contract 统一
except Exception as exc:
... # phase="extract"
try:
import_response = graph_import_service.import_extracted_graph(req, extract_response)
except ...:
... # phase="import"不需要引入新的框架或复杂状态机,只要保证两个阶段的 error code / phase 准确即可。
| status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, | ||
| detail=_error("GRAPH_EXTRACT_FAILED", str(exc), "extract"), | ||
| ) from exc | ||
|
|
There was a problem hiding this comment.
这里不建议在 500/runtime error response 中直接返回 str(exc)。Graph extract/import 处理的是原始文档、LLM 输出、schema、graph config 和 HugeGraph/LLM provider 调用,异常字符串里可能包含用户文本片段、prompt、provider response、URL、credential-like 信息或内部实现细节。
建议区分:
ValueError这类明确用户输入错误,可以继续返回相对具体的 400 message;Exception/ runtime 500 只返回通用安全 message,例如"Graph extraction failed. Check server logs for details.";- 详细异常继续用
log.error(..., exc_info=True)记录在服务端。
这个规则也需要同步应用到 /graph/import、/graph/extract-and-import 和 async job failed result。
| if job.status == GraphExtractJobStatus.FAILED: | ||
| raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=job.error.model_dump()) | ||
| if job.result is None: | ||
| raise HTTPException( |
There was a problem hiding this comment.
这里会把 async job 执行阶段保存的原始错误直接暴露给客户端。当前测试还断言了 message == "llm failed",这会把“回显 runtime exception”固化成 API contract。
建议把 job failed result 的 public message 改成安全通用描述,例如:
{
"code": "GRAPH_EXTRACT_JOB_FAILED",
"message": "Graph extraction job failed. Check server logs for details.",
"phase": "extract",
"job_id": "..."
}原始异常可以保留在 server log 中。这样和同步 endpoint 的 500 sanitization 策略一致,也避免 job API 成为敏感信息泄露路径。
| extract_response = extract_service.extract_sync(req) | ||
| import_response = graph_import_service.import_graph( | ||
| GraphImportRequest( | ||
| schema=req.schema, |
There was a problem hiding this comment.
这里可以直接复用 GraphImportService.import_extracted_graph(req, extract_response),避免 API 层重复构造 GraphImportRequest。
当前重复逻辑不算严重 bug,但后续如果 import options、target graph、status/warnings/meta 规则继续演进,很容易出现 service helper 和 API handler 两处逻辑分叉。建议保持简单,把转换逻辑收敛到 service 层:
import_response = graph_import_service.import_extracted_graph(req, extract_response)这属于小幅清理,不需要额外抽象。
| from typing import Optional | ||
|
|
||
| from fastapi import APIRouter, HTTPException, status | ||
| from fastapi import APIRouter, HTTPException, Request, status |
There was a problem hiding this comment.
GitHub 对这个文件提示存在 hidden / bidirectional Unicode text。这个不一定是恶意问题,也可能是编辑器或复制粘贴带入的控制字符,但合并前建议清理掉。
建议:
- 用能显示 invisible/control chars 的编辑器检查该文件;
- 移除 bidi/control chars;
- 如果项目愿意,可以加一个很轻量的 CI 检查,禁止 Python 源文件包含 bidi control characters。
这个问题本身不涉及业务逻辑,但会降低代码审查可信度,也有供应链安全层面的顾虑。
| ) | ||
| for chunk, proceeded_chunk in zip(chunks, proceeded_chunks): | ||
| log.debug( | ||
| "[LLM] %s input: %s \n output:%s", |
There was a problem hiding this comment.
这里不建议默认记录完整 input chunk 和 LLM output。Graph 抽取输入通常是业务文档、用户原文或内部知识文本,LLM output 也可能包含原文片段或敏感实体。即使是 debug 级别,在集中日志、排障环境或临时打开 debug 时也有泄露风险。
建议改成只记录非敏感元信息,例如 chunk index、chunk length、hash、output length、解析状态等:
log.debug(
"[LLM] %s chunk_index=%s input_len=%s output_len=%s",
self.__class__.__name__,
idx,
len(chunk),
len(proceeded_chunk or ""),
)如果确实需要 raw prompt/output dump,建议放到显式 opt-in 的本地诊断开关里,默认关闭。
| else: | ||
| with ThreadPoolExecutor(max_workers=worker_count) as executor: | ||
| proceeded_chunks = list( | ||
| executor.map(lambda chunk: self.extract_property_graph_by_llm(schema, chunk), chunks) |
There was a problem hiding this comment.
这里的 per-request chunk 并发本身合理,但目前缺少进程级/服务级 backpressure。实际 LLM 并发会变成:
同步 HTTP 并发请求数 × 每请求 max_parallel_chunks
+
async running jobs × 每 job max_parallel_chunks
这可能远超 graph_extract_max_parallel_chunks 的直觉含义,也容易打爆 provider rate limit 或本地 Ollama 资源。建议最小化处理:
- 增加一个进程内全局 semaphore / limiter,限制总 LLM calls;
- 或者至少在配置/API 文档中明确:当前只限制单请求内 chunk 并发,生产部署需要通过 ASGI worker/concurrency、反向代理或外部限流控制总并发。
不建议为这个 PR 引入复杂队列系统,但需要把并发边界讲清楚。
| assert response.status_code == status.HTTP_200_OK | ||
| assert response.json()["import_result"]["status"] == "succeeded" | ||
| extract_service.extract_sync.assert_called_once() | ||
| import_service.import_graph.assert_called_once() |
There was a problem hiding this comment.
这个测试建议补两个关键 contract case,否则当前 bug 不会被捕获:
import_service.import_graph()返回GraphImportResponse(status="partial")/status="failed"时,断言 response 顶层status也应是partial/failed,不能固定 succeeded。extract_service.extract_sync()抛FlowOutputValidationError/ValueError/RuntimeError时,断言 error detail 的phase和code属于 extract/request 阶段,而不是 import 阶段。
这两个测试能直接锁住 /graph/extract-and-import 的 public API contract。
| assert result_response.json()["detail"]["code"] == "GRAPH_EXTRACT_JOB_FAILED" | ||
| assert result_response.json()["detail"]["message"] == "llm failed" | ||
| assert result_response.json()["detail"]["phase"] == "extract" | ||
|
|
There was a problem hiding this comment.
这个断言建议调整。当前测试把“job failed result 回显原始 exception message”固定成了预期行为,但这对 graph extraction API 不太安全:runtime exception 可能包含用户输入、LLM 输出、provider response 或内部配置。
建议改成:
- public response 只断言通用安全 message;
- code / phase / job_id 保持可定位;
- 原始 exception 只在 server log 中保留,不作为 API contract 暴露。
There was a problem hiding this comment.
目前 Request Changes,不建议直接合并。
主要原因不是已有 review 中那些已经修复的 legacy texts、inline schema target graph、queue full rollback、import data validation、write_to_graph service guard 等问题,而是当前组合 API 和生产安全边界仍有几个会影响 public contract 的问题:
/graph/extract-and-import顶层status目前无条件返回"succeeded",但import_graph()可能返回partial或failed。这会让调用方误判整体写入成功,是 API contract 级别的问题。/graph/extract-and-import把 extract 和 import 两个阶段放在同一个 try-catch 里,导致 extract 阶段的异常也会被包装成 import phase 的错误,排障和客户端重试策略会被误导。- 多个 500/runtime error path 仍然把
str(exc)直接返回给客户端,async job failed result 也会暴露原始异常 message。Graph 抽取处理的是用户原文、LLM 输出、schema、graph config,异常内容可能包含敏感文本、provider response 或内部连接信息,建议 500 只返回通用安全信息,详细异常留在 server log。 PropertyGraphExtract的 debug log 会记录完整 chunk input 和 LLM output。即使是 debug 级别,在集中日志或排障环境中也容易泄露原始文档内容和模型输出。- chunk-level 并发目前只限制单请求内
max_parallel_chunks,但没有全局 LLM call backpressure。同步 HTTP 并发和 async job 并发叠加时,实际 LLM 并发可能远超配置预期。这个可以后续完善,但至少需要在文档或配置说明中明确生产部署边界。
建议合并前至少修复前 4 点,并补上对应回归测试:
- import 返回
partial/failed时,extract-and-import 顶层 status 不应固定 succeeded; - extract 阶段异常和 import 阶段异常应返回正确 phase/code;
- runtime 500/job failed result 不应回显敏感异常内容;
- 默认日志路径不应记录 raw input chunk 和 raw LLM output。
当前评分我给 7/10。修掉上述 contract/security 问题后,我认为可以提升到 8/10 左右并考虑合并。这里不建议继续引入复杂抽象,保持小改、清晰、可测试即可。
| return GraphExtractAndImportResponse( | ||
| status="succeeded", | ||
| extract_result=extract_response, | ||
| import_result=import_response, |
There was a problem hiding this comment.
这里不能无条件返回 "succeeded"。graph_import_service.import_graph() 的返回值本身可能是 partial 或 failed,但当前顶层 response 仍固定为 succeeded,会让调用方误判“抽取+写入”整体成功。
建议至少改成:
return GraphExtractAndImportResponse(
status=import_response.status,
extract_result=extract_response,
import_result=import_response,
)如果后续想表达更复杂语义,也可以定义一个很小的状态合成规则:extract succeeded + import succeeded => succeeded;import partial => partial;import failed => failed。这个点建议合并前修复,并补 partial/failed 的 API contract test。
There was a problem hiding this comment.
Fixed in e761776. /graph/extract-and-import now returns status=import_response.status, so partial or failed import results are reflected in the top-level response. Added regression coverage for this contract.
| ) | ||
| try: | ||
| extract_response = extract_service.extract_sync(req) | ||
| import_response = graph_import_service.import_graph( |
There was a problem hiding this comment.
这里建议拆成两个 try-catch。当前这个 try 同时覆盖 extract 和 import 两个阶段,但下面所有异常都被包装成 import phase。这样如果 extract_service.extract_sync(req) 因 LLM 输出非法、schema/config 问题、scheduler/runtime 问题失败,客户端会看到 import 阶段错误,排障和重试策略都会被误导。
建议结构保持简单:
try:
extract_response = extract_service.extract_sync(req)
except FlowOutputValidationError as exc:
...
phase="extract"
except ValueError as exc:
...
phase="request" # 或 extract,按现有 contract 统一
except Exception as exc:
...
phase="extract"
try:
import_response = graph_import_service.import_extracted_graph(req, extract_response)
except ...不需要引入新的框架或复杂状态机,只要保证两个阶段的 error code / phase 准确即可。
There was a problem hiding this comment.
Fixed in e761776. The handler now separates extraction and import try/except blocks: extraction failures return GRAPH_EXTRACT_* codes with phase extract, while import failures return GRAPH_IMPORT_* codes with phase import. Regression tests cover runtime and flow-output failures for both phases.
| status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, | ||
| detail=_error("GRAPH_EXTRACT_FAILED", str(exc), "extract"), | ||
| ) from exc | ||
|
|
There was a problem hiding this comment.
这里不建议在 500/runtime error response 中直接返回 str(exc)。Graph extract/import 处理的是原始文档、LLM 输出、schema、graph config 和 HugeGraph/LLM provider 调用,异常字符串里可能包含用户文本片段、prompt、provider response、URL、credential-like 信息或内部实现细节。
建议区分:
ValueError这类明确用户输入错误,可以继续返回相对具体的 400 message;Exception/ runtime 500 只返回通用安全 message,例如"Graph extraction failed. Check server logs for details.";- 详细异常继续用
log.error(..., exc_info=True)记录在服务端。
这个规则也需要同步应用到 /graph/import、/graph/extract-and-import 和 async job failed result。
There was a problem hiding this comment.
Fixed in e761776. Runtime and flow-output 500 responses now use generic public messages; client/actionable ValueError paths remain 400. Added tests asserting raw provider/runtime text is not echoed in the API response.
| if job.status == GraphExtractJobStatus.FAILED: | ||
| raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=job.error.model_dump()) | ||
| if job.result is None: | ||
| raise HTTPException( |
There was a problem hiding this comment.
这里会把 async job 执行阶段保存的原始错误直接暴露给客户端。当前 job store 中 GraphExtractError.message 是 str(exc),因此这里的 job.error.model_dump() 会把 runtime exception 原样返回。
建议把 job failed result 的 public message 改成安全通用描述,例如:
{
"code": "GRAPH_EXTRACT_JOB_FAILED",
"message": "Graph extraction job failed. Check server logs for details.",
"phase": "extract",
"job_id": "..."
}原始异常可以保留在 server log 中。这样和同步 endpoint 的 500 sanitization 策略一致,也避免 job API 成为敏感信息泄露路径。
There was a problem hiding this comment.
Fixed in e761776. Job execution failures now store a generic GRAPH_EXTRACT_JOB_FAILED public message; the raw exception stays in server logs via log.exception. The job failure test now asserts sensitive exception text is not returned.
| log.debug( | ||
| "[LLM] %s input: %s \n output:%s", | ||
| self.__class__.__name__, | ||
| chunk, |
There was a problem hiding this comment.
这里不建议默认记录完整 input chunk 和 LLM output。Graph 抽取输入通常是业务文档、用户原文或内部知识文本,LLM output 也可能包含原文片段或敏感实体。即使是 debug 级别,在集中日志、排障环境或临时打开 debug 时也有泄露风险。
建议改成只记录非敏感元信息,例如 chunk index、chunk length、hash、output length、解析状态等:
log.debug(
"[LLM] %s chunk_index=%s input_len=%s output_len=%s",
self.__class__.__name__,
idx,
len(chunk),
len(proceeded_chunk or ""),
)如果确实需要 raw prompt/output dump,建议放到显式 opt-in 的本地诊断开关里,默认关闭。
There was a problem hiding this comment.
Fixed in e761776. The debug log now records only chunk index, input length, and output length. I also removed raw edge payloads from invalid-edge warning logs and added regression coverage for both log paths.
| else: | ||
| with ThreadPoolExecutor(max_workers=worker_count) as executor: | ||
| proceeded_chunks = list( | ||
| executor.map(lambda chunk: self.extract_property_graph_by_llm(schema, chunk), chunks) |
There was a problem hiding this comment.
这里的 per-request chunk 并发本身合理,但目前缺少进程级/服务级 backpressure。实际 LLM 并发会变成:
同步 HTTP 并发请求数 × 每请求 max_parallel_chunks
+
async running jobs × 每 job max_parallel_chunks
这可能远超 graph_extract_max_parallel_chunks 的直觉含义,也容易打爆 provider rate limit 或本地 Ollama 资源。建议最小化处理:
- 增加一个进程内全局 semaphore / limiter,限制总 LLM calls;
- 或者至少在配置/API 文档中明确:当前只限制单请求内 chunk 并发,生产部署需要通过 ASGI worker/concurrency、反向代理或外部限流控制总并发。
不建议为这个 PR 引入复杂队列系统,但需要把并发边界讲清楚。
There was a problem hiding this comment.
Not changed in this PR. Current controls are per-request max_parallel_chunks plus async job worker/capacity limits. A process/provider-wide semaphore would be a broader runtime policy change across sync request handling and background jobs, so it is better designed separately from this API contract and safety follow-up.
| extract_response = extract_service.extract_sync(req) | ||
| import_response = graph_import_service.import_graph( | ||
| GraphImportRequest( | ||
| schema=req.schema, |
There was a problem hiding this comment.
这里可以直接复用 GraphImportService.import_extracted_graph(req, extract_response),避免 API 层重复构造 GraphImportRequest。
当前重复逻辑不算严重 bug,但后续如果 import options、target graph、status/warnings/meta 规则继续演进,很容易出现 service helper 和 API handler 两处逻辑分叉。建议保持简单,把转换逻辑收敛到 service 层:
import_response = graph_import_service.import_extracted_graph(req, extract_response)这属于小幅清理,不需要额外抽象。
There was a problem hiding this comment.
Not changed in this PR. The route still constructs GraphImportRequest explicitly so extraction failure handling and import failure handling remain separated and visible at the API boundary. Reusing import_extracted_graph() can be a later cleanup, but it is not required for the correctness and safety fixes here.
|
|
||
| def test_confirmed_extract_and_import_runs_extraction_before_import(): | ||
| extract_service = Mock() | ||
| extract_service.extract_sync.return_value = GraphExtractResponse( |
There was a problem hiding this comment.
这个测试建议补两个关键 contract case,否则当前 bug 不会被捕获:
import_service.import_graph()返回GraphImportResponse(status="partial")/status="failed"时,断言 response 顶层status也应是partial/failed,不能固定 succeeded。extract_service.extract_sync()抛FlowOutputValidationError/ValueError/RuntimeError时,断言 error detail 的phase和code属于 extract/request 阶段,而不是 import 阶段。
这两个测试能直接锁住 /graph/extract-and-import 的 public API contract。
There was a problem hiding this comment.
Fixed in e761776. Added tests for top-level status tracking import partial/failed results, extraction runtime-error sanitization, import runtime-error sanitization, and phase-specific flow-output error codes.
| assert result_response.json()["detail"]["code"] == "GRAPH_EXTRACT_JOB_FAILED" | ||
| assert result_response.json()["detail"]["message"] == "llm failed" | ||
| assert result_response.json()["detail"]["phase"] == "extract" | ||
|
|
There was a problem hiding this comment.
这个断言建议调整。当前测试把“job failed result 回显原始 exception message”固定成了预期行为,但这对 graph extraction API 不太安全:runtime exception 可能包含用户输入、LLM 输出、provider response 或内部配置。
建议改成:
- public response 只断言通用安全 message;
- code / phase / job_id 保持可定位;
- 原始 exception 只在 server log 中保留,不作为 API contract 暴露。
There was a problem hiding this comment.
Fixed in e761776. The assertion now checks the generic public job-failure message and verifies the raw exception text is not present in the response.
| from typing import Optional | ||
|
|
||
| from fastapi import APIRouter, HTTPException, status | ||
| from fastapi import APIRouter, HTTPException, Request, status |
There was a problem hiding this comment.
GitHub 对这个文件提示存在 hidden / bidirectional Unicode text。这个不一定是恶意问题,也可能是编辑器或复制粘贴带入的控制字符,但合并前建议清理掉。
建议:
- 用能显示 invisible/control chars 的编辑器检查该文件;
- 移除 bidi/control chars;
- 如果项目愿意,可以加一个很轻量的 CI 检查,禁止 Python 源文件包含 bidi control characters。
这个问题本身不涉及业务逻辑,但会降低代码审查可信度,也有供应链安全层面的顾虑。
There was a problem hiding this comment.
Checked the current file with a Unicode control-character scan; no hidden control or bidi characters are present in hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.py. No code change is needed for this item.
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
.github/workflows/hugegraph-python-client.yml (2)
40-48:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win缓存键建议纳入
uv.lock,否则依赖变化时可能复用陈旧缓存。Line 46 只基于
pyproject.toml计算 key,锁文件更新后可能仍命中旧缓存,导致 CI 结果不稳定或难复现。建议修复(示例)
- key: ${{ runner.os }}-uv-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml') }} + key: ${{ runner.os }}-uv-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml', '**/uv.lock') }}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/hugegraph-python-client.yml around lines 40 - 48, The cache key in the "Cache dependencies" step only includes pyproject.toml in the hashFiles function, which means changes to the lock file (uv.lock) won't invalidate the cache. Modify the key parameter in the actions/cache@v4 step to include both pyproject.toml and uv.lock in the hashFiles function call, so that updates to either file will properly invalidate the cached dependencies and prevent using stale cache entries.
42-48:⚠️ Potential issue | 🟠 Major缓存策略存在污染风险,建议为不可信上下文隔离缓存或使用只读模式。
当前工作流在
pull_request事件上执行来自fork的代码时,虽然不像pull_request_target那样严重,但fork中的工作流仍能向仓库共享缓存写入内容。恢复键${{ runner.os }}-uv-${{ matrix.python-version }}-过于宽泛,允许不同提交的缓存条目相互覆盖,可能导致依赖污染。建议以下改进之一:
- 在PR中使用只读缓存模式,仅在推送到受信分支时允许缓存写入
- 为fork PR添加上下文前缀以隔离缓存(如
${{ runner.os }}-pr-${{ github.event.pull_request.head.ref }})- 收紧恢复键为完整的依赖哈希,避免跨版本的广泛回退
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/hugegraph-python-client.yml around lines 42 - 48, The cache configuration in the actions/cache action is too permissive for untrusted contexts like pull requests from forks. Modify the cache action by adding a conditional read-only parameter that sets read-only mode to true when the event is pull_request, ensuring cache write operations only occur on trusted pushes to branches. Additionally, tighten the restore-keys by removing the overly broad trailing wildcard pattern and either using the complete dependency hash or adding a context-specific prefix (such as event name or branch reference) to prevent cache entries from different commits from overwriting each other and causing dependency pollution.Source: Linters/SAST tools
.github/workflows/hugegraph-llm.yml (1)
56-63:⚠️ Potential issue | 🟠 Major限制缓存写入以防止投毒攻击
使用
restore-keys的actions/cache配置存在缓存投毒风险。pull_request 工作流具有缓存写权限,恶意 PR 可向共享缓存注入污染内容,之后特权工作流(如主分支推送)会从回退键恢复这些污染条目。建议通过条件if: github.event_name != 'pull_request'防止非可信事件写入缓存:建议修复
- name: Cache dependencies uses: actions/cache@v4 + if: github.event_name != 'pull_request' with: path: | ~/.cache/uv ~/nltk_data key: ${{ runner.os }}-uv-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml', 'uv.lock') }} restore-keys: | ${{ runner.os }}-uv-${{ matrix.python-version }}-🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/hugegraph-llm.yml around lines 56 - 63, The `actions/cache@v4` step has a cache poisoning vulnerability because pull_request workflows can write to the shared cache, allowing malicious PRs to inject contaminated content that privileged workflows (like main branch pushes) will later restore via the restore-keys fallback mechanism. Add an `if` condition to the cache step with the value `github.event_name != 'pull_request'` to prevent untrusted pull_request events from writing to the cache, allowing only cache reads during PR workflows while maintaining write access for trusted workflows on main branches.
🧹 Nitpick comments (1)
hugegraph-llm/src/tests/api/test_graph_extract_api.py (1)
602-658: 🧹 Nitpick | 🔵 Trivial | 🏗️ Heavy lift建议拆分该测试模块以满足 600 行上限。
当前文件已超过 600 行(到 Line 658),本次继续在同一文件增加多组测试,后续维护与冲突成本会持续上升。建议按主题拆分(如请求契约/错误语义/路由注册/服务元数据)并保留共享夹具。
As per coding guidelines,hugegraph-llm/**/*.py: Keep each Python file under 600 lines for maintainability.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hugegraph-llm/src/tests/api/test_graph_extract_api.py` around lines 602 - 658, The test file test_graph_extract_api.py exceeds the 600-line limit (currently at 658 lines) and needs to be refactored by splitting tests into separate modules organized by topic. Extract route registration tests like test_existing_routes_still_register and test_rag_demo_registers_graph_extract_routes_once into a new file (e.g., test_graph_extract_routes.py), while keeping any shared fixtures and helper functions in a common location. Ensure imports and dependencies are properly updated in both the new and existing test files to maintain functionality.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/hugegraph-llm.yml:
- Around line 38-42: The fixed sleep duration of 10 seconds in the "Prepare
HugeGraph Server Environment" step is unreliable and causes intermittent test
failures when the service takes longer to start. Replace the static `sleep 10`
command with a health check polling mechanism that repeatedly attempts to
connect to the HugeGraph server (e.g., by making HTTP requests to
http://localhost:8080 or checking the server's health endpoint) and only
proceeds with the next step once the server responds successfully, using a loop
with retry attempts and appropriate timeouts to ensure the service is fully
ready before running tests.
- Line 30: Add an explicit permissions block to the build job to enforce minimal
privilege access. Currently the job relies on default GITHUB_TOKEN permissions
which are overly broad. Add a permissions section under the build job definition
specifying only the minimum required permissions needed for the job's
operations, such as contents: read for repository access. This reduces security
risk by explicitly limiting what the GITHUB_TOKEN can do if the workflow is
compromised.
- Around line 43-57: Replace the tag-based references (`@v4`, `@v5`) for third-party
Actions with pinned commit SHAs to mitigate supply chain risks. Update
actions/checkout to use a specific commit SHA instead of `@v4`,
actions/setup-python to use a specific commit SHA instead of `@v5`, and
actions/cache to use a specific commit SHA instead of `@v4`. Additionally, add the
persist-credentials: false configuration to the actions/checkout step to prevent
workflow token exposure.
- Around line 50-53: The uv installation in the "Install uv" step uses an
unpinned installation script which poses supply chain security risks and reduces
CI/CD reproducibility. Modify the curl command that downloads the uv installer
to include a specific version number in the URL (the official installer supports
pinning versions via the URL parameter). Update the installation URL
https://astral.sh/uv/install.sh to include a version specification to ensure a
consistent, reproducible installation across CI/CD runs. Optionally, consider
adding script integrity verification before execution to further enhance
security.
In @.github/workflows/hugegraph-python-client.yml:
- Line 11: The build job is missing an explicit permissions block, which means
it relies on default GitHub Actions permissions that may be too broad for
security best practices. Add a permissions block to the build job with minimized
permissions, specifying only the necessary permissions such as contents: read.
This reduces the risk of token misuse and follows the principle of least
privilege. Place the permissions block as a top-level key under the build job,
before or alongside other job configuration like runs-on.
- Around line 35-38: The uv installation step in the "Install uv" task uses an
unpinned remote script without integrity verification, creating a supply chain
security risk. Replace the curl command that downloads and pipes install.sh
directly to sh with either an official GitHub Action like astral-sh/setup-uv@v1
(if available), or pin the installation to a specific uv version and add SHA256
checksum verification before execution. Ensure the PATH configuration in the
echo command is properly updated to work with whichever installation method is
chosen.
- Around line 28-42: The GitHub Actions workflow is using version tags (v4, v5)
instead of pinned commit SHAs, which introduces supply chain security risks.
Additionally, the actions/checkout action is missing the persist-credentials:
false configuration. Fix this by: (1) replacing actions/checkout@v4 with a
pinned commit SHA and add persist-credentials: false to disable credential
persistence, (2) replace actions/setup-python@v5 with the equivalent action
pinned to a specific commit SHA, and (3) replace actions/cache@v4 with the
equivalent action pinned to a specific commit SHA. Each action reference should
use the format uses: owner/repo@commit-sha instead of the version tag format.
---
Outside diff comments:
In @.github/workflows/hugegraph-llm.yml:
- Around line 56-63: The `actions/cache@v4` step has a cache poisoning
vulnerability because pull_request workflows can write to the shared cache,
allowing malicious PRs to inject contaminated content that privileged workflows
(like main branch pushes) will later restore via the restore-keys fallback
mechanism. Add an `if` condition to the cache step with the value
`github.event_name != 'pull_request'` to prevent untrusted pull_request events
from writing to the cache, allowing only cache reads during PR workflows while
maintaining write access for trusted workflows on main branches.
In @.github/workflows/hugegraph-python-client.yml:
- Around line 40-48: The cache key in the "Cache dependencies" step only
includes pyproject.toml in the hashFiles function, which means changes to the
lock file (uv.lock) won't invalidate the cache. Modify the key parameter in the
actions/cache@v4 step to include both pyproject.toml and uv.lock in the
hashFiles function call, so that updates to either file will properly invalidate
the cached dependencies and prevent using stale cache entries.
- Around line 42-48: The cache configuration in the actions/cache action is too
permissive for untrusted contexts like pull requests from forks. Modify the
cache action by adding a conditional read-only parameter that sets read-only
mode to true when the event is pull_request, ensuring cache write operations
only occur on trusted pushes to branches. Additionally, tighten the restore-keys
by removing the overly broad trailing wildcard pattern and either using the
complete dependency hash or adding a context-specific prefix (such as event name
or branch reference) to prevent cache entries from different commits from
overwriting each other and causing dependency pollution.
---
Nitpick comments:
In `@hugegraph-llm/src/tests/api/test_graph_extract_api.py`:
- Around line 602-658: The test file test_graph_extract_api.py exceeds the
600-line limit (currently at 658 lines) and needs to be refactored by splitting
tests into separate modules organized by topic. Extract route registration tests
like test_existing_routes_still_register and
test_rag_demo_registers_graph_extract_routes_once into a new file (e.g.,
test_graph_extract_routes.py), while keeping any shared fixtures and helper
functions in a common location. Ensure imports and dependencies are properly
updated in both the new and existing test files to maintain functionality.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 22616416-fb0b-44cd-b8fa-76a715816f95
📒 Files selected for processing (46)
.github/workflows/auto-pr-comment.yml.github/workflows/hugegraph-llm.yml.github/workflows/hugegraph-python-client.ymlhugegraph-llm/src/hugegraph_llm/api/graph_extract_api.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.pyhugegraph-llm/src/hugegraph_llm/config/llm_config.pyhugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.pyhugegraph-llm/src/hugegraph_llm/flows/graph_extract.pyhugegraph-llm/src/hugegraph_llm/flows/import_graph_data.pyhugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.pyhugegraph-llm/src/hugegraph_llm/models/llms/init_llm.pyhugegraph-llm/src/hugegraph_llm/nodes/base_node.pyhugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.pyhugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.pyhugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.pyhugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.pyhugegraph-llm/src/hugegraph_llm/services/__init__.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_service.pyhugegraph-llm/src/hugegraph_llm/state/ai_state.pyhugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.pyhugegraph-llm/src/tests/api/test_graph_extract_api.pyhugegraph-llm/src/tests/api/test_graph_extract_jobs.pyhugegraph-llm/src/tests/api/test_graph_import_api.pyhugegraph-llm/src/tests/document/test_graph_extract_configurable_split.pyhugegraph-llm/src/tests/flows/test_graph_extract_flow.pyhugegraph-llm/src/tests/models/llms/test_init_llm.pyhugegraph-llm/src/tests/nodes/test_base_node.pyhugegraph-llm/src/tests/nodes/test_extract_node.pyhugegraph-llm/src/tests/nodes/test_request_graph_config.pyhugegraph-llm/src/tests/operators/document_op/test_chunk_split.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_schema_manager.pyhugegraph-llm/src/tests/operators/llm_op/test_info_extract.pyhugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.pyhugegraph-llm/src/tests/utils/test_graph_index_utils.py
💤 Files with no reviewable changes (1)
- .github/workflows/auto-pr-comment.yml
✅ Files skipped from review due to trivial changes (1)
- hugegraph-llm/src/hugegraph_llm/services/init.py
🚧 Files skipped from review as they are similar to previous changes (37)
- hugegraph-llm/src/hugegraph_llm/config/llm_config.py
- hugegraph-llm/src/tests/nodes/test_extract_node.py
- hugegraph-llm/src/hugegraph_llm/flows/import_graph_data.py
- hugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.py
- hugegraph-llm/src/tests/document/test_graph_extract_configurable_split.py
- hugegraph-llm/src/tests/utils/test_graph_index_utils.py
- hugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.py
- hugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.py
- hugegraph-llm/src/tests/nodes/test_base_node.py
- hugegraph-llm/src/tests/operators/document_op/test_chunk_split.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.py
- hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.py
- hugegraph-llm/src/hugegraph_llm/nodes/base_node.py
- hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py
- hugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.py
- hugegraph-llm/src/tests/operators/llm_op/test_info_extract.py
- hugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.py
- hugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.py
- hugegraph-llm/src/tests/flows/test_graph_extract_flow.py
- hugegraph-llm/src/hugegraph_llm/state/ai_state.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.py
- hugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.py
- hugegraph-llm/src/tests/nodes/test_request_graph_config.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.py
- hugegraph-llm/src/tests/models/llms/test_init_llm.py
- hugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.py
- hugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_schema_manager.py
- hugegraph-llm/src/tests/api/test_graph_extract_jobs.py
- hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py
- hugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.py
- hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py
- hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.py
- hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.py
- hugegraph-llm/src/tests/api/test_graph_import_api.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py
|
Updated in Addressed review items:
Not changed in this PR:
Local verification:
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
hugegraph-llm/src/tests/api/test_graph_extract_api.py (1)
620-659: 🧹 Nitpick | 🔵 Trivial | 🏗️ Heavy lift测试文件已超过 600 行,建议按主题拆分。
当前文件已到 Line 659,本次继续追加测试会进一步增加维护成本。建议按“extract API 合约 / service 行为 / flow 透传 / 路由注册”拆分到多个测试模块。
As per coding guidelines,
hugegraph-llm/**/*.py: Keep each Python file under 600 lines for maintainability.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hugegraph-llm/src/tests/api/test_graph_extract_api.py` around lines 620 - 659, The test file test_graph_extract_api.py has exceeded the 600-line limit for maintainability. Split the file into multiple test modules organized by concern: create separate files for extract API contract tests, service behavior tests, flow pass-through tests, and route registration tests (such as test_rag_demo_registers_graph_extract_routes_once). Move the relevant test functions and fixtures to their respective modules while maintaining import dependencies and ensuring all tests remain executable.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hugegraph-llm/src/hugegraph_llm/services/graph_extract_service.py`:
- Around line 317-321: The exception object is being directly included in the
client-visible warnings list on line 319 in the exception handler, which could
expose sensitive error details like internal addresses or credentials. Replace
the warnings.append call that includes f"update_vid_embeddings failed: {exc}"
with a generic fixed message that does not include the exception details. The
detailed exception information is already being captured in the log statement on
line 318, so only provide a client-safe message without the exc variable in the
warnings list.
---
Nitpick comments:
In `@hugegraph-llm/src/tests/api/test_graph_extract_api.py`:
- Around line 620-659: The test file test_graph_extract_api.py has exceeded the
600-line limit for maintainability. Split the file into multiple test modules
organized by concern: create separate files for extract API contract tests,
service behavior tests, flow pass-through tests, and route registration tests
(such as test_rag_demo_registers_graph_extract_routes_once). Move the relevant
test functions and fixtures to their respective modules while maintaining import
dependencies and ensuring all tests remain executable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 101e0062-5efb-4799-9e53-c0632190c708
📒 Files selected for processing (43)
hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.pyhugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.pyhugegraph-llm/src/hugegraph_llm/config/llm_config.pyhugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.pyhugegraph-llm/src/hugegraph_llm/flows/graph_extract.pyhugegraph-llm/src/hugegraph_llm/flows/import_graph_data.pyhugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.pyhugegraph-llm/src/hugegraph_llm/models/llms/init_llm.pyhugegraph-llm/src/hugegraph_llm/nodes/base_node.pyhugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.pyhugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.pyhugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.pyhugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.pyhugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.pyhugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.pyhugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.pyhugegraph-llm/src/hugegraph_llm/services/__init__.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.pyhugegraph-llm/src/hugegraph_llm/services/graph_extract_service.pyhugegraph-llm/src/hugegraph_llm/state/ai_state.pyhugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.pyhugegraph-llm/src/tests/api/test_graph_extract_api.pyhugegraph-llm/src/tests/api/test_graph_extract_jobs.pyhugegraph-llm/src/tests/api/test_graph_import_api.pyhugegraph-llm/src/tests/document/test_graph_extract_configurable_split.pyhugegraph-llm/src/tests/flows/test_graph_extract_flow.pyhugegraph-llm/src/tests/models/llms/test_init_llm.pyhugegraph-llm/src/tests/nodes/test_base_node.pyhugegraph-llm/src/tests/nodes/test_extract_node.pyhugegraph-llm/src/tests/nodes/test_request_graph_config.pyhugegraph-llm/src/tests/operators/document_op/test_chunk_split.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.pyhugegraph-llm/src/tests/operators/hugegraph_op/test_schema_manager.pyhugegraph-llm/src/tests/operators/llm_op/test_info_extract.pyhugegraph-llm/src/tests/operators/llm_op/test_property_graph_extract.pyhugegraph-llm/src/tests/utils/test_graph_index_utils.py
✅ Files skipped from review due to trivial changes (2)
- hugegraph-llm/src/hugegraph_llm/services/init.py
- hugegraph-llm/src/tests/operators/document_op/test_chunk_split.py
🚧 Files skipped from review as they are similar to previous changes (37)
- hugegraph-llm/src/tests/utils/test_graph_index_utils.py
- hugegraph-llm/src/hugegraph_llm/nodes/index_node/build_semantic_index.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/fetch_graph_data.py
- hugegraph-llm/src/hugegraph_llm/operators/document_op/chunk_split.py
- hugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.py
- hugegraph-llm/src/hugegraph_llm/config/llm_config.py
- hugegraph-llm/src/hugegraph_llm/nodes/base_node.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/commit_to_hugegraph.py
- hugegraph-llm/src/hugegraph_llm/operators/index_op/build_semantic_index.py
- hugegraph-llm/src/tests/nodes/test_extract_node.py
- hugegraph-llm/src/hugegraph_llm/operators/llm_op/info_extract.py
- hugegraph-llm/src/hugegraph_llm/nodes/document_node/chunk_split.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_schema_manager.py
- hugegraph-llm/src/tests/nodes/test_base_node.py
- hugegraph-llm/src/tests/models/llms/test_init_llm.py
- hugegraph-llm/src/hugegraph_llm/state/ai_state.py
- hugegraph-llm/src/hugegraph_llm/utils/hugegraph_utils.py
- hugegraph-llm/src/tests/document/test_graph_extract_configurable_split.py
- hugegraph-llm/src/tests/operators/llm_op/test_info_extract.py
- hugegraph-llm/src/hugegraph_llm/nodes/llm_node/extract_info.py
- hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/schema.py
- hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/schema_manager.py
- hugegraph-llm/src/tests/flows/test_graph_extract_flow.py
- hugegraph-llm/src/hugegraph_llm/flows/import_graph_data.py
- hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_responses.py
- hugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.py
- hugegraph-llm/src/hugegraph_llm/flows/update_vid_embeddings.py
- hugegraph-llm/src/hugegraph_llm/api/graph_extract_api.py
- hugegraph-llm/src/hugegraph_llm/models/llms/init_llm.py
- hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py
- hugegraph-llm/src/tests/api/test_graph_import_api.py
- hugegraph-llm/src/tests/api/test_graph_extract_jobs.py
- hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py
- hugegraph-llm/src/hugegraph_llm/services/graph_extract_jobs.py
- hugegraph-llm/src/tests/nodes/test_request_graph_config.py
- hugegraph-llm/src/hugegraph_llm/api/models/graph_extract_requests.py
|
Follow-up Local verification after the follow-up:
Remote CI on this PR is now green for Ruff, HugeGraph-LLM, HG-Python-Client, license/header checks. |
|
Update pushed in 9cecb9b:
Local verification:
|
|
Updated in Addressed the latest property-validation review items:
Local verification:
CI for |
imbajin
left a comment
There was a problem hiding this comment.
Focused follow-up review: leaving two correctness comments only. I am intentionally not commenting on the global concurrency/backpressure topic in this pass.
| for vertex in vertices: | ||
| if not isinstance(vertex, dict) or "label" not in vertex or "properties" not in vertex: | ||
| raise FlowOutputValidationError("canonical property graph vertex must include label and properties") | ||
| required_edge_keys = {"label", "outV", "outVLabel", "inV", "inVLabel", "properties"} |
There was a problem hiding this comment.
这里的 /graph/extract workflow output contract 校验还偏弱:当前只确认 vertex/edge 是 dict 且包含必要字段,但没有校验 label/outV/outVLabel/inV/inVLabel 是非空 string,也没有校验 properties 是 object。
这会导致 /graph/import 会拒绝的非法 property graph payload,在 /graph/extract 里可能被当成 succeeded 返回。虽然正常 LLM operator 路径已经有一轮过滤,但 service 层是 workflow output 的最后边界,建议和 GraphImportRequest.validate_data() 的 contract 对齐,至少补齐:
vertex.label非空 string;vertex.properties是 dict;edge.label/outV/outVLabel/inV/inVLabel都是非空 string;edge.properties是 dict。
建议补一个 contract test:scheduler 返回 {"vertices": [{"label": "person", "properties": null}], "edges": []} 时,/graph/extract 应返回 GRAPH_EXTRACT_INVALID_FLOW_OUTPUT,而不是成功响应。
There was a problem hiding this comment.
Fixed in 10524d2. GraphExtractService now validates workflow property-graph output against the import contract: vertex label must be a non-empty string, vertex properties must be an object, edge label/outV/outVLabel/inV/inVLabel must be non-empty strings, and edge properties must be an object. Added endpoint/service regression coverage where the scheduler returns properties=null and /graph/extract responds with GRAPH_EXTRACT_INVALID_FLOW_OUTPUT.
| # TODO: transform to Enum first (better in earlier step) | ||
| data_type = property_label["data_type"] | ||
| cardinality = property_label["cardinality"] | ||
| if not self._check_property_data_type(data_type, cardinality, value): |
There was a problem hiding this comment.
这里的属性类型校验语义和 request model 里的 inline-schema 校验不一致,建议合并前统一,否则 named schema / 内部 workflow 路径会和 public request path 表现不同。
具体例子:request 层 _is_schema_property_single_value() 对 BYTE/INT/LONG 明确排除了 bool,但 Commit2Graph._check_single_data_type() 用 isinstance(value, int),Python 下 True/False 会被当成 int 接受;同时 request/operator 两边对 FLOAT/DOUBLE 都只接受 float,会把 JSON 里的整数 1 拒掉,是否符合 HugeGraph numeric 语义需要确认。
建议把 property value validator 提成一个共享 helper,GraphImportRequest 和 Commit2Graph 共用:
- integer 类型排除 bool;
- float/double 是否接受 int 要和 HugeGraph 实际行为对齐;若接受 numeric,建议允许
int | float但排除 bool; - named schema 和 inline schema 路径保持同一套 contract。
同时补 operator 层测试覆盖:INT=True、DOUBLE=1、以及 named schema 绕过 request validator 时的行为。
There was a problem hiding this comment.
Fixed in 10524d2. Property value validation is now shared in hugegraph_llm.utils.schema_property and used by both GraphImportRequest and Commit2Graph. Integer types exclude bool, and FLOAT/DOUBLE accept JSON numeric int/float values while still excluding bool. Added request and operator coverage for INT=True, DOUBLE=1, and the named/internal import path that bypasses request-side inline-schema validation.
|
Updated in Addressed the latest review items:
Local verification:
The branch was rebased onto current |
|
Correction on the previous workflow note: I checked
|
Summary
content_type/contentsupport for raw text and pre-split chunks, request-bounded chunk parallelism, metadata, and structured error responses.Relation to apache#351
This builds on the initial synchronous
POST /graph/extractendpoint from apache#351. The deprecatedtextsalias remains accepted: a string maps tocontent_type=text, and a list maps to pre-split chunks. Multi-document extraction remains caller-managed through multiple API requests instead of hidden batch semantics in the synchronous endpoint.Write API Safety
/graph/importand/graph/extract-and-importrequirewrite_to_graph=true.client_config.graphso the target graph is explicit in the request and response metadata.Job Endpoint Notes
/graph/extract/jobsuses an in-memory, process-local job store.Tests
uv run ruff format --check .uv run ruff check .uv run pytest hugegraph-llm/src/tests/api/test_graph_extract_api.py hugegraph-llm/src/tests/api/test_graph_import_api.py hugegraph-llm/src/tests/api/test_graph_extract_jobs.py -qSKIP_EXTERNAL_SERVICES=true uv run pytest hugegraph-llm/src/tests/api -v --tb=shortSKIP_EXTERNAL_SERVICES=true uv run pytest hugegraph-llm/src/tests/config/ hugegraph-llm/src/tests/document/ hugegraph-llm/src/tests/operators/ hugegraph-llm/src/tests/models/ hugegraph-llm/src/tests/indices/ hugegraph-llm/src/tests/test_utils.py -v --tb=shortReview
Summary by CodeRabbit
发布说明
新功能
/graph/import和组合提取-导入端点/graph/extract-and-import改进