Skip to content

[Improve][Core] Support transform and value constraint option rules in CLI - #11109

Merged
davidzollo merged 2 commits into
apache:devfrom
yzeng1618:dev-cli-fix
Jun 19, 2026
Merged

[Improve][Core] Support transform and value constraint option rules in CLI#11109
davidzollo merged 2 commits into
apache:devfrom
yzeng1618:dev-cli-fix

Conversation

@yzeng1618

Copy link
Copy Markdown
Member

Purpose of this pull request

#10977
This PR improves SeaTunnel CLI connector metadata handling by supporting transform plugin option rules and value constraints.

Does this PR introduce any user-facing change?

Yes.

Previously, SeaTunnel CLI metadata lookup focused on source and sink plugins, so transform plugin options and value constraints could be missing from generated guidance.

After this change, CLI users can retrieve transform plugin metadata, and generated prompts/details include value constraints such as required non-blank transform SQL queries. This helps the CLI produce more accurate SeaTunnel job configurations.

How was this patch tested?

Added unit tests

Check list

@DanielLeens DanielLeens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. I reviewed the full current diff and traced the actual CLI metadata flow end to end rather than only looking at the formatter helpers.

What this PR fixes

  • User pain: the CLI already had source/sink metadata plumbing, but transform option rules and valueConstraints were still falling out of the real consumption path, even when the engine API or exported runtime metadata already exposed them.
  • Fix approach: this PR extends the get_connector_info tool contract to allow connector_type="transform", preserves valueConstraints when parsing runtime API / runtime JSON metadata, and teaches SkillExecutor to collect metadata targets from source, sink, and transform through the same path.
  • One-line summary: this makes transform metadata a first-class input to the CLI instead of treating it as a side channel.

Simple example

  • Before this change, a request that used a Sql transform could still enrich source/sink metadata, but the transform-side option rules and constraint text were not part of the same CLI metadata flow.
  • After this change, the CLI can surface transform details through get_connector_info, include transform metadata in prompt enrichment, and render any returned valueConstraints instead of dropping them.

Full runtime path I checked

transform detail query
  -> seatunnel-cli/seatunnel_cli/agents.py:69-91
     -> get_connector_info accepts connector_type = source / sink / transform
  -> seatunnel-cli/seatunnel_cli/connectors.py:get_connector_detail() [1006-1057]
     -> _fetch_option_rules("transform", name) or _load_runtime_metadata()
     -> _api_response_to_detail() / _runtime_metadata_to_detail() [183-195, 407-419]
     -> _format_connector_detail_typed()
        -> render "Value Constraints" when present [1115-1118]

pipeline generation path
  -> seatunnel-cli/seatunnel_cli/skills.py:parse_structured_plan()
     -> PipelineSlot.transform is preserved
  -> SkillExecutor.fill_and_check() [530-557]
     -> _metadata_targets_for_pipeline() [289-298]
     -> collect required options from source / sink / transform
  -> SkillExecutor.fetch_all_metadata() [559-575]
     -> fetch_connector_metadata(name, type)
     -> format_metadata_for_prompt()
        -> render required / conditional / value constraint sections [627-630]
  -> Config generation prompt now receives transform metadata through the normal path

Review result

  • seatunnel-cli/seatunnel_cli/agents.py:69-91 now exposes the correct transform-aware tool contract.
  • seatunnel-cli/seatunnel_cli/connectors.py:183-195,251-275,407-419,627-630,1006-1057,1115-1118,1200-1214 now preserves and renders transform-side valueConstraints from both runtime API and runtime JSON sources.
  • seatunnel-cli/seatunnel_cli/skills.py:276-298,530-575 removes the old source/sink-only assumption and routes transform metadata through the same collection path used by the planner/executor flow.
  • I did not find a source-level correctness blocker on the current head.

Compatibility and side effects

  • Compatibility: fully backward compatible from my side. This does not change Java-side option-rule contracts, connector config keys, defaults, or serialization formats. It only lets the CLI consume metadata that already exists.
  • Performance: the only extra work is one additional metadata lookup per unique transform (name, type) pair, and that path is still deduplicated.
  • Error handling / logging: I did not see a new exception-swallowing or sensitive-logging issue in the changed code.

Tests and stability

  • The added unit coverage exercises runtime API parsing, runtime JSON parsing, prompt formatting, transform detail lookup, runtime transform listing, and SkillExecutor.fetch_all_metadata() hitting the transform path.
  • The new tests in seatunnel-cli/tests/test_connector_metadata.py:25-213 are deterministic mock-based unit tests. I do not see a flaky pattern: no timing waits, no environment-sensitive resources, no port conflicts, no shared global state leakage.
  • Stability rating: Stable.
  • The remaining blind spot is that there is still no live engine / exported-runtime-metadata integration test for this CLI path, and fill_and_check()'s transform-required-options branch in seatunnel-cli/seatunnel_cli/skills.py:530-557 is not covered directly yet. I would treat that as follow-up hardening, not a merge blocker.

Merge conclusion

Conclusion: can merge

  1. Blocking items
  • None from my side on the latest head.
  1. Suggested follow-up
  • If you want one extra hardening step later, add one focused test around the fill_and_check() clarification path in seatunnel-cli/seatunnel_cli/skills.py:530-557, since this PR also changes that branch.

Overall, this is a clean and focused fix. It closes a real metadata-consumption gap in the CLI without widening the change scope into the runtime itself.

At the time I checked the PR metadata, labeler and Notify test workflow were already green, and Build was still queued. So from my side the remaining gate is CI completion, not a new source-level blocker.

@yzeng1618
yzeng1618 requested a review from SEZ9 June 17, 2026 01:07
@SEZ9

SEZ9 commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

@yzeng1618
Great work on this PR! Extending the metadata lookup to include transform plugins and supporting valueConstraints is a crucial enhancement. It directly addresses the blind spots in the LLM prompt generation from the initial CLI implementation (#10789) and will significantly reduce runtime errors caused by missing constraints (like blank SQL queries).

The implementation looks solid overall. To make this feature even more robust and complete, here are a few suggestions regarding tests and documentation:

1. Test Coverage Enhancements:
While the current unit tests (test_connector_metadata.py) excellently cover the happy paths, I suggest adding a few edge-case tests for _value_constraint_to_dict() and _format_value_constraint(). Specifically:

  • Fault tolerance: What happens if the conditionTree is malformed, missing entirely, or if expectValue is an unexpected type? Ensuring graceful degradation without throwing unhandled exceptions will make the CLI more resilient.
  • Operator formatting: Currently, custom operators like "extension" might produce slightly awkward prompt strings (e.g., `query` extension must not be blank). We might want to ensure the LLM fully understands this phrasing, or eventually map these to more natural language.

2. Documentation Updates:
Since this is a user-facing improvement that enhances the capabilities of the CLI, we should update the documentation to reflect it:

  • Please update seatunnel-cli/README.md and seatunnel-cli/README.zh-CN.md. Under the "Features" or "Architecture" sections, explicitly mention that the CLI now provides full metadata, option rules, and constraint validation for Transform plugins as well.
  • It would also be helpful to add a brief note in the interactive /connectors command help section indicating that transform constraints are now fully supported.

Thanks Again!

@DanielLeens

Copy link
Copy Markdown
Contributor

Thanks @SEZ9 for calling out the CLI metadata gap. I rechecked the PR state before replying: from Daniel's side there is still no newer code after the head I already reviewed (c89c44a), so I am not starting another full review round on unchanged code. If @yzeng1618 pushes an update or wants clarification on any of the remaining review items, I am happy to re-check the latest head.

@yzeng1618

Copy link
Copy Markdown
Member Author

Thanks @SEZ9 for the careful review and the concrete suggestions! All of these are addressed in the latest head ab80b9e ("Harden CLI transform metadata constraints"). A point-by-point rundown:

1. Test Coverage Enhancements

  • Fault tolerance: _value_constraint_to_dict() already guards the condition tree (tree = constraint.get("conditionTree") or {} plus an isinstance(tree, dict) check), so a malformed/missing tree degrades gracefully to just the expression. I've now added a dedicated edge-case test, test_value_constraint_tolerates_missing_or_malformed_condition_tree, covering a missing tree, a None tree, and a non-dict tree. For unexpected expectValue types I added _constraint_value_to_text(): strings pass through, structured values are serialized via json.dumps, and anything non-serializable falls back to str(). This is covered by test_value_constraint_formats_complex_expect_value_as_json (a list value renders as ["batch", "streaming"]), with no unhandled exceptions on any branch.

  • Operator formatting: Good catch. _format_value_constraint() now special-cases the extension operator, so it renders - query must not be blank instead of the awkward - query extension must not be blank. The prompt test asserts the natural phrasing and explicitly assertNotIns the old one, so the LLM only ever sees the clean wording.

2. Documentation Updates

  • Updated both seatunnel-cli/README.md and seatunnel-cli/README.zh-CN.md: a new "Transform Metadata" entry under Features, a clarified Architecture/bundled-metadata section stating that source, sink, and transform plugins all carry full option rules and value constraints, and an updated /connectors row.

  • Added a note to the interactive /connectors output in cli.py: "Transform option rules and value constraints are supported during generation."

Happy to tweak the constraint phrasing further if you'd prefer different wording. Thanks again!

@yzeng1618
yzeng1618 requested a review from DanielLeens June 18, 2026 05:44

@nzw921rx nzw921rx left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 good job

@DanielLeens DanielLeens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up update. I re-reviewed the latest head from scratch, with extra attention on the two gaps Daniel called out in the previous round: transform-aware required-option coverage in fill_and_check() and the user-facing docs/help text.

What this PR fixes:

  • User pain: transform metadata and value constraints were available from the runtime metadata sources, but they were not fully consumed through the CLI's real prompt-enrichment and detail-display path.
  • Fix approach: the PR keeps source/sink/transform on one metadata-target path, preserves transform valueConstraints in connectors.py, and now also adds the missing fill_and_check() regression plus README/CLI text updates.
  • One-line summary: the current head closes the remaining CLI-side gaps and makes transform metadata support consistent across lookup, prompting, testing, and docs.

Runtime path I checked:

transform detail lookup
  -> agents.py:get_connector_info schema
  -> connectors.py:get_connector_detail(...)
      -> query_types include source/sink/transform
      -> runtime API / runtime JSON metadata
      -> _api_response_to_detail() / _runtime_metadata_to_detail()
          -> preserve value_constraints
      -> _format_connector_detail_typed()
          -> print Value Constraints

config generation
  -> skills.py:_metadata_targets_for_pipeline()
      -> source + sink + transform
  -> SkillExecutor.fill_and_check()
      -> transform required options now participate in missing-info detection
  -> SkillExecutor.fetch_all_metadata()
      -> transform metadata also reaches prompt enrichment

Key findings:

  • The normal path really does hit the change: once a pipeline contains a transform, both fill_and_check() and fetch_all_metadata() now route through _metadata_targets_for_pipeline().
  • The latest commit addresses the specific follow-ups from the previous review:
    • it adds the fill_and_check() regression for transform required options
    • it keeps valueConstraints visible in both prompt formatting and connector detail rendering
    • it updates both README.md and README.zh-CN.md, plus the CLI help text
  • I do not see a regression on the source/sink path, and the new tests remain deterministic and lightweight.

Testing note:

  • I did source-level review only in this pass.
  • I did not run the PR locally.
  • The updated unit tests are stable from a structure perspective: no timing, port, network, or ordering dependency.

Conclusion: can merge

  1. Blocking items
  • None from the current head.
  1. Suggested follow-up
  • None beyond ordinary CI completion.

Overall, the current version looks consistent to me. The transform metadata path, value-constraint rendering, regression coverage, and docs are now aligned, so I do not see a remaining blocker on the code path I reviewed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants