Skip to content

[Data] Skip schema call in to_tf if tf.TypeSpec is provided#42917

Merged
c21 merged 5 commits intoray-project:masterfrom
c21:tf
Feb 8, 2024
Merged

[Data] Skip schema call in to_tf if tf.TypeSpec is provided#42917
c21 merged 5 commits intoray-project:masterfrom
c21:tf

Conversation

@c21
Copy link
Contributor

@c21 c21 commented Feb 1, 2024

Why are these changes needed?

This PR is to skip Dataset.schema() call from Dataset.to_tf(). Dataset.schema() relies on limit to early stop execution, and sometimes the stop is not triggered timely so a lot of tasks get executed. This introduced problem to cause memory spilling. In addition, sometimes, it returns None (does not work with limit push down), and it breaks followed logic in to_tf, which all relies on schema() to work.

In this PR:

  • Introduce two optional parameters in to_tf: feature_type_spec and label_type_spec (by default they are None). So user can set tf.TypeSpec explicitly and the Dataset.schema() call will be skipped.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@c21 c21 changed the title [Data] Auto-infer tf.TypeSpec of columns in to_tf [Data] Skip schema call in to_tf if tf.TypeSpec is provided Feb 2, 2024
@c21 c21 requested a review from omatthew98 as a code owner February 7, 2024 19:23
c21 added 5 commits February 7, 2024 12:16
Signed-off-by: Cheng Su <scnju13@gmail.com>
Signed-off-by: Cheng Su <scnju13@gmail.com>
Signed-off-by: Cheng Su <scnju13@gmail.com>
Signed-off-by: Cheng Su <scnju13@gmail.com>
Signed-off-by: Cheng Su <scnju13@gmail.com>
@c21 c21 merged commit c90b476 into ray-project:master Feb 8, 2024
@c21 c21 deleted the tf branch February 8, 2024 21:40
c21 added a commit to c21/ray that referenced this pull request Feb 9, 2024
…y-project#42917)

This PR is to skip `Dataset.schema()` call from `Dataset.to_tf()`. `Dataset.schema()` relies on `limit` to early stop execution, and sometimes the stop is not triggered timely so a lot of tasks get executed. This introduced problem to cause memory spilling. In addition, sometimes, it returns `None` (does not work with limit push down), and it breaks followed logic in `to_tf`, which all relies on `schema()` to work.

In this PR:
* Introduce two optional parameters in `to_tf`: `feature_type_spec` and `label_type_spec` (by default they are `None`). So user can set `tf.TypeSpec` explicitly and the `Dataset.schema()` call will be skipped.

Signed-off-by: Cheng Su <scnju13@gmail.com>
zhe-thoughts pushed a commit that referenced this pull request Feb 9, 2024
…2917) (#43073)

This PR is to skip `Dataset.schema()` call from `Dataset.to_tf()`. `Dataset.schema()` relies on `limit` to early stop execution, and sometimes the stop is not triggered timely so a lot of tasks get executed. This introduced problem to cause memory spilling. In addition, sometimes, it returns `None` (does not work with limit push down), and it breaks followed logic in `to_tf`, which all relies on `schema()` to work.

In this PR:
* Introduce two optional parameters in `to_tf`: `feature_type_spec` and `label_type_spec` (by default they are `None`). So user can set `tf.TypeSpec` explicitly and the `Dataset.schema()` call will be skipped.

Signed-off-by: Cheng Su <scnju13@gmail.com>
ratnopam pushed a commit to ratnopam/ray that referenced this pull request Feb 11, 2024
…y-project#42917)

This PR is to skip `Dataset.schema()` call from `Dataset.to_tf()`. `Dataset.schema()` relies on `limit` to early stop execution, and sometimes the stop is not triggered timely so a lot of tasks get executed. This introduced problem to cause memory spilling. In addition, sometimes, it returns `None` (does not work with limit push down), and it breaks followed logic in `to_tf`, which all relies on `schema()` to work.

In this PR:
* Introduce two optional parameters in `to_tf`: `feature_type_spec` and `label_type_spec` (by default they are `None`). So user can set `tf.TypeSpec` explicitly and the `Dataset.schema()` call will be skipped.

Signed-off-by: Cheng Su <scnju13@gmail.com>
Signed-off-by: Ratnopam Chakrabarti <ratnopamc@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants