[Data] - Optimize memory usage for One Hot Encoder (ray-project#56565)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Previously, the vector that was holding the values from OneHotEncoder
was of type `int64`. We can reduce this to `uint8`, which should result
in 8x lower memory usage

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Goutam V. <goutam@anyscale.com>
Signed-off-by: Marco Stephan <marco@magic.dev>

Author: goutamvenkat-anyscale

python/ray/data/preprocessors/encoder.py

@@ -285,10 +285,18 @@ def safe_get(v: Any, stats: Dict[str, int]):
 
             stats = self.stats_[f"unique_values({column})"]
             num_categories = len(stats)
-            one_hot = np.zeros((len(df), num_categories), dtype=int)
+            one_hot = np.zeros((len(df), num_categories), dtype=np.uint8)
+            # Integer indices for each category in the column
             codes = df[column].apply(lambda v: safe_get(v, stats)).to_numpy()
-            valid_rows = codes != -1
-            one_hot[np.nonzero(valid_rows)[0], codes[valid_rows].astype(int)] = 1
+            # Filter to only the rows that have a valid category
+            valid_category_mask = codes != -1
+            # Dimension should be (num_rows, ) - 1D boolean array
+            non_zero_indices = np.nonzero(valid_category_mask)[0]
+            # Mark the corresponding categories as 1
+            one_hot[
+                non_zero_indices,
+                codes[valid_category_mask],
+            ] = 1
             df[output_column] = one_hot.tolist()
 
         return df