-
Notifications
You must be signed in to change notification settings - Fork 329
Support for Bucketizer #378
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@suhsteve can you help reviewing this? |
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Show resolved
Hide resolved
Co-Authored-By: Steve Suh <[email protected]>
Co-Authored-By: Steve Suh <[email protected]>
Co-Authored-By: Steve Suh <[email protected]>
Co-Authored-By: Steve Suh <[email protected]>
I have updated after the comments, let me know whether the |
@suhsteve I have re-written this now so that SerDe can understand a double[][] - I used a 'A' as the identifier, if there are any rules about which letter to use let me know and i'll change it. |
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
I've made the changes after the code review, let me know any other changes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor change. LGTM! Thanks @GoEddie
Co-Authored-By: Steve Suh <[email protected]>
Great thanks :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have minor comments, but generally looks good to me.
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
src/csharp/Microsoft.Spark.E2ETest/IpcTests/ML/Feature/BucketizerTests.cs
Outdated
Show resolved
Hide resolved
src/scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/api/dotnet/SerDe.scala
Outdated
Show resolved
Hide resolved
src/scala/microsoft-spark-2.4.x/src/main/scala/org/apache/spark/api/dotnet/SerDe.scala
Outdated
Show resolved
Hide resolved
src/scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/api/dotnet/SerDe.scala
Show resolved
Hide resolved
src/scala/microsoft-spark-3.0.x/src/main/scala/org/apache/spark/api/dotnet/SerDe.scala
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @GoEddie!
This implements Bucketizer (https://spark.apache.org/docs/latest/ml-features#bucketizer) and completes #313
This implements the concrete methods on Bucketizer that are needed to run the transform.