Skip to content

DataFrameColumn: Unable to Filter by Null via ElementWiseXYZ Methods (System.NotSupportedException) #6110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #6144
JD-Robbs opened this issue Mar 3, 2022 · 0 comments · Fixed by #6723
Closed
Tracked by #6144
Labels
Microsoft.Data.Analysis All DataFrame related issues and PRs P2 Priority of the issue for triage purpose: Needs to be fixed at some point.
Milestone

Comments

@JD-Robbs
Copy link

JD-Robbs commented Mar 3, 2022

System Information (please complete the following information):

  • OS & Version: Windows 10
  • ML.NET Version: Latest
  • .NET Version: .NET 6.0

Describe the bug
When passing null into ElementWiseXYZ methods, a System.NotSupportedException is thrown: 'Specified method is not supported.'

To Reproduce
Steps to reproduce the behavior:

        var col = new DoubleDataFrameColumn("col", new double?[] {1.23, null, 2, 3 });
        var dfTest = new DataFrame(col);

        var filtered2Df = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(2));
        var filteredNullDf = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(null));

Expected behavior
I expected the resulting DataFrame stored in filteredNullDf to not include the value null - much like filered2Df does not include the value 2.

The only way of making it work is something like this:

         dfTest.Filter(new PrimitiveDataFrameColumn<bool>("filter", dfTest["col"].Cast<double?>().Select(v => v != null)));
@luisquintanilla luisquintanilla added the Microsoft.Data.Analysis All DataFrame related issues and PRs label Mar 8, 2022
@michaelgsharp michaelgsharp added the P2 Priority of the issue for triage purpose: Needs to be fixed at some point. label Mar 8, 2022
@michaelgsharp michaelgsharp added this to the ML.NET Future milestone Mar 8, 2022
@ghost ghost added the in-pr label Jun 6, 2023
@ghost ghost removed the in-pr label Jul 7, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Aug 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Microsoft.Data.Analysis All DataFrame related issues and PRs P2 Priority of the issue for triage purpose: Needs to be fixed at some point.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants