Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Implement Java/JNI function applyNullMask #16764

Open
ttnghia opened this issue Sep 6, 2024 · 2 comments
Open

[FEA] Implement Java/JNI function applyNullMask #16764

ttnghia opened this issue Sep 6, 2024 · 2 comments
Labels
feature request New feature or request good first issue Good for newcomers

Comments

@ttnghia
Copy link
Contributor

ttnghia commented Sep 6, 2024

We need a function in Java/JNI to apply a null mask onto some provided input column. This functionality is used a lot in Spark-rapids plugin.

Currently, we can achieve this using copy_if_else but we have to create and provide a null scalar for the else (if not valid) option, which is a bit inconvenient and incurs additional maintenance overhead.

@ttnghia ttnghia added feature request New feature or request good first issue Good for newcomers labels Sep 6, 2024
@bdice
Copy link
Contributor

bdice commented Sep 6, 2024

@ttnghia Can you explain your concerns about maintenance? It seems like copy_if_else is a good way to do this.

Otherwise, can you use cudf::column::set_null_mask directly?

@ttnghia
Copy link
Contributor Author

ttnghia commented Sep 6, 2024

Yes, copy_if_else is the current workaround for doing it:

std::unique_ptr<column> copy_if_else(
  scalar const& lhs,
  column_view const& rhs,
  column_view const& boolean_mask,

However, we always have to create the null scalar (lhs) and destroy it after using. That is the overhead that I mentioned.

For implementing this feature, we will need superimpose_nulls instead of cudf::column::set_null_mask since we are dealing with generic (nested) columns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants