Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] fix some shellcheck warnings in package-building scripts #6641

Merged
merged 36 commits into from
Oct 17, 2024

Conversation

vnherdeiro
Copy link
Contributor

@vnherdeiro vnherdeiro commented Sep 4, 2024

Contributes to #6498 continues from #6621

Now I am tackling ./build-python.sh and ./build-cran-package.sh

All feedback is welcome!

Please forgive the polluted commit history.

Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

Left a first round of comments for your consideration.

build-cran-package.sh Outdated Show resolved Hide resolved
build-cran-package.sh Outdated Show resolved Hide resolved
build-cran-package.sh Outdated Show resolved Hide resolved
build-cran-package.sh Outdated Show resolved Hide resolved
build-python.sh Outdated Show resolved Hide resolved
@jameslamb
Copy link
Collaborator

Please forgive the polluted commit history.

No worries! We'll squash all of the history when this is merged anyway.

@vnherdeiro
Copy link
Contributor Author

That is a prompt review @jameslamb Thanks, will amend soonish!

@vnherdeiro
Copy link
Contributor Author

vnherdeiro commented Sep 7, 2024

@jameslamb thanks for the review and code suggestions about the more extent quotemarks! Do you mind if I start a new PR soon with changes for the files you had asked my to postpone from #6621 :

  • .ci/test.sh
  • .ci/test-r-package.sh
  • .ci/setup.sh
    ?

@jameslamb jameslamb self-requested a review September 15, 2024 03:23
Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this.

I'm confused by several of these changes. As I've asked in previous reviews... please explain the reasoning for your proposes changes here. Several of these are not obvious at all, and it's difficult to review without knowing the underlying shellcheck warnings you're trying to address.

build-python.sh Outdated Show resolved Hide resolved
build-python.sh Outdated
@@ -352,26 +356,26 @@ if test "${BUILD_WHEEL}" = true; then
python -m build \
--wheel \
--outdir ../dist \
${BUILD_ARGS} \
${BUILD_ARGS:+${BUILD_ARGS}} \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, explain this. What specifically is shellcheck complaining about that led to this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://www.shellcheck.net/wiki/SC2086 The problem comes from BUILD_ARGS being empty and interpolated, it passes an argument (which is the empty string, and the command fail.s We need a ternary check for this case.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but I just pushed commits reverting this for the reasons mentioned in #6641 (comment).

The claim you've made here that this being empty would result in the command failing is simply not true. You can try this yourself:

sh build-python.sh bdist_wheel

That will reach this line with BUILD_ARGS empty, and nothing will fail. We even intentionally set BUILD_ARGS to the empty string when --precompile is passed.

LightGBM/build-python.sh

Lines 295 to 299 in 668bf5d

if test "${INSTALL}" = true; then
if test "${PRECOMPILE}" = true; then
BUILD_SDIST=true
BUILD_WHEEL=false
BUILD_ARGS=""

And I use sh build-python.sh install --precompile every time I develop on LightGBM's Python package.

build-python.sh Outdated
fi
# ref for use of '--find-links': https://stackoverflow.com/a/52481267/3986677
pip install \
${PIP_INSTALL_ARGS} \
${PIP_INSTALL_ARGS:+ -o "$PIP_INSTALL_ARGS"} \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had explained it here-above. I cannot get a link, so pasting:

This one used to raise https://www.shellcheck.net/wiki/SC2086

Here using "$BUILD_ARGS" breaks down if it's empty. It seems to suggest using an array to group all arguments possibly appended to BUILD_ARGS. So we have this ternary expression checking for string emptiness if yes, the return is null. That's the way I understand it. Per the shellcheck docs:

This is better than an unquoted value because the alternative value can be properly quoted, e.g. wget ${output:+ -o "$output"}.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had explained it here-above

I see. I had not seen #6641 (comment) when I wrote this, maybe it's another one that wasn't published originally because it was part of an unpublished review (as you mentioned in #6641 (comment)).

I've pushed commits reverting this, for the reasons explained in #6641 (comment)

build-python.sh Show resolved Hide resolved
@jameslamb
Copy link
Collaborator

Do you mind if I start a new PR soon with changes for the files you had asked my to postpone from #6621

Looks like you decided to do that in #6646. That's fine, will review there.

@jameslamb jameslamb changed the title fixes for some shellcheck warnings [ci] fix some shellcheck warnings in package-building scripts Sep 15, 2024
@jameslamb
Copy link
Collaborator

@vnherdeiro will you have time in the next week to come back and work with us on this? If not, I'll close this and fix the remaining shellcheck issues myself.

@vnherdeiro
Copy link
Contributor Author

@jameslamb I am back and willing to continue working on this.

@jameslamb
Copy link
Collaborator

Alright thanks. You can start here: #6641 (comment)

@vnherdeiro
Copy link
Contributor Author

@jameslamb I am still figuring out how github review works, I had input comments and responses to your own but missed the submit review action, hence they were not visitble to you (or anyone but me). Completed that now.

@jameslamb jameslamb dismissed their stale review October 17, 2024 02:11

I have commits on this PR now.

@jameslamb jameslamb self-requested a review October 17, 2024 02:11
Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed commits to finish this up.

Now that I have commits on here, my approval shouldn't count toward a merge. Hopefully another reviewer will be able to look soon and we can merge this.

After that, I'll put up a PR to start running shellcheck in LightGBM's CI. Thanks for your help with this project, @vnherdeiro !

Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@StrikerRUS StrikerRUS merged commit 9893615 into microsoft:master Oct 17, 2024
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants