release v4.7.0

[jameslamb]

It has been almost 1 year since the last release of LightGBM: https://github.com/microsoft/LightGBM/releases/tag/v4.6.0

Many changes have built up on master since then. Let's use this PR to decide on a list of what else we want to get into a release, then try to put up a release soon.

Release checklist:

Copied from #6796 with a few changes.

deadline

None that I know of yet, though there are WARNINGs on the check results at https://cran.r-project.org/web/checks/check_results_lightgbm.html which might turn into errors or a CRAN threat request.

before merge

• Update VERSION.txt number.
• Update version in Appveyor config file.
• Update version in configure file of R-package: /gha run r-configure.
• Change development.mode from unreleased to release in pkgdown config file.
• Update version in python-package/pyproject.toml
• Add release branch to RTD versions as both "Active" and "Hidden", trigger a new build, check docs
• docs changes are called out with appropriate Sphinx directives ((https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.htm):
  • additions = .. version-added::
  • deprecations = .. version-deprecated::
  • other changes = .. version-changed::
• All new parameters in config.h have *New in version {version}* comments added
• resolve all issues and merge all PRs we want to get into the release:
  • [python-package] Add decision_function() to LGBMClassifier #7159
  • [python-package] support all-0 sample weights in scikit-learn 1.9 #7128
  • [CUDA] Fix division by zero in histogram construction with discrete data #7123
  • [CUDA] SIGFPE (Floating point exception) with discrete data when n_unique_values * n_features exceeds threshold #7122
  • [python-package] Add return type annotations to predict methods in sklearn module #7116
  • [ROCm] add ROCm support (pt. 2) #7039
  • [CUDA] [fix] enable max_depth in cuda tree learner (fix #6969) #6994 (which needs [CUDA] [fix] enable max_depth in cuda tree learner - fix build #7017)
  • [RFC] [ci] remove Azure DevOps CI jobs? #6949
  • [python-package] Cannot import lightgbm after installation on windows with uv, lib_lightgbm.dll not found #6936
  • [python-package] scikit-learn fit() methods: add eval_X, eval_y, deprecate eval_set #6857
  • NuGet package is missing README #6827
  • [cpp] fix: to_if_else missing check for threshold when missing_type is not none #6818
  • deprecation warnings for [R-Package] Fix typo in lgb.interprete #6732
• Run the valgrind checks (docs link)

after merge

• Push v* tag to trigger the release action (TODO: update based on the outcome of [RFC] [ci] remove Azure DevOps CI jobs? #6949 (comment))
• Update stable tag at GitHub.

code to update tags (click me)

NEW_VERSION="4.7.0"
git checkout master
git pull upstream master
git fetch upstream --tags
git tag -d stable
git push upstream :refs/tags/stable
git tag stable
git tag "v${NEW_VERSION}"
git push upstream stable "v${NEW_VERSION}"

• Add new tag to RTD versions and trigger a new build.
• Trigger a new build of the stable version docs on readthedocs (see [ci] Fix version matching between RTD pages and R-package pages #6673 (comment))
• Remove the release branch release/v{n}.{n}.{n} from RTD versions

after the release is created by Azure DevOps job

• Add release notes from release-drafter to the automatically-created GitHub release, convert it from Draft to published.
• Manually attach arm64 macOS wheel to the release (no longer necessary as of [RFC] [ci] remove Azure DevOps CI jobs? #6949)
• Manually attach aarch64 Linux wheel to the release (no longer necessary as of [RFC] [ci] remove Azure DevOps CI jobs? #6949)
• Upload release to CRAN
• Upload release to test PyPI
• Upload release to PyPI.

code to publish to PyPI (summary)

mkdir /tmp/lgb-release
cd /tmp/lgb-release

NEW_VERSION="4.7.0"

# NOTE: requires a GitHub personal access token with "repo" scope
git checkout v${NEW_VERSION}
bash .ci/download-artifacts.sh $(git rev-parse HEAD)

# NOTE: manually downloaded arm64 wheels from CI

# config docs: https://packaging.python.org/en/latest/specifications/pypirc/
twine upload \
    -r testpypi \
    ./release-artifacts/*.whl

twine upload \
    -r testpypi \
    ./release-artifacts/lightgbm-${NEW_VERSION}.tar.gz

(gh is the GitHub CLI, see https://cli.github.com/manual/gh_release_download)

Then confirmed that installing the latest wheel works.

pip install -i https://test.pypi.org/simple/ "lightgbm==${NEW_VERSION}"
python ./examples/python-guide/logistic_regression.py

Then pushed them to real PyPI.

cd /tmp/lgb-release

twine upload \
    ./release-artifacts/*.whl

twin upload \
    ./release-artifacts/lightgbm-${NEW_VERSION}.tar.gz

• update conda-forge recipe
  • usually a PR will automatically be opened a day or 2 after publishing an sdist to PyPI, e.g. lightgbm v4.5.0 conda-forge/lightgbm-feedstock#60
• Update cran-comments.md when new release is accepted on CRAN.
• Upload release to NuGet.
• Update version and commit hash in Homebrew formula (docs)
• put up another PR incrementing development version here in GitHub (example: bump development version to 4.4.0.99 #6485)

code to open that dev version PR (click me)

RELEASE_PR_NUMBER="6439"
RELEASE_VERSION=$(cat ./VERSION.txt)
DEV_VERSION="${RELEASE_VERSION}.99"

git checkout -b ci/dev-version

echo "${DEV_VERSION}" > ./VERSION.txt
sed \
    -i .bak \
    "s|${RELEASE_VERSION}|${DEV_VERSION}|g" \
    .appveyor.yml

sed \
    -i .bak \
    "s|version = \"${RELEASE_VERSION}\"|version = \"${DEV_VERSION}\"|g" \
    ./python-package/pyproject.toml

sed \
    -i .bak \
    's|mode\: release|mode\: unreleased|g' \
    ./R-package/pkgdown/_pkgdown.yml

docker run \
    --rm \
    -v $(pwd):/opt/LightGBM \
    -w /opt/LightGBM \
    ubuntu:22.04 \
    ./R-package/recreate-configure.sh

git add \
    ./.appveyor.yml \
    ./R-package/configure \
    ./R-package/pkgdown/_pkgdown.yml \
    ./VERSION.txt \
    ./python-package/pyproject.toml

commit_msg="bump development version to ${DEV_VERSION}"
git commit -m "${commit_msg}"
git push upstream ci/dev-version

gh pr create \
    --repo microsoft/LightGBM \
    --base 'master' \
    --label 'maintenance' \
    --title "${commit_msg}" \
    --body "now that v${RELEASE_VERSION} has been released: #${RELEASE_PR_NUMBER}"

Notes for Reviewers

I believe this should be v4.7.0 instead of v4.6.1 because of some small breaking changes:

• [ci] [docs] update handling of artifacts, stop publishing zip archive of complete source code #7080
• [R-package] [ci] drop support for R 3.x (fixes #7023) #7038
• [ci] [C++] drop support for Visual Studio 2015 (fixes #6803) #7022
• [c++] default to C++17 for most builds #7016
• [CUDA] default to C++17 for CUDA builds #7013
• [ci] Drop support for Python 3.7 and Python 3.8 #6896
• [python-package] drop support for h2o datatable #6894
• [python-package] Load parameters from model string #6852
• [R-package] make finalize() in R6 classes private #6833

See https://github.com/microsoft/LightGBM/releases.

[jameslamb]

@StrikerRUS @shiyu1994 @guolinke @borchero @jmoralez please edit the description if you want to add other items for the release.

@shiyu1994 will you be able to help with these soon?

• [ROCm] add ROCm support (pt. 2) #7039
• [CUDA] [fix] enable max_depth in cuda tree learner (fix #6969) #6994
• [CUDA] Fix division by zero in histogram construction with discrete data #7123
• [CUDA] SIGFPE (Floating point exception) with discrete data when n_unique_values * n_features exceeds threshold #7122

I'm not sure we have other maintainers / contributors who are available to help with them. If you cannot help get them merged, we may have to skip them for the release.

[borchero]

Looks great, thanks for the summary! I would have loved to include proper polars support in the next release but I think this will be a little tight on my end 🙈

[shiyu1994]

@shiyu1994 will you be able to help with these soon?

Thank you for help with the release. Will try to finish these soon.

[jameslamb]

I would have loved to include proper polars support in the next release but I think this will be a little tight on my end 🙈

I would have too, but that's ok. If it doesn't make it this release hopefully it can be in the next one.

[shiyu1994]

Given that our Slack workspace has expired due to the termination of Slack service in China, I think we should start a new group for the communication of our maintainers. What do you think is the most proper approach? It can be a new slack workspace, or an alternative like Pumble, or a Wechat group.

@jameslamb @StrikerRUS @borchero @guolinke

[jameslamb]

I'm willing to try WeChat!

But I want to be sure it is private to maintainers, so we can discuss private topics like security reports and possible new maintainers. So probably should be different from #5715

Could you create a group and send us email invitations?

[shiyu1994]

I'm willing to try WeChat!

But I want to be sure it is private to maintainers, so we can discuss private topics like security reports and possible new maintainers. So probably should be different from #5715

Could you create a group and send us email invitations?

Sure! Will create a group and send the QR code to you.

[shiyu1994]

@jameslamb @StrikerRUS @borchero Sent the QR code of WeChat group to your email, please check and join the group.

[jameslamb]

@jameslamb @StrikerRUS @borchero Sent the QR code of WeChat group to your email, please check and join the group.

I'm getting "Too many attempts. Try again later." Will try again tomorrow.