Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs][python] allow usage and compilation of 32-bit library #2188

Merged
merged 9 commits into from
Aug 17, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/Installation-Guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ Installation Guide

Here is the guide for the build of LightGBM CLI version.

All instructions below are aimed to compile 64-bit version of LightGBM.
It is worth to compile 32-bit version only in very rare special cases of environmental limitations.
32-bit version is slow and untested, so use it on your own risk and don't forget to adjust some commands in this guide.

For the build of Python-package and R-package, please refer to `Python-package`_ and `R-package`_ folders respectively.

Also you can download artifacts of the latest successful build in master branch: |download artifacts|.
Expand Down
15 changes: 14 additions & 1 deletion python-package/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Installation
Preparation
'''''''''''

32-bit Python is not supported. Please install 64-bit version.
32-bit Python is not supported. Please install 64-bit version. If you have a strong need to install with 32-bit Python, refer to `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__.

`setuptools <https://pypi.org/project/setuptools>`_ is needed.

Expand Down Expand Up @@ -135,6 +135,17 @@ Build with MinGW-w64 on Windows
It is recommended to use **Visual Studio** for its better multithreading efficiency in **Windows** for many-core systems
(see `Question 4 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#4-i-am-using-windows-should-i-use-visual-studio-or-mingw-for-compiling-lightgbm>`__ and `Question 8 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#8-cpu-usage-is-low-like-10-in-windows-when-using-lightgbm-on-very-large-datasets-with-many-core-systems>`__).

Build 32-bit Version with 32-bit Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: sh

pip install lightgbm --install-option=--bit32

By default, installation in environment with 32-bit Python is prohibited. However, you can remove this prohibition on your own risk by passing ``bit32`` option.

It is **strongly not recommended** to use this version of LightGBM!

Install from GitHub
'''''''''''''''''''

Expand Down Expand Up @@ -163,6 +174,8 @@ Run ``python setup.py install --gpu`` to enable GPU support. All remarks from `B

Run ``python setup.py install --hdfs`` to enable HDFS support. All remarks from `Build HDFS Version section <#build-hdfs-version>`__ are actual in this case.

Run ``python setup.py install --bit32``, if you want to use 32-bit version. All remarks from `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__ are actual in this case.

If you get any errors during installation or due to any other reasons, you may want to build dynamic library from sources by any method you prefer (see `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst>`__) and then just run ``python setup.py install --precompile``.

Troubleshooting
Expand Down
27 changes: 18 additions & 9 deletions python-package/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,12 @@ def silent_call(cmd, raise_error=False, error_msg=''):
return 1


def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False,
def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False,
use_hdfs=False, boost_root=None, boost_dir=None,
boost_include_dir=None, boost_librarydir=None,
opencl_include_dir=None, opencl_library=None,
openmp_include_dir=None, openmp_library=None):
openmp_include_dir=None, openmp_library=None,
nomp=False, bit32=False):

if os.path.exists(os.path.join(CURRENT_DIR, "build_cpp")):
shutil.rmtree(os.path.join(CURRENT_DIR, "build_cpp"))
Expand Down Expand Up @@ -134,7 +135,7 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False,
else:
status = 1
lib_path = os.path.join(CURRENT_DIR, "compile", "windows", "x64", "DLL", "lib_lightgbm.dll")
if not any((use_gpu, use_mpi, use_hdfs)):
if not any((use_gpu, use_mpi, use_hdfs, nomp, bit32)):
logger.info("Starting to compile with MSBuild from existing solution file.")
platform_toolsets = ("v142", "v141", "v140")
for pt in platform_toolsets:
Expand All @@ -150,10 +151,11 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False,
if status != 0 or not os.path.exists(lib_path):
logger.warning("Compilation with MSBuild from existing solution file failed.")
if status != 0 or not os.path.exists(lib_path):
arch = "Win32" if bit32 else "x64"
vs_versions = ("Visual Studio 16 2019", "Visual Studio 15 2017", "Visual Studio 14 2015")
for vs in vs_versions:
logger.info("Starting to compile with %s." % vs)
status = silent_call(cmake_cmd + ["-G", vs, "-A", "x64"])
status = silent_call(cmake_cmd + ["-G", vs, "-A", arch])
if status == 0:
break
else:
Expand Down Expand Up @@ -222,6 +224,7 @@ class CustomInstall(install):
('mpi', None, 'Compile MPI version'),
('nomp', None, 'Compile version without OpenMP support'),
('hdfs', 'h', 'Compile HDFS version'),
('bit32', None, 'Compile 32-bit version'),
('precompile', 'p', 'Use precompiled library'),
('boost-root=', None, 'Boost preferred installation prefix'),
('boost-dir=', None, 'Directory with Boost package configuration file'),
Expand Down Expand Up @@ -249,16 +252,25 @@ def initialize_options(self):
self.hdfs = 0
self.precompile = 0
self.nomp = 0
self.bit32 = 0

def run(self):
if (8 * struct.calcsize("P")) != 64:
if self.bit32:
logger.warning("You're installing 32-bit version. "
"This version is slow and untested, so use it on your own risk.")
else:
raise Exception("Cannot install LightGBM in 32-bit Python, "
"please use 64-bit Python instead.")
open(LOG_PATH, 'wb').close()
if not self.precompile:
copy_files(use_gpu=self.gpu)
compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_mpi=self.mpi, nomp=self.nomp,
compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_mpi=self.mpi,
use_hdfs=self.hdfs, boost_root=self.boost_root, boost_dir=self.boost_dir,
boost_include_dir=self.boost_include_dir, boost_librarydir=self.boost_librarydir,
opencl_include_dir=self.opencl_include_dir, opencl_library=self.opencl_library,
openmp_include_dir=self.openmp_include_dir, openmp_library=self.openmp_library)
openmp_include_dir=self.openmp_include_dir, openmp_library=self.openmp_library,
nomp=self.nomp, bit32=self.bit32)
install.run(self)
if os.path.isfile(LOG_PATH):
os.remove(LOG_PATH)
Expand All @@ -281,9 +293,6 @@ def run(self):


if __name__ == "__main__":
if (8 * struct.calcsize("P")) != 64:
raise Exception('Cannot install LightGBM in 32-bit Python, please use 64-bit Python instead.')

CURRENT_DIR = os.path.abspath(os.path.dirname(__file__))
LOG_PATH = os.path.join(os.path.expanduser('~'), 'LightGBM_compilation.log')
LOG_NOTICE = "The full version of error log was saved into {0}".format(LOG_PATH)
Expand Down