SCons: Add 'split_libmodules' option to workaround linker issue

The new 'split_libmodules=yes' option is useful to work around linker
command line size limitations when linking a huge number of objects.
We're currently over 64k chars when linking libmodules.a on Windows
with MinGW, which triggers issues as seen in #30892.

Even on Linux, we can also reach linker command line size limitations
by adding more custom modules.

We force this option to True for MinGW on Windows, which fixes #30892.

Additional changes to lib splitting:

- Fix linking of the split module libs with interdependent symbols,
  hacking our way into LINKCOM and SHLINKCOM to set the `--start-group`
  and `--end-group` flags.
- Fix Python 3 compatibility in `methods.split_lib()`.
- Drop seemingly obsolete condition for 'msys' on 'posix'.
- Drop the unnecessary 'split_drivers' as the drivers lib is no longer
  too big since we moved all thirdparty builds to modules.

Co-authored-by: Hein-Pieter van Braam-Stewart <hp@tmm.cx>
This commit is contained in:
Rémi Verschelde
2019-12-09 19:22:08 +01:00
parent cd9d513285
commit c320a82213
5 changed files with 31 additions and 19 deletions

View File

@ -307,7 +307,7 @@ def split_lib(self, libname, src_list = None, env_lib = None):
else:
fname = env.File(f)[0].path
fname = fname.replace("\\", "/")
base = string.join(fname.split("/")[:2], "/")
base = "/".join(fname.split("/")[:2])
if base != cur_base and len(list) > max_src:
if num > 0:
lib = env_lib.add_library(libname + str(num), list)
@ -320,12 +320,6 @@ def split_lib(self, libname, src_list = None, env_lib = None):
lib = env_lib.add_library(libname + str(num), list)
lib_list.append(lib)
if len(lib_list) > 0:
if os.name == 'posix' and sys.platform == 'msys':
env.Replace(ARFLAGS=['rcsT'])
lib = env_lib.add_library(libname + "_collated", lib_list)
lib_list = [lib]
lib_base = []
env_lib.add_source_files(lib_base, "*.cpp")
lib = env_lib.add_library(libname, lib_base)
@ -333,6 +327,24 @@ def split_lib(self, libname, src_list = None, env_lib = None):
env.Prepend(LIBS=lib_list)
# When we split modules into arbitrary chunks, we end up with linking issues
# due to symbol dependencies split over several libs, which may not be linked
# in the required order. We use --start-group and --end-group to tell the
# linker that those archives should be searched repeatedly to resolve all
# undefined references.
# As SCons doesn't give us much control over how inserting libs in LIBS
# impacts the linker call, we need to hack our way into the linking commands
# LINKCOM and SHLINKCOM to set those flags.
if '-Wl,--start-group' in env['LINKCOM'] and '-Wl,--start-group' in env['SHLINKCOM']:
# Already added by a previous call, skip.
return
env['LINKCOM'] = str(env['LINKCOM']).replace('$_LIBFLAGS',
'-Wl,--start-group $_LIBFLAGS -Wl,--end-group')
env['SHLINKCOM'] = str(env['LINKCOM']).replace('$_LIBFLAGS',
'-Wl,--start-group $_LIBFLAGS -Wl,--end-group')
def save_active_platforms(apnames, ap):