1df14233eSab# 2df14233eSab# CDDL HEADER START 3df14233eSab# 4df14233eSab# The contents of this file are subject to the terms of the 5df14233eSab# Common Development and Distribution License (the "License"). 6df14233eSab# You may not use this file except in compliance with the License. 7df14233eSab# 8df14233eSab# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9df14233eSab# or http://www.opensolaris.org/os/licensing. 10df14233eSab# See the License for the specific language governing permissions 11df14233eSab# and limitations under the License. 12df14233eSab# 13df14233eSab# When distributing Covered Code, include this CDDL HEADER in each 14df14233eSab# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15df14233eSab# If applicable, add the following below this CDDL HEADER, with the 16df14233eSab# fields enclosed by brackets "[]" replaced with your own identifying 17df14233eSab# information: Portions Copyright [yyyy] [name of copyright owner] 18df14233eSab# 19df14233eSab# CDDL HEADER END 20df14233eSab# 21df14233eSab 22df14233eSab# 23df14233eSab# Copyright 2008 Sun Microsystems, Inc. All rights reserved. 24df14233eSab# Use is subject to license terms. 25df14233eSab# 26df14233eSab 27df14233eSab 28df14233eSabWhy 32-bit libelf is not Large File Aware 29df14233eSab----------------------------------------- 30df14233eSab 31df14233eSabThe ELF format uses unsigned 32-bit integers for offsets, so the 32df14233eSabtheoretical limit on a 32-bit ELF object is 4GB. However, libelf 33df14233eSabimposes a 2GB limit on the objects it can create. The Solaris 34df14233eSablink-editor and related tools are all based on libelf, so the 35df14233eSab32-bit version of the link-editor also has a 2GB limit, despite 36df14233eSabthe theoretical limit of 4GB. 37df14233eSab 38df14233eSabLarge file support (LFS) is a half step between the 32 and 64-bit 39df14233eSabworlds, in which an otherwise 32-bit limited process is allowed to 40df14233eSabread and write data to a file that can be larger than 2GB (the extent 41df14233eSabof a signed 32-bit integer, as represented by the system type off_t). 42df14233eSabLFS is useful if the program only needs to access a small subset of 43df14233eSabthe file data at any given time (e.g. /usr/bin/cat). It is less useful 44df14233eSabif the program needs to access a large amount of data at once --- having 45df14233eSabbeen freed from the file limit, the program will simply hit the virtual 46df14233eSabmemory limit (4GB). 47df14233eSab 48df14233eSabIn particular, the link-editor generally requires twice as much 49df14233eSabmemory as the size of the output object, half to hold the input 50df14233eSabobjects, and half to hold the result. This means that a 32-bit 51df14233eSablink-editor process will hit the 2GB file size limit and the 4GB 52df14233eSabaddress space limit at roughly the same time. As a result, a 53df14233eSablarge file aware 32-bit version of libelf has no significant value. 54df14233eSabDespite this, the question of what it would take to make libelf 55df14233eSablarge file aware comes up from time to time. 56df14233eSab 57df14233eSabThe first step would be to provide alternative versions of 58df14233eSaball public data structures that involve the off_t data type. 59df14233eSabThese structs, found in /usr/include/libelf.h, are: 60df14233eSab 61df14233eSab /* 62df14233eSab * Archive member header 63df14233eSab */ 64df14233eSab typedef struct { 65df14233eSab char *ar_name; 66df14233eSab time_t ar_date; 67df14233eSab uid_t ar_uid; 68df14233eSab gid_t ar_gid; 69df14233eSab mode_t ar_mode; 70df14233eSab off_t ar_size; 71df14233eSab char *ar_rawname; 72df14233eSab } Elf_Arhdr; 73df14233eSab 74df14233eSab 75df14233eSab /* 76df14233eSab * Data descriptor 77df14233eSab */ 78df14233eSab typedef struct { 79df14233eSab Elf_Void *d_buf; 80df14233eSab Elf_Type d_type; 81df14233eSab size_t d_size; 82df14233eSab off_t d_off; /* offset into section */ 83df14233eSab size_t d_align; /* alignment in section */ 84df14233eSab unsigned d_version; /* elf version */ 85df14233eSab } Elf_Data; 86df14233eSab 87df14233eSabAs off_t is a signed type, these alternative versions would have to use 88df14233eSaban off64_t type instead. 89df14233eSab 90df14233eSabIn addition to providing alternative large file aware Elf_Arhdr and 91df14233eSabElf_Data types, it would be necessary to implement large file aware 92df14233eSabversions of the public functions that use them, also found in 93df14233eSab/usr/include/libelf.h: 94df14233eSab 95df14233eSab /* 96df14233eSab * Function declarations 97df14233eSab */ 98df14233eSab unsigned elf_flagdata(Elf_Data *, Elf_Cmd, unsigned); 99df14233eSab Elf_Arhdr *elf_getarhdr(Elf *); 100df14233eSab off_t elf_getbase(Elf *); 101df14233eSab Elf_Data *elf_getdata(Elf_Scn *, Elf_Data *); 102df14233eSab Elf_Data *elf_newdata(Elf_Scn *); 103df14233eSab Elf_Data *elf_rawdata(Elf_Scn *, Elf_Data *); 104df14233eSab off_t elf_update(Elf *, Elf_Cmd); 105df14233eSab Elf_Data *elf32_xlatetof(Elf_Data *, const Elf_Data *, unsigned); 106df14233eSab Elf_Data *elf32_xlatetom(Elf_Data *, const Elf_Data *, unsigned); 107df14233eSab Elf_Data *elf64_xlatetof(Elf_Data *, const Elf_Data *, unsigned); 108df14233eSab Elf_Data *elf64_xlatetom(Elf_Data *, const Elf_Data *, unsigned); 109df14233eSab 110df14233eSabIt is important to note that these new versions cannot replace the 111df14233eSaboriginal definitions. Those must continue to be available to support 112*bebb829dSRod Evansnon-large-file-aware programs. These new types and functions would be in 113df14233eSabaddition to the pre-existing versions. 114df14233eSab 115df14233eSabWhen you make code like this large file aware, it is necessary to undertake 116df14233eSaba careful analysis of the code to ensure that all the surrounding code uses 117df14233eSabvariable types large enough to handle the increased range. Hence, this work 118df14233eSabis more complicated than simply supplying variants that use a bigger 119df14233eSaboff_t and rebuilding --- that is just the first step. 120df14233eSab 121df14233eSabThere are two standard preprocessor definitions used to control 122df14233eSablarge file support: 123df14233eSab 124df14233eSab _LARGEFILE64_SOURCE 125df14233eSab _FILE_OFFSET_BITS 126df14233eSab 127df14233eSabThese preprocessor definitions would be used to determine whether 128df14233eSaba given program linked against libelf would see the regular, or 129*bebb829dSRod Evansthe large file aware versions of the above types and routines. 130df14233eSabThis is the same approach used in other large file capable software, 131df14233eSabsuch as libc. 132df14233eSab 133df14233eSabFinally, all the applications that rely on libelf would need to be made 134df14233eSablarge file aware. As with libelf itself, there is more to such an effort 135df14233eSabthan recompiling with preprocessor macros set. The code in these 136df14233eSabapplications would need to be examined carefully. Some of these programs 137df14233eSabare very old, and were not originally written with such type portability 138df14233eSabin mind. Such code can be difficult to transition. 139df14233eSab 140df14233eSabTo work around the 2GB limit in 32-bit libelf: 141df14233eSab 142df14233eSab - The fundamental limits of a 32-bit address space mean 143df14233eSab that a program this large should be 64-bit. Only a 64-bit 144df14233eSab address space has enough room for that much code, plus the 145df14233eSab stack and heap needed to do useful work with it. 146df14233eSab 147df14233eSab - The 64-bit version of libelf is also able to process 148df14233eSab 32-bit objects, and does not have a 2GB file size limit. 149df14233eSab Therefore, the 64-bit link-editor can be used to build a 32-bit 150df14233eSab executable which is >2GB. The resulting program will consume over 151df14233eSab half the available address space just to start running. However, 152df14233eSab there may be enough address space left for it to do useful work. 153df14233eSab 154df14233eSab Note that the 32-bit limit for sharable objects remains at 155df14233eSab 2GB --- imposed by the runtime linker, which is also not large 156df14233eSab file aware. 157