Staticlib rename internal symbols#156950
Conversation
|
btw if this underwent a lot of changes and needs re-testing, I'm up for it (I compile Rust to 6 targets). |
This comment has been minimized.
This comment has been minimized.
Functionally, there shouldn't be any changes; it's mostly more "Rusty" modifications. After #155338 is landed, I will rebase the current PR, and you can test it after that. |
… r=petrochenkov Staticlib hide internal symbols According to issue rust-lang#104707, when building a staticlib, all Rust internal symbols — mangled symbols, `#[rustc_std_internal_symbol]` items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only `#[no_mangle]` symbols via a linker version script. `-Zstaticlib-hide-internal-symbols` directly post-processes ELF object files in the archive: parsing the `SHT_SYMTAB` sections and setting `STV_HIDDEN` visibility on any `GLOBAL/WEAK` defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead. Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect. **Update**: The rename counterpart (`-Zstaticlib-rename-internal-symbols`) is in rust-lang#156950. The test code are as follows: 1.a std rust staticlib: ```rust use std::collections::HashMap; use std::panic::{catch_unwind, AssertUnwindSafe}; #[no_mangle] pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b } #[no_mangle] pub extern "C" fn my_hash_lookup(key: u64) -> u64 { let mut map = HashMap::new(); for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); } *map.get(&key).unwrap_or(&0) } pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() } #[no_mangle] pub extern "C" fn my_format_number(n: i32) -> i32 { let s = format!("number: {}", n); s.len() as i32 } #[no_mangle] pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 { match catch_unwind(AssertUnwindSafe(|| { if b == 0 { panic!("division by zero!"); } a / b })) { Ok(result) => result, Err(_) => -1, } } #[no_mangle] pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); } ``` 1.b downstream c program: ```c extern int my_add(int a, int b); extern unsigned long my_hash_lookup(unsigned long key); extern int my_format_number(int n); extern int my_safe_div(int a, int b); extern void my_uncaught_panic(void); int main() { int failures = 0; if (my_add(10, 20) != 30) failures++; if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++; if (my_format_number(42) != 10) failures++; if (my_safe_div(100, 5) != 20) failures++; if (my_safe_div(100, 0) != -1) failures++; pid_t pid = fork(); if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 1.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin 616K 584K 33K (5%) 246 5 241 lto_fat 525K 525K 0 (0%) 6 5 1 opt_s 1.7M 1.5M 204K (12%) 1735 5 1730 opt_z 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin_z 602K 570K 32K (5%) 246 5 241 lto_fat_z 514K 514K 0 (0%) 6 5 1 full 514K 514K 0 (0%) 6 5 1 ``` 1.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin 616K 599K 18K (2%) 246 5 241 lto_fat 525K 535K -1% (-1%) 6 5 1 opt_s 1.7M 1.5M 162K (9%) 1735 5 1730 opt_z 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin_z 602K 585K 18K (2%) 246 5 241 lto_fat_z 514K 524K -1% (-1%) 6 5 1 full 514K 523K -1% (-1%) 6 5 1 ``` 2.a no_std rust staticlib ```rust #![no_std] #![feature(core_intrinsics)] use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } #[no_mangle] pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) } #[no_mangle] pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 { if data.is_null() { return 0; } let slice = unsafe { core::slice::from_raw_parts(data, len) }; let mut sum: u8 = 0; for &byte in slice { sum = sum.wrapping_add(byte); } sum } fn internal_helper() -> i32 { 42 } #[no_mangle] pub extern "C" fn call_internal() -> i32 { internal_helper() } #[no_mangle] pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); } ``` 2.b downstream c program ```c extern int embedded_add(int a, int b); extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len); extern int call_internal(void); extern void embedded_trigger_abort(void); int main() { int failures = 0; if (embedded_add(10, 20) != 30) failures++; unsigned char data[] = {1, 2, 3}; if (embedded_checksum(data, 3) != 6) failures++; if (call_internal() != 42) failures++; pid_t pid = fork(); if (pid == 0) { embedded_trigger_abort(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 2.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 429K 56K (11%) 490 4 486 lto_thin 180K 180K 0 (0%) 4 4 0 lto_fat 179K 179K 0 (0%) 4 4 0 opt_s 485K 429K 56K (11%) 490 4 486 opt_z 485K 429K 56K (11%) 490 4 486 lto_thin_z 180K 180K 0 (0%) 4 4 0 lto_fat_z 179K 179K 0 (0%) 4 4 0 full 179K 179K 0 (0%) 4 4 0 ``` 2.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 447K 39K (7%) 490 4 486 lto_thin 180K 189K -5% (-5%) 4 4 0 lto_fat 179K 189K -5% (-5%) 4 4 0 opt_s 485K 448K 38K (7%) 490 4 486 opt_z 485K 448K 38K (7%) 490 4 486 lto_thin_z 180K 189K -5% (-5%) 4 4 0 lto_fat_z 179K 189K -5% (-5%) 4 4 0 full 179K 189K -5% (-5%) 4 4 0 ``` Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled. r? @bjorn3 @petrochenkov
… r=petrochenkov Staticlib hide internal symbols According to issue rust-lang#104707, when building a staticlib, all Rust internal symbols — mangled symbols, `#[rustc_std_internal_symbol]` items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only `#[no_mangle]` symbols via a linker version script. `-Zstaticlib-hide-internal-symbols` directly post-processes ELF object files in the archive: parsing the `SHT_SYMTAB` sections and setting `STV_HIDDEN` visibility on any `GLOBAL/WEAK` defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead. Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect. **Update**: The rename counterpart (`-Zstaticlib-rename-internal-symbols`) is in rust-lang#156950. The test code are as follows: 1.a std rust staticlib: ```rust use std::collections::HashMap; use std::panic::{catch_unwind, AssertUnwindSafe}; #[no_mangle] pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b } #[no_mangle] pub extern "C" fn my_hash_lookup(key: u64) -> u64 { let mut map = HashMap::new(); for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); } *map.get(&key).unwrap_or(&0) } pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() } #[no_mangle] pub extern "C" fn my_format_number(n: i32) -> i32 { let s = format!("number: {}", n); s.len() as i32 } #[no_mangle] pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 { match catch_unwind(AssertUnwindSafe(|| { if b == 0 { panic!("division by zero!"); } a / b })) { Ok(result) => result, Err(_) => -1, } } #[no_mangle] pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); } ``` 1.b downstream c program: ```c extern int my_add(int a, int b); extern unsigned long my_hash_lookup(unsigned long key); extern int my_format_number(int n); extern int my_safe_div(int a, int b); extern void my_uncaught_panic(void); int main() { int failures = 0; if (my_add(10, 20) != 30) failures++; if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++; if (my_format_number(42) != 10) failures++; if (my_safe_div(100, 5) != 20) failures++; if (my_safe_div(100, 0) != -1) failures++; pid_t pid = fork(); if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 1.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin 616K 584K 33K (5%) 246 5 241 lto_fat 525K 525K 0 (0%) 6 5 1 opt_s 1.7M 1.5M 204K (12%) 1735 5 1730 opt_z 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin_z 602K 570K 32K (5%) 246 5 241 lto_fat_z 514K 514K 0 (0%) 6 5 1 full 514K 514K 0 (0%) 6 5 1 ``` 1.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin 616K 599K 18K (2%) 246 5 241 lto_fat 525K 535K -1% (-1%) 6 5 1 opt_s 1.7M 1.5M 162K (9%) 1735 5 1730 opt_z 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin_z 602K 585K 18K (2%) 246 5 241 lto_fat_z 514K 524K -1% (-1%) 6 5 1 full 514K 523K -1% (-1%) 6 5 1 ``` 2.a no_std rust staticlib ```rust #![no_std] #![feature(core_intrinsics)] use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } #[no_mangle] pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) } #[no_mangle] pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 { if data.is_null() { return 0; } let slice = unsafe { core::slice::from_raw_parts(data, len) }; let mut sum: u8 = 0; for &byte in slice { sum = sum.wrapping_add(byte); } sum } fn internal_helper() -> i32 { 42 } #[no_mangle] pub extern "C" fn call_internal() -> i32 { internal_helper() } #[no_mangle] pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); } ``` 2.b downstream c program ```c extern int embedded_add(int a, int b); extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len); extern int call_internal(void); extern void embedded_trigger_abort(void); int main() { int failures = 0; if (embedded_add(10, 20) != 30) failures++; unsigned char data[] = {1, 2, 3}; if (embedded_checksum(data, 3) != 6) failures++; if (call_internal() != 42) failures++; pid_t pid = fork(); if (pid == 0) { embedded_trigger_abort(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 2.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 429K 56K (11%) 490 4 486 lto_thin 180K 180K 0 (0%) 4 4 0 lto_fat 179K 179K 0 (0%) 4 4 0 opt_s 485K 429K 56K (11%) 490 4 486 opt_z 485K 429K 56K (11%) 490 4 486 lto_thin_z 180K 180K 0 (0%) 4 4 0 lto_fat_z 179K 179K 0 (0%) 4 4 0 full 179K 179K 0 (0%) 4 4 0 ``` 2.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 447K 39K (7%) 490 4 486 lto_thin 180K 189K -5% (-5%) 4 4 0 lto_fat 179K 189K -5% (-5%) 4 4 0 opt_s 485K 448K 38K (7%) 490 4 486 opt_z 485K 448K 38K (7%) 490 4 486 lto_thin_z 180K 189K -5% (-5%) 4 4 0 lto_fat_z 179K 189K -5% (-5%) 4 4 0 full 179K 189K -5% (-5%) 4 4 0 ``` Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled. r? @bjorn3 @petrochenkov
… r=petrochenkov Staticlib hide internal symbols According to issue rust-lang#104707, when building a staticlib, all Rust internal symbols — mangled symbols, `#[rustc_std_internal_symbol]` items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only `#[no_mangle]` symbols via a linker version script. `-Zstaticlib-hide-internal-symbols` directly post-processes ELF object files in the archive: parsing the `SHT_SYMTAB` sections and setting `STV_HIDDEN` visibility on any `GLOBAL/WEAK` defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead. Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect. **Update**: The rename counterpart (`-Zstaticlib-rename-internal-symbols`) is in rust-lang#156950. The test code are as follows: 1.a std rust staticlib: ```rust use std::collections::HashMap; use std::panic::{catch_unwind, AssertUnwindSafe}; #[no_mangle] pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b } #[no_mangle] pub extern "C" fn my_hash_lookup(key: u64) -> u64 { let mut map = HashMap::new(); for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); } *map.get(&key).unwrap_or(&0) } pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() } #[no_mangle] pub extern "C" fn my_format_number(n: i32) -> i32 { let s = format!("number: {}", n); s.len() as i32 } #[no_mangle] pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 { match catch_unwind(AssertUnwindSafe(|| { if b == 0 { panic!("division by zero!"); } a / b })) { Ok(result) => result, Err(_) => -1, } } #[no_mangle] pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); } ``` 1.b downstream c program: ```c extern int my_add(int a, int b); extern unsigned long my_hash_lookup(unsigned long key); extern int my_format_number(int n); extern int my_safe_div(int a, int b); extern void my_uncaught_panic(void); int main() { int failures = 0; if (my_add(10, 20) != 30) failures++; if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++; if (my_format_number(42) != 10) failures++; if (my_safe_div(100, 5) != 20) failures++; if (my_safe_div(100, 0) != -1) failures++; pid_t pid = fork(); if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 1.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin 616K 584K 33K (5%) 246 5 241 lto_fat 525K 525K 0 (0%) 6 5 1 opt_s 1.7M 1.5M 204K (12%) 1735 5 1730 opt_z 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin_z 602K 570K 32K (5%) 246 5 241 lto_fat_z 514K 514K 0 (0%) 6 5 1 full 514K 514K 0 (0%) 6 5 1 ``` 1.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin 616K 599K 18K (2%) 246 5 241 lto_fat 525K 535K -1% (-1%) 6 5 1 opt_s 1.7M 1.5M 162K (9%) 1735 5 1730 opt_z 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin_z 602K 585K 18K (2%) 246 5 241 lto_fat_z 514K 524K -1% (-1%) 6 5 1 full 514K 523K -1% (-1%) 6 5 1 ``` 2.a no_std rust staticlib ```rust #![no_std] #![feature(core_intrinsics)] use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } #[no_mangle] pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) } #[no_mangle] pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 { if data.is_null() { return 0; } let slice = unsafe { core::slice::from_raw_parts(data, len) }; let mut sum: u8 = 0; for &byte in slice { sum = sum.wrapping_add(byte); } sum } fn internal_helper() -> i32 { 42 } #[no_mangle] pub extern "C" fn call_internal() -> i32 { internal_helper() } #[no_mangle] pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); } ``` 2.b downstream c program ```c extern int embedded_add(int a, int b); extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len); extern int call_internal(void); extern void embedded_trigger_abort(void); int main() { int failures = 0; if (embedded_add(10, 20) != 30) failures++; unsigned char data[] = {1, 2, 3}; if (embedded_checksum(data, 3) != 6) failures++; if (call_internal() != 42) failures++; pid_t pid = fork(); if (pid == 0) { embedded_trigger_abort(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 2.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 429K 56K (11%) 490 4 486 lto_thin 180K 180K 0 (0%) 4 4 0 lto_fat 179K 179K 0 (0%) 4 4 0 opt_s 485K 429K 56K (11%) 490 4 486 opt_z 485K 429K 56K (11%) 490 4 486 lto_thin_z 180K 180K 0 (0%) 4 4 0 lto_fat_z 179K 179K 0 (0%) 4 4 0 full 179K 179K 0 (0%) 4 4 0 ``` 2.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 447K 39K (7%) 490 4 486 lto_thin 180K 189K -5% (-5%) 4 4 0 lto_fat 179K 189K -5% (-5%) 4 4 0 opt_s 485K 448K 38K (7%) 490 4 486 opt_z 485K 448K 38K (7%) 490 4 486 lto_thin_z 180K 189K -5% (-5%) 4 4 0 lto_fat_z 179K 189K -5% (-5%) 4 4 0 full 179K 189K -5% (-5%) 4 4 0 ``` Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled. r? @bjorn3 @petrochenkov
… r=petrochenkov Staticlib hide internal symbols According to issue rust-lang#104707, when building a staticlib, all Rust internal symbols — mangled symbols, `#[rustc_std_internal_symbol]` items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only `#[no_mangle]` symbols via a linker version script. `-Zstaticlib-hide-internal-symbols` directly post-processes ELF object files in the archive: parsing the `SHT_SYMTAB` sections and setting `STV_HIDDEN` visibility on any `GLOBAL/WEAK` defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead. Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect. **Update**: The rename counterpart (`-Zstaticlib-rename-internal-symbols`) is in rust-lang#156950. The test code are as follows: 1.a std rust staticlib: ```rust use std::collections::HashMap; use std::panic::{catch_unwind, AssertUnwindSafe}; #[no_mangle] pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b } #[no_mangle] pub extern "C" fn my_hash_lookup(key: u64) -> u64 { let mut map = HashMap::new(); for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); } *map.get(&key).unwrap_or(&0) } pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() } #[no_mangle] pub extern "C" fn my_format_number(n: i32) -> i32 { let s = format!("number: {}", n); s.len() as i32 } #[no_mangle] pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 { match catch_unwind(AssertUnwindSafe(|| { if b == 0 { panic!("division by zero!"); } a / b })) { Ok(result) => result, Err(_) => -1, } } #[no_mangle] pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); } ``` 1.b downstream c program: ```c extern int my_add(int a, int b); extern unsigned long my_hash_lookup(unsigned long key); extern int my_format_number(int n); extern int my_safe_div(int a, int b); extern void my_uncaught_panic(void); int main() { int failures = 0; if (my_add(10, 20) != 30) failures++; if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++; if (my_format_number(42) != 10) failures++; if (my_safe_div(100, 5) != 20) failures++; if (my_safe_div(100, 0) != -1) failures++; pid_t pid = fork(); if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 1.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin 616K 584K 33K (5%) 246 5 241 lto_fat 525K 525K 0 (0%) 6 5 1 opt_s 1.7M 1.5M 204K (12%) 1735 5 1730 opt_z 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin_z 602K 570K 32K (5%) 246 5 241 lto_fat_z 514K 514K 0 (0%) 6 5 1 full 514K 514K 0 (0%) 6 5 1 ``` 1.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin 616K 599K 18K (2%) 246 5 241 lto_fat 525K 535K -1% (-1%) 6 5 1 opt_s 1.7M 1.5M 162K (9%) 1735 5 1730 opt_z 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin_z 602K 585K 18K (2%) 246 5 241 lto_fat_z 514K 524K -1% (-1%) 6 5 1 full 514K 523K -1% (-1%) 6 5 1 ``` 2.a no_std rust staticlib ```rust #![no_std] #![feature(core_intrinsics)] use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } #[no_mangle] pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) } #[no_mangle] pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 { if data.is_null() { return 0; } let slice = unsafe { core::slice::from_raw_parts(data, len) }; let mut sum: u8 = 0; for &byte in slice { sum = sum.wrapping_add(byte); } sum } fn internal_helper() -> i32 { 42 } #[no_mangle] pub extern "C" fn call_internal() -> i32 { internal_helper() } #[no_mangle] pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); } ``` 2.b downstream c program ```c extern int embedded_add(int a, int b); extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len); extern int call_internal(void); extern void embedded_trigger_abort(void); int main() { int failures = 0; if (embedded_add(10, 20) != 30) failures++; unsigned char data[] = {1, 2, 3}; if (embedded_checksum(data, 3) != 6) failures++; if (call_internal() != 42) failures++; pid_t pid = fork(); if (pid == 0) { embedded_trigger_abort(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 2.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 429K 56K (11%) 490 4 486 lto_thin 180K 180K 0 (0%) 4 4 0 lto_fat 179K 179K 0 (0%) 4 4 0 opt_s 485K 429K 56K (11%) 490 4 486 opt_z 485K 429K 56K (11%) 490 4 486 lto_thin_z 180K 180K 0 (0%) 4 4 0 lto_fat_z 179K 179K 0 (0%) 4 4 0 full 179K 179K 0 (0%) 4 4 0 ``` 2.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 447K 39K (7%) 490 4 486 lto_thin 180K 189K -5% (-5%) 4 4 0 lto_fat 179K 189K -5% (-5%) 4 4 0 opt_s 485K 448K 38K (7%) 490 4 486 opt_z 485K 448K 38K (7%) 490 4 486 lto_thin_z 180K 189K -5% (-5%) 4 4 0 lto_fat_z 179K 189K -5% (-5%) 4 4 0 full 179K 189K -5% (-5%) 4 4 0 ``` Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled. r? @bjorn3 @petrochenkov
Rollup merge of #155338 - cezarbbb:staticlib-symbol-hygiene, r=petrochenkov Staticlib hide internal symbols According to issue #104707, when building a staticlib, all Rust internal symbols — mangled symbols, `#[rustc_std_internal_symbol]` items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only `#[no_mangle]` symbols via a linker version script. `-Zstaticlib-hide-internal-symbols` directly post-processes ELF object files in the archive: parsing the `SHT_SYMTAB` sections and setting `STV_HIDDEN` visibility on any `GLOBAL/WEAK` defined symbol that is not in the exported symbol set, without changing the binding. This is an in-place modification (only writing the st_other byte per matching entry), with zero overhead. Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect. **Update**: The rename counterpart (`-Zstaticlib-rename-internal-symbols`) is in #156950. The test code are as follows: 1.a std rust staticlib: ```rust use std::collections::HashMap; use std::panic::{catch_unwind, AssertUnwindSafe}; #[no_mangle] pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b } #[no_mangle] pub extern "C" fn my_hash_lookup(key: u64) -> u64 { let mut map = HashMap::new(); for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); } *map.get(&key).unwrap_or(&0) } pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() } #[no_mangle] pub extern "C" fn my_format_number(n: i32) -> i32 { let s = format!("number: {}", n); s.len() as i32 } #[no_mangle] pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 { match catch_unwind(AssertUnwindSafe(|| { if b == 0 { panic!("division by zero!"); } a / b })) { Ok(result) => result, Err(_) => -1, } } #[no_mangle] pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); } ``` 1.b downstream c program: ```c extern int my_add(int a, int b); extern unsigned long my_hash_lookup(unsigned long key); extern int my_format_number(int n); extern int my_safe_div(int a, int b); extern void my_uncaught_panic(void); int main() { int failures = 0; if (my_add(10, 20) != 30) failures++; if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++; if (my_format_number(42) != 10) failures++; if (my_safe_div(100, 5) != 20) failures++; if (my_safe_div(100, 0) != -1) failures++; pid_t pid = fork(); if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 1.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin 616K 584K 33K (5%) 246 5 241 lto_fat 525K 525K 0 (0%) 6 5 1 opt_s 1.7M 1.5M 204K (12%) 1735 5 1730 opt_z 1.7M 1.5M 204K (12%) 1735 5 1730 lto_thin_z 602K 570K 32K (5%) 246 5 241 lto_fat_z 514K 514K 0 (0%) 6 5 1 full 514K 514K 0 (0%) 6 5 1 ``` 1.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin 616K 599K 18K (2%) 246 5 241 lto_fat 525K 535K -1% (-1%) 6 5 1 opt_s 1.7M 1.5M 162K (9%) 1735 5 1730 opt_z 1.7M 1.5M 162K (9%) 1735 5 1730 lto_thin_z 602K 585K 18K (2%) 246 5 241 lto_fat_z 514K 524K -1% (-1%) 6 5 1 full 514K 523K -1% (-1%) 6 5 1 ``` 2.a no_std rust staticlib ```rust #![no_std] #![feature(core_intrinsics)] use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} } #[no_mangle] pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) } #[no_mangle] pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 { if data.is_null() { return 0; } let slice = unsafe { core::slice::from_raw_parts(data, len) }; let mut sum: u8 = 0; for &byte in slice { sum = sum.wrapping_add(byte); } sum } fn internal_helper() -> i32 { 42 } #[no_mangle] pub extern "C" fn call_internal() -> i32 { internal_helper() } #[no_mangle] pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); } ``` 2.b downstream c program ```c extern int embedded_add(int a, int b); extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len); extern int call_internal(void); extern void embedded_trigger_abort(void); int main() { int failures = 0; if (embedded_add(10, 20) != 30) failures++; unsigned char data[] = {1, 2, 3}; if (embedded_checksum(data, 3) != 6) failures++; if (call_internal() != 42) failures++; pid_t pid = fork(); if (pid == 0) { embedded_trigger_abort(); _exit(0); } else { waitpid(pid, &status, 0); } return failures; } ``` The test results with different compiler flags(which might cause binary size reduction) are as follows: 2.c result with `-Zstaticlib-hide-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 429K 56K (11%) 490 4 486 lto_thin 180K 180K 0 (0%) 4 4 0 lto_fat 179K 179K 0 (0%) 4 4 0 opt_s 485K 429K 56K (11%) 490 4 486 opt_z 485K 429K 56K (11%) 490 4 486 lto_thin_z 180K 180K 0 (0%) 4 4 0 lto_fat_z 179K 179K 0 (0%) 4 4 0 full 179K 179K 0 (0%) 4 4 0 ``` 2.d result with `-Zstaticlib-hide-internal-symbols + -Zstaticlib-rename-internal-symbols` ``` settings OFF ON -Zsave ALL OFF.dynsym ON.dynsym ------------------------------------------------------------------------ default 485K 447K 39K (7%) 490 4 486 lto_thin 180K 189K -5% (-5%) 4 4 0 lto_fat 179K 189K -5% (-5%) 4 4 0 opt_s 485K 448K 38K (7%) 490 4 486 opt_z 485K 448K 38K (7%) 490 4 486 lto_thin_z 180K 189K -5% (-5%) 4 4 0 lto_fat_z 179K 189K -5% (-5%) 4 4 0 full 179K 189K -5% (-5%) 4 4 0 ``` Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled. r? @bjorn3 @petrochenkov
2445a49 to
55c16ed
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
|
Can we now remove the |
|
@rustbot ready |
| Endianness::Big => value.to_be_bytes(), | ||
| }; | ||
| buf[offset..offset + 8].copy_from_slice(&bytes); | ||
| } |
There was a problem hiding this comment.
This file is getting rather large with all this binary edit code. Maybe the binary edit code can be split into a separate file?
There was a problem hiding this comment.
That makes sense. The logic for the hide and rename flags has now been moved to symbol_edit.rs. The specific name of the new file can be discussed further.
There was a problem hiding this comment.
I'd personally prefer changes implementing the new feature and changes moving the code around to be in different PRs (the order is not important).
There was a problem hiding this comment.
The operation to move the hide flag code has been submitted in a new PR (PR #157691). I think the current PR can be blocked until that PR is merged.
55c16ed to
a86abb9
Compare
a86abb9 to
0762f85
Compare
| Ok(result) => result, | ||
| Err(_) => -1, | ||
| } | ||
| } |
There was a problem hiding this comment.
Looks like the lib.rs and main.c files are identical to those in tests/run-make/staticlib-hide-internal-symbols.
So the new rename-internal-symbols tests can reuse them from there.
| staticlib_hide_internal_symbols: bool = (false, parse_bool, [TRACKED], | ||
| "hide non-exported symbols in ELF static libraries by setting STV_HIDDEN"), | ||
| staticlib_rename_internal_symbols: bool = (false, parse_bool, [TRACKED], | ||
| "rename Rust internal symbols when building staticlibs to avoid conflicts"), |
There was a problem hiding this comment.
| "rename Rust internal symbols when building staticlibs to avoid conflicts"), | |
| "[hide|rename] non-exported Rust symbols when building staticlibs [by setting STV_HIDDEN|to avoid conflicts]"), |
If staticlib_hide_internal_symbols and staticlib_rename_internal_symbols affect the same set of symbols, then the descriptions should be the same for both options.
| fn build( | ||
| self: Box<Self>, | ||
| output: &Path, | ||
| exported_symbols: Option<FxHashSet<String>>, |
There was a problem hiding this comment.
| exported_symbols: Option<FxHashSet<String>>, | |
| symbols: Option<ArchiveSymbols>, |
Some combinations of the passed 3 parameters are impossible, could you move them into a structure like
struct ArchiveSymbols {
exported: FxHashSet<String>,
rename_suffix: Option<String>,
hide: bool
}
| natvis_debugger_visualizers: Default::default(), | ||
| lint_level_specs: CodegenLintLevelSpecs::from_tcx(tcx), | ||
| metadata_symbol: exported_symbols::metadata_symbol_name(tcx), | ||
| rename_suffix: format!("_rs{:x}", tcx.stable_crate_id(LOCAL_CRATE)), |
There was a problem hiding this comment.
| rename_suffix: format!("_rs{:x}", tcx.stable_crate_id(LOCAL_CRATE)), | |
| symbol_rename_suffix: format!(".rs{:x}", tcx.stable_crate_id(LOCAL_CRATE)), |
mangled_rust_symbol_rs12345 is not a valid mangled rust name, according to the spec.
I think we can use a vendor specific suffix to make it valid.
(Also change the field name to be more specific.)
There was a problem hiding this comment.
It uses format!(".rs{:x}", ...) now.
| }; | ||
|
|
||
| // Rename requires a two-pass approach: first collect all internal symbols | ||
| // across all .o files to ensure consistent renaming, then apply. |
There was a problem hiding this comment.
I still don't understand why the first pass is necessary.
The elf_apply_rename_impl logic can equally well use the condition !exported.contains(name) instead of rename_set.contains(name).
There was a problem hiding this comment.
The collect pass is needed because rename applies to undefined symbols too (cross-object references). Without it we can't tell apart internal undefs from external ones like malloc(rename tests failed).
But one thing I agree is that hide-only doesn't need this pass, only hide/hide + rename will entry this pass. So hide-only skips this pass entirely at present.
| Some(patches) | ||
| } | ||
|
|
||
| fn hide_patches(data: &[u8], exported: &FxHashSet<String>) -> Option<Vec<Patch>> { |
There was a problem hiding this comment.
This function can be inlined into fn apply_hide.
| } | ||
| } | ||
|
|
||
| pub(super) fn apply_hide(data: &[u8], exported: &FxHashSet<String>) -> Vec<u8> { |
There was a problem hiding this comment.
Wait, why does this return a Vec<u8>?
It was a Cow<[u8]> the last time I reviewed.
We don't need to clone the data if there are no patches.
| } | ||
| } | ||
|
|
||
| pub(super) fn rename_impl( |
There was a problem hiding this comment.
| pub(super) fn rename_impl( | |
| pub(super) fn apply_rename( |
This is an equivalent of apply_hide from what I see.
Let it use a similar naming, and return a Cow as well.
|
|
||
| let data_ptr = data.as_ptr() as usize; | ||
|
|
||
| let mut renames: Vec<RenameEntry> = Vec::new(); |
There was a problem hiding this comment.
This is an equivalent of patches from apply_hide.
We could actually collect patches and renames in the same single pass.
There was a problem hiding this comment.
Done. elf_edit_impl / macho_edit_impl now collect both hide patches and rename entries in a single pass.
|
|
||
| if renames.is_empty() { | ||
| return None; | ||
| } |
There was a problem hiding this comment.
And everything below is an analogue of apply_patches, except significantly more complex.
Perhaps makes sense to move it to a separate function as well.
I'd also follow the same general high level approach here - first parsing and patch collection, then patch application without parsing, although I'm not sure how practical it will be in this case.
There was a problem hiding this comment.
The current modifications can be referenced in the answer above.
|
I didn't look at the strtab building code in detail yet, probably in the next review round. |
|
Reminder, once the PR becomes ready for a review, use |
0762f85 to
1d6f596
Compare
…t, r=petrochenkov Move symbol hiding code to a new file Move the symbol visibility editing code (`apply_hide` and helpers) from `archive.rs` into a new `symbol_edit.rs` module. This is a pure code move with no functional changes — extracted to reduce the size of `archive.rs` and to provide a natural home for future symbol editing logic (e.g. renaming, rust-lang#156950). Requested in rust-lang#156950 (comment). r? @petrochenkov
|
@rustbot ready |
View all comments
Follow-up to #155338.
-Zstaticlib-rename-internal-symbolsappends a crate-specific suffix (_rs{StableCrateId}) to non-exported symbols, resolving duplicate symbol conflicts when linking multiple Rust staticlibs into the same binary.The implementation collects all defined
GLOBAL/WEAKsymbol names not in the exported set across all .o files, then renames them by extending the strtab and patching symbol name offsets. When combined with-Zstaticlib-hide-internal-symbols, the renamed symbols also receiveSTV_HIDDENvisibility.Supported on ELF targets (Linux, BSD, etc.) and Apple targets (macOS, iOS, etc.). On unsupported targets (Windows), a warning is emitted and the flag has no effect.
r?@bjorn3 @petrochenkov