Too much dirty data in the library

[behdad]

Building HarfBuzz without harfrust/fontations, we have only 6kb data section in the library according to size. With harfrust/fontations we have over 80kb. I have not been able to track it down using objdump so far.

[behdad]

Humm. objdump -h build/src/libharfbuzz.so shows a .data.rel.ro section of size 76kb. This is the culprit I believe. But there's no symbol assigned to much of that:

$ objdump -t -C build/src/libharfbuzz.so | grep .data.rel.ro
00000000002dc7b8 l     O .data.rel.ro	0000000000001020              read_fonts::tables::post::DEFAULT_GLYPH_NAMES
00000000002cc5e0 l     O .data.rel.ro	0000000000000040              supported_face_loaders
00000000002cc620 l     O .data.rel.ro	0000000000000048              supported_font_funcs
00000000002cc560 l     O .data.rel.ro	0000000000000060              _hb_all_shapers
00000000002cc3e0 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_khmer
00000000002cc060 l     O .data.rel.ro	0000000000000000              __dso_handle
00000000002cc320 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_hebrew
00000000002cc380 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_indic
00000000002cc2c0 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_hangul
00000000002cc440 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_myanmar
00000000002cc4a0 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_thai
00000000002cc1a0 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_arabic
00000000002cc500 l     O .data.rel.ro	0000000000000060              _hb_ot_shaper_use
00000000002de4b0 g     O .data.rel.ro	0000000000000008              std::panicking::EMPTY_PANIC

[behdad]

00000000002dc7b8 l O .data.rel.ro 0000000000001020 read_fonts::tables::post::DEFAULT_GLYPH_NAMES

googlefonts/fontations#1671

[behdad]

Using bloaty (https://github.com/ahaoboy/bloaty-metafile), I see that ot_shape_plan::new is contributing to most of this section, but I don't understand why.

[behdad]

@nicoburns Do you have any insights here?

[behdad]

I see that ot_shape_plan::new is contributing to most of this section, but I don't understand why.

Chances are those don't have anything to do with ot_shape_plan::new and are just attributed to it because that's the closest symbol. Not sure.

[nicoburns]

https://github.com/RazrFalcon/cargo-bloat is also worth trying. And I've also seen people recommend https://github.com/dtolnay/cargo-llvm-lines (but I haven't used it myself). In general, I've been pretty unimpressed with the tooling for analyzing binary bloat. None of the tools even seem to account for the entire binary. And I also haven't found any that do a good job of attributing static data (as opposed to code).

Creating a build profile with full debug info and optimizations enabled (so as to reasonably approximate the final binary), but with LTO disabled (because the tools don't work with it enabled) can also help give the analysis tools more to work with:

[profile.bloat]
inherits = "release"
debug = "full"
strip = false
lto = false

That points to a few large functions. But doesn't help to explain why these functions are large:

 1.4%   2.8%  20.0KiB   harfrust harfrust::hb::ot_shape_plan::hb_ot_shape_plan_t::new
 1.0%   2.1%  15.1KiB   harfrust harfrust::hb::tag_table::tags_from_complex_language
 1.0%   2.0%  14.8KiB   harfrust harfrust::hb::ot_shape::position_by_plan
 1.0%   2.0%  14.2KiB   harfrust harfrust::hb::ot_shape::substitute_pre
 0.9%   1.9%  13.4KiB   harfrust harfrust::hb::aat::layout_morx_table::apply
 0.8%   1.6%  11.6KiB   harfrust harfrust::hb::ot_shaper_vowel_constraints::preprocess_text_vowel_constraints
 0.7%   1.4%  10.2KiB   harfrust harfrust::hb::ot_shaper_use::setup_syllables
 0.7%   1.4%   9.8KiB   harfrust harfrust::hb::aat::AatCache::new
 0.6%   1.1%   8.3KiB   harfrust harfrust::hb::face::hb_font_t::shape_with_plan

---

At RustWeek, @cbiffle hinted that he might have a plan around how to develop a tool that did this a lot more accurately (he does work is an embedded context where every byte counts as you may only have an overall budget of 2kb for the entire binary). But it sounded like it would be a significant project to develop a new tool rather than a quick fix.

I suspect that the best approach may be a tool which acts as a linker and tracks symbols and their origins as it links them.

[nicoburns]

Not sure if it's useful but I was able to generate this JSON file meta.json which can be anaylyzed with https://esbuild.github.io/analyze

Using the profile.bloat from above:

cargo build --profile bloat --example shape
bloaty ./target/bloat/examples/shape -d sections,symbols -n 0  --csv | bloaty-metafile > meta.json