Skip to content

ESP32 no-op build costs ~161s where AVR, ESP8266 and STM32 cost 0.4s #1411

Closed

Description

@stevenmunn312-tech

fbuild recognises there is no work to do, says so, and then takes three minutes to finish
saying it. Its own timer reports it, and nothing is written to disk during that time — no
object file, no archive, no .elf, no .bin.

$ fbuild build -e esp32_esp32_esp32s3_opi -v --no-timestamp
Board: Espressif ESP32-S3-DevKitC-1-N8 / ESP32S3 @ 240MHz
No-op fingerprint matched; reusing existing ESP32 artifacts.
Flash: 732.27KB / 16.00MB (4.5%)
RAM:   79.40KB / 320.00KB (24.8%)
build succeeded in 181.5s (flash: 749848 bytes, ram: 81308 bytes)

Minimal reproduction — it needs no project at all

A throwaway directory with an empty lib/, no libraries of any kind, and a three-line
sketch:

[env:esp32_esp32_esp32s3_opi]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
build_flags = -DCORE_DEBUG_LEVEL=0 -DBOARD_HAS_PSRAM
board_build.memory_type = dio_opi
board_upload.flash_size = 16MB
board_build.partitions = default_16MB.csv
#include <Arduino.h>
void setup() { pinMode(2, OUTPUT); }
void loop() { digitalWrite(2, HIGH); delay(500); digitalWrite(2, LOW); delay(500); }

Build it once, then again:

No-op fingerprint matched; reusing existing ESP32 artifacts.
Flash: 361.84KB / 16.00MB (2.2%)
RAM:   72.76KB / 320.00KB (22.7%)
build succeeded in 161.2s (flash: 370527 bytes, ram: 74507 bytes)

161.2s to confirm that a blink sketch with no libraries has not changed.

It is specific to the ESP32 orchestrator

Same project, same host, same 2.5.21, same vendored lib/; only the environment differs:

Environment Objects in the build tree Populate No-op fbuild's own line
arduino_avr_uno 56 26.2s 0.4s reusing existing AVR artifacts
esp8266_nodemcuv2 113 39.0s 0.4s reusing existing ESP8266 artifacts
stm32_blackpill_f411ce 232 71.3s 0.4s reusing existing STM32 artifacts
esp32-s3-devkitc-1 (opi PSRAM) 255 ~6m 181.5s reusing existing ESP32 artifacts

The STM32 row is the size-matched control: 232 objects against ESP32's 255, within 10%,
and 450x faster. Populate time meanwhile tracks object count across all four (26.2 / 39.0
/ 71.3s for 56 / 113 / 232), so the compile path looks healthy everywhere — it is only the
reuse path, and only on one platform.

The project's own content is nearly irrelevant to the cost

Project Files in lib/ No-op
Our scaffold, vendored FastLED 4,506 181.5s
Same, with ci/examples/tests/docs/.git pruned 3,158 163.7s
Bare blink, no libraries at all 0 161.2s

Everything a project can contribute is that ~20s spread. The remaining ~161s is a floor an
empty ESP32 project pays, and the other three platforms do not pay at all.

What it is not

Not Windows file I/O, not the size of the vendored library tree, not translation-unit
count, not framework download or install — each is held constant or beaten outright by one
of the rows above.

A possible lead, offered as a lead and not a diagnosis

#214 describes the pre-cache behaviour as "the orchestrator still invoked the uncached
resolver, causing every build to repeat the entire directory walk", and wires the
library-selection cache into teensy and stm32 — two of the fast controls above. ESP32 is
named in neither that issue nor any follow-up we could find.

The bare-project floor rules out the project's libraries as the load, but not the
framework's: the ESP32 Arduino framework tree is 10,059 files here, and it is identical for
every ESP32 project, which would fit a fixed cost that no amount of emptying the project
reduces. We have not read the source; this is only where we would look first.

Impact

It is the dominant cost of an fbuild upload for us and does not shrink with the size of the
change. Re-flashing an unchanged design is 3m 40s through fbuild against 26.8s through
arduino-cli, and 181.5s of that difference is this.

Environment

fbuild 2.5.21, Windows 11. Measured through
Design Studio for FastLED.

Activity

  1. stevenmunn312-tech commented on Sep 2, 2026

    @stevenmunn312-tech
    Author

    Worth separating this from #1347 up front, since both read as "ESP32 is slow on Windows".

    #1347 is a cache miss: persist_failed from tempdir nondeterminism makes every
    framework rebuild cold, so the ~4s-per-file wall gets paid again.

    This is a cache hit that is slow anyway. fbuild prints No-op fingerprint matched; reusing existing ESP32 artifacts, writes no object file, archive, .elf or .bin — the
    build tree is byte-for-byte untouched afterwards — and still takes 161–181s.

    It is also measured in a stable project directory that has been building for weeks, not a
    fresh temporary one, so #1347's tempdir mechanism does not apply to these numbers.

    Our cold builds do corroborate #1347 independently, though — I have left that data point
    on that issue rather than here.

  2. added a commit that references this issue on Sep 3, 2026
    814ac6c
  3. added 2 commits that reference this issue on Sep 3, 2026
    87493b2
    8f866e5
  4. zackees commented on Sep 3, 2026

    @zackees
    Member

    @stevenmunn312-tech — released as fbuild 2.5.22 (on PyPI now):

    uv tool install --upgrade fbuild==2.5.22     # or: pip install -U fbuild==2.5.22
    

    Verified against the published wheel on the bare S3 blink from your reproduction — 161.2s → 0.3s, which is the 0.4s you measured for AVR, ESP8266 and STM32. The ESP32 row is no longer the outlier.

    Root cause and the per-MCU survey are written up in the reply on #1347 and in #1413. Short version: the SDK-libs completion check required <mcu>/lib/libfreertos.a, ESP32-S3 is the one MCU that keeps that archive in its memory-type variant dirs instead, so the check never passed and every build reinstalled 298 MB before reaching the fast path.

    Two notes on your write-up, since you asked the right questions:

    The first build after upgrading pays a one-time framework-core-cache warm (~3½ minutes here) because the cache key moves with the fbuild version; everything after is 0.3s.

    If it persists, please re-file rather than assuming it's the same bug — with FBUILD_PERF_LOG=1 fbuild build and the tail of %USERPROFILE%\.fbuild\prod\daemon\daemon.log. The perf line names the phase that ate the time.

    The remaining ~7s of a fresh-project build (only 25% of which is linking) is now tracked separately in #1414.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions