* fix(fs): bound getFiles() result size and harden traversal
The recursive filesystem traversal in getFiles() could grow its result
vector without limit, contributing to heap pressure on memory-constrained
devices during client config starts. The implementation also relied on
unbounded strcpy and per-entry String() allocations for dot-suffix
filtering, and leaked the directory handle when the opened path was not
a directory.
Rework the traversal to address all of those at once:
- Add optional maxCount (default 64) and wasLimited out-param. The walk
stops once maxCount entries have been collected and wasLimited is set,
so callers can distinguish a complete listing from a truncated one.
Default arguments preserve backward compatibility for existing callers.
- Extract the recursive walk into a collectFiles() helper in an
anonymous namespace, accumulating into a single caller-owned vector
instead of building and inserting temporary vectors at every recursion
level.
- Replace strcpy with strlcpy so file_name writes are bounded by the
destination buffer.
- Replace String(name).endsWith(".") with a stack-based
pathEndsWithDot() helper to avoid a heap allocation per entry.
- Close the directory handle when the opened path is not a directory,
and close the file in both branches of recursion instead of only the
leaf branch (the previous code leaked the handle when depth was
exhausted).
- Compile-gate the std::bad_alloc reserve probe-down behind
__cpp_exceptions / __EXCEPTIONS so the fallback only compiles where
bad_alloc is actually catchable. No-exception targets (nRF52, RP2040,
STM32, ESP32-C6) skip the probe and rely on the bounded maxCount
alone, which is safe because the default cap is small enough to fit
comfortably in available RAM on every supported platform.
* fix(phone-api): skip manifest scan for node-info-only config requests
A client can request a node-info-only config response by sending
want_config_id with the SPECIAL_NONCE_ONLY_NODES nonce. The manifest
walk in handleStartConfig() ran unconditionally even though
STATE_SEND_FILEMANIFEST short-circuits past the manifest on that path,
so the SPI lock and recursive filesystem traversal were wasted work.
On memory-constrained devices this extra heap pressure was enough to
trigger allocation aborts and spontaneous reboots before the node-info
response could complete.
Gate the manifest call behind config_nonce != SPECIAL_NONCE_ONLY_NODES
so the node-info-only path no longer touches the filesystem at all, and
release any stale manifest storage left over from a previous run.
For normal full-config requests, also tighten the scan limits from
depth 10 / unbounded to depth 3 / max 64 entries via the new getFiles()
parameters. A 10-level recursive listing was never useful for a client
file picker and amplified heap pressure for no benefit. Log a warning
when the listing was truncated so operators can distinguish a bounded
result from a genuinely empty filesystem.
A few small cleanups fall out of the above:
- Replace raw spiLock->lock()/unlock() with a LockGuard so the lock is
released even if getFiles() throws on exception-enabled targets.
- Use a releaseFilesManifest() helper (swap-with-empty idiom) in
close() and the STATE_SEND_FILEMANIFEST drain path instead of the
previous clear()+shrink_to_fit() pair. Equivalent behavior, but the
swap idiom is the canonical vector-freeing pattern and centralizing
it avoids divergence between call sites.
- Use %zu for the size_t log arguments.
* fix(fs): harden bounded directory traversal
Close each child directory entry before recursive descent so bounded manifest scans do not stack file handles on constrained filesystems.
Treat null or overlong paths as truncated listings, handle reserve length failures like allocation failures, and keep manifest constants aligned with repository naming.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* bug #4184: fix config file loss due to filesystem write errors
* Use SafeFile for atomic file writing (with xor checksum readback)
* Write db.proto last because it could be the largest file on the FS (and less critical)
* Don't keep a tmp file around while writing db.proto (because too big to fit two files in the filesystem)
* generate a new critial fault if we encounter errors writing to flash
either CriticalErrorCode_FLASH_CORRUPTION_RECOVERABLE or CriticalErrorCode_FLASH_CORRUPTION_UNRECOVERABLE
(depending on if the second write attempt worked)
* reformat the filesystem if we detect it is corrupted (then rewrite our config files) (only on nrf52 - not sure
yet if we should bother on ESP32)
* If we have to format the FS, make sure to preserve the oem.proto if it exists
* add logLegacy() so old C code in libs can log via our logging
* move filesList() to a better location (used only in developer builds)
* Reformat with "trunk fmt" to match our coding conventions
* for #4395: don't use .exists() to before attempting file open
If a LFS filesystem is corrupted, .exists() can fail when a mere .open()
attempt would have succeeded. Therefore better to do the .open() in hopes that
we can read the file (in case we need to reformat to fix the FS).
(Seen and confirmed in stress testing)
* for #4395 more fixes, see below for details:
* check for LFS assertion failures during file operations (needs customized lfs_util.h to provide suitable hooks)
* Remove fsCheck() because checking filesystem by writing to it is very high risk, it makes likelyhood that we will
be able to read the config protobufs quite low.
* Update the LFS inside of adafruitnrf52 to 1.7.2 (from their old 1.6.1) to get the following fix:
https://github.com/littlefs-project/littlefs/commit/97d8d5e96a7781596708664f18f2ea6c3a179330
* use disable_adafruit_usb.py now that we are (temporarily?) using a forked adafruit lib
We need to reach inside the adafruit project and turn off USE_TINYUSB, just doing that
from platformio.ini is no longer sufficient.
Tested on a wio-sdk-wm1110 board (which is the only board that had this problem)
---------
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
* Make STM compile again and update toolchain. The binary is too big for the flash. WIP
* Making progress with OSFS, still WIP
* more progress, still too big. Adding RAK3172 to the equasion
* Make STM compile again and update toolchain. The binary is too big for the flash. WIP
* Making progress with OSFS, still WIP
* more progress, still too big. Adding RAK3172 to the equasion
* still too big
* minimize build
* trunk fmt
* fix a couple of symbol clashes
* trunk fmt
* down to 101% with a release vs. debug build and omitting the flash strings
* fix compilation
* fix compilation once more
* update protobufs linkage
* - Toolchain updated
- Fixed macro error
* silence compiler warning
note: do something about this assert...
* new toolkit and fix Power.cpp
* STM32WL make it fit (#4330)
* Add option to exclude I2C parts
The I2C hals and related code uses a significant amount of flash space and aren't required for a basic node.
* Add option to disable Admin and NodeInfo modules
Disabled by default in minimal build. This saves a significant amount of flash
* Disable unused hals
These use up significant flash
* Add float support for printf for debugging
Makes serial look nice for debugging
* This breaks my build for some reason
* These build flags can save a bit of flash
* Don't disable NodeInfo and Admin modules in minimal build
They fit in flash
* Don't include printf float support by default
Only useful for debugging
---------
Co-authored-by: Adam Lawson <dev@goshawk22.uk>
---------
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Adam Lawson <dev@goshawk22.uk>
* Wio-e5 / STM32WL wip
* Stubbing some FS stuff out
* Wio-e5 / STM32WL wip
* Stubbing some FS stuff out
* Wio-e5 / STM32WL wip
* Stubbing some FS stuff out
* Wio-e5 / STM32WL wip
* Stubbing some FS stuff out
* LittleFS compiles. Can't check with actual device.
* make cppcheck happy again
* Guard against accelerometer thread
* Missed a spot
* Upload via ST-LINK
* Derive MAC address from UID
* upload port
* Trunk it
* Guard it
* Maybe fix the cache error on startup.
* Latest RadioLib ref to fix SubGHZ
* revert nasty Sub-GHz Hack
* Boots and radio inits with RadioLib 6.0, LittleFS doesn't seem to work
---------
Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: GUVWAF <thijs@havinga.eu>
* lay groundwork for a possible future architecture
* switch from feature opt-out to feature opt-in
* lay groundwork for a possible future architecture
* switch from feature opt-out to feature opt-in
* fix USE_RTC in variant.h for rak4631_epaper and t-echo
* ensure Screen.h is not included without configuration.h
Co-authored-by: Peter Lawrence <12226419+majbthrd@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
@mc-hamster seems to work pretty good for me, so I'll send a PR to you
for the dev-http branch.
I'll push out an android alpha build later today (once the build is
complete). Once this new device load is out in the field _future_
device builds will support updating spiffs from android. (i.e. device
loads older than 1.1.9 must be updated to 1.1.9 or later before spiffs
support is implemented on the device side - so some users might need to
update twice before the new spiffs contents will appear on their device)
