deuxyeux/RTNode-HeltecV4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v1.0.8: Fix heap exhaustion from unbounded random_blobs

Browse Source 
Root cause: Python Reticulum trims random_blobs per destination entry
(MAX_RANDOM_BLOBS=64 in-memory, PERSIST_RANDOM_BLOBS=32 on disk).
The C++ firmware had these constants defined but NEVER enforced them,
causing unbounded growth. With 21 paths x 60+ blobs x ~90 bytes each,
the destination table alone consumed ~57KB of the ESP32 324KB heap.

Fixes:
- Trim random_blobs after insert (matching Python behavior)
- Trim random_blobs on deserialization from flash
- Trim random_blobs to PERSIST_RANDOM_BLOBS on serialization
- Enforce _path_table_maxpersist when writing path table (was declared
  but never used - write_path_table saved everything)
- Reduce MCU constants: MAX_RANDOM_BLOBS 64->16, PERSIST_RANDOM_BLOBS 32->8
- Reduce path_table_maxsize 128->24, maxpersist 32->12
- Add memory diagnostic after path table load
- Trim loaded paths to maxsize on startup via cull_path_table()

Results: destination_table 21KB->5.8KB, free heap 63K(22%)->156K(49%)
This commit is contained in:
James L
2026-02-27 17:51:20 -05:00
parent 59784a34fd
commit 4e5d4ee8ad
4 changed files with 95 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
lib/microReticulum/src/Utilities/Persistence.h
View File

@@ -4,6 +4,7 @@
// then they MUST be included BEFORE this header is included.
#include "Transport.h"
#include "Type.h"
#include "Utilities/OS.h"
#include <ArduinoJson.h>
@@ -278,7 +279,23 @@ namespace ArduinoJson {
dst["received_from"] = src._received_from;
dst["announce_hops"] = src._hops;
dst["expires"] = src._expires;
dst["random_blobs"] = src._random_blobs;
// Trim random_blobs to PERSIST_RANDOM_BLOBS before writing to disk
if (src._random_blobs.size() > RNS::Type::Transport::PERSIST_RANDOM_BLOBS) {
std::vector<RNS::Bytes> blob_vec(src._random_blobs.begin(), src._random_blobs.end());
std::sort(blob_vec.begin(), blob_vec.end(), [](const RNS::Bytes& a, const RNS::Bytes& b) {
uint64_t ts_a = RNS::Utilities::OS::from_bytes_big_endian(a.data() + 5, 5);
uint64_t ts_b = RNS::Utilities::OS::from_bytes_big_endian(b.data() + 5, 5);
return ts_a > ts_b;
});
std::set<RNS::Bytes> trimmed;
for (size_t i = 0; i < RNS::Type::Transport::PERSIST_RANDOM_BLOBS && i < blob_vec.size(); i++) {
trimmed.insert(blob_vec[i]);
}
dst["random_blobs"] = trimmed;
}
else {
dst["random_blobs"] = src._random_blobs;
}
/*
//dst["interface_hash"] = src._receiving_interface;
if (src._receiving_interface) {
@@ -320,6 +337,19 @@ namespace ArduinoJson {
dst._hops = src["announce_hops"];
dst._expires = src["expires"];
dst._random_blobs = src["random_blobs"].as<std::set<RNS::Bytes>>();
// Trim random_blobs loaded from flash to MAX_RANDOM_BLOBS
if (dst._random_blobs.size() > RNS::Type::Transport::MAX_RANDOM_BLOBS) {
std::vector<RNS::Bytes> blob_vec(dst._random_blobs.begin(), dst._random_blobs.end());
std::sort(blob_vec.begin(), blob_vec.end(), [](const RNS::Bytes& a, const RNS::Bytes& b) {
uint64_t ts_a = RNS::Utilities::OS::from_bytes_big_endian(a.data() + 5, 5);
uint64_t ts_b = RNS::Utilities::OS::from_bytes_big_endian(b.data() + 5, 5);
return ts_a > ts_b;
});
dst._random_blobs.clear();
for (size_t i = 0; i < RNS::Type::Transport::MAX_RANDOM_BLOBS && i < blob_vec.size(); i++) {
dst._random_blobs.insert(blob_vec[i]);
}
}
/*
//dst._receiving_interface = src["interface_hash"];
RNS::Bytes interface_hash = src["interface_hash"];