Debugging and Diagnostics

Build types, debug logging, and crash handling in libpromeki.

This page explains the tools libpromeki provides for diagnosing problems at development time and in production. It covers:

• Build Types — choosing the right CMake build type
• The Logger System — the Logger system and convenience macros
• Per-Module Debug Logging — per-module debug logging with PROMEKI_DEBUG
• Crash Handling — automatic crash reports for fatal signals

---

Build Types

libpromeki defines three CMake build types. The build type controls optimisation level, debug symbols, and whether per-module debug logging (promekiDebug()) is compiled in.

Build type	Optimisation	Debug symbols	promekiDebug()	NDEBUG	Typical use
Debug	-O0	Yes	Compiled in	Not set	Step-through debugging, sanitizers
DevRelease (default)	-O3	Yes (-g)	Compiled in	Set	Day-to-day development, CI, profiling
Release	-O3	No	Compiled out	Set	Distribution / final packaging

DevRelease — the recommended default

When no CMAKE_BUILD_TYPE is specified, the build defaults to DevRelease. This gives you release-level performance (-O3) plus two features that are invaluable during development and production troubleshooting:

• Debug symbols (-g) — so crash reports, core dumps, and profilers show meaningful function names and line numbers.
• **promekiDebug() compiled in** — the per-module debug logging system is available and can be activated at runtime via the PROMEKI_DEBUG environment variable without recompiling.

NDEBUG is still set, so assert() macros are disabled and the library runs at full speed.

# These are equivalent — DevRelease is the default:
cmake -B build
cmake -B build -DCMAKE_BUILD_TYPE=DevRelease

Debug

The Debug build type disables optimisation entirely and enables debug symbols. Use it when you need to step through code in a debugger (GDB, LLDB) or run sanitizers (ASan, TSan, UBSan).

cmake -B build -DCMAKE_BUILD_TYPE=Debug

Release

The Release build type enables full optimisation with no debug symbols and compiles out all promekiDebug() calls. Use it for final distribution builds where binary size and performance are paramount and debug logging overhead must be zero.

cmake -B build -DCMAKE_BUILD_TYPE=Release

---

The Logger System

All logging in libpromeki goes through the Logger class. The logger is thread-safe and asynchronous — log messages are enqueued and written by a dedicated background thread so the calling thread is never blocked on I/O.

Convenience Macros

In practice you will almost always use the convenience macros rather than calling Logger methods directly:

Macro	Level	When to use
promekiDebug(fmt, ...)	Debug	Per-module diagnostic output (see Per-Module Debug Logging)
promekiInfo(fmt, ...)	Info	Normal operational messages
promekiWarn(fmt, ...)	Warn	Recoverable problems worth investigating
promekiErr(fmt, ...)	Err	Errors that affect correctness

All macros accept printf-style format strings:

promekiInfo("Opened %s (%d x %d)", path.cstr(), w, h);
promekiWarn("Buffer underrun at %s", tc.toString().first().cstr());
promekiErr("Failed to bind socket: %s", err.desc().cstr());

Every log line automatically includes a timestamp, source file and line number, log level, and the name of the calling thread.

Configuring the Logger

The default logger writes to the console (stderr). You can also direct output to a file and adjust the minimum log level:

// Send log output to a file (in addition to the console)
Logger::defaultLogger().setLogFile("/tmp/myapp.log");
 
// Only show warnings and errors on the console
Logger::defaultLogger().setLogLevel(Logger::Warn);
 
// Disable console output entirely (file-only logging)
Logger::defaultLogger().setConsoleLoggingEnabled(false);

You can also install custom formatters for file and console output:

Logger::defaultLogger().setFileFormatter([](const Logger::LogFormat &fmt) -> String {
        return String::sprintf("[%c] %s",
                Logger::levelToChar(fmt.entry->level),
                fmt.entry->msg.cstr());
});

Flushing Log Output

Because logging is asynchronous, the log queue may contain unwritten messages when your program exits or when you need to inspect a log file mid-run. Use sync() to block until the queue is drained:

promekiInfo("About to do something risky...");
promekiLogSync();  // block until all queued messages are written

---

Per-Module Debug Logging (promekiDebug)

The promekiDebug() macro provides fine-grained, per-module debug logging that can be compiled in but only activated at runtime for the modules you care about. This lets you instrument the library heavily without any runtime cost until you need it.

How It Works

Each source file that wants to emit debug output places the PROMEKI_DEBUG macro near the top, inside the promeki namespace:

#include <promeki/myclass.h>
#include <promeki/logger.h>
 
PROMEKI_NAMESPACE_BEGIN
 
PROMEKI_DEBUG(MyClass)
 
MyClass::MyClass() {
        promekiDebug("MyClass(%p): created, size %d", (void *)this, _size);
}

The PROMEKI_DEBUG(Name) macro registers the module name with the debug database. At startup, the library checks the PROMEKI_DEBUG environment variable and enables logging for any module whose name appears in the comma-separated list.

Activating Debug Output

Set the PROMEKI_DEBUG environment variable to a comma-separated list of module names:

# Enable debug output for ThreadPool and MediaIO:
PROMEKI_DEBUG=ThreadPool,MediaIO ./myapp
 
# Enable debug output for everything (if modules are named):
PROMEKI_DEBUG=ThreadPool,Thread,Logger,MediaIO,RtpMediaIO ./myapp

When enabled, promekiDebug() messages appear alongside normal log output at the Debug level. When disabled, the module's promekiDebug() calls skip the logging call entirely — just a branch on a local bool.

What Happens in Release Builds?

In a Release build, PROMEKI_DEBUG_ENABLE is not defined, so the promekiDebug() macro expands to nothing. The compiler eliminates all debug logging calls entirely — zero overhead.

If you set the PROMEKI_DEBUG environment variable in a Release build, the library will print a warning:

[W] PROMEKI_DEBUG is set but promekiDebug() messages are compiled out.
    Rebuild with -DCMAKE_BUILD_TYPE=DevRelease or Debug.

Benchmarking with PROMEKI_BENCHMARK

Two companion macros let you time sections of code, gated on the same per-module enable flag:

void MyClass::processFrame(const Frame &frame) {
        PROMEKI_BENCHMARK_BEGIN(processFrame)
        // ... expensive work ...
        PROMEKI_BENCHMARK_END(processFrame)
        // prints: "[MyClass] processFrame took 0.003217000 sec"
}

Like promekiDebug(), these are compiled out in Release builds and only run when the module's debug flag is enabled.

---

Crash Handling

The CrashHandler class installs signal handlers for five fatal POSIX signals: SIGSEGV, SIGABRT, SIGBUS, SIGFPE, and SIGILL. When a crash occurs, it writes a detailed report to stderr and to a log file, then re-raises the signal so the OS can generate a core dump.

What the Crash Report Contains

• Signal name and number (e.g. SIGSEGV)
• Fault address and signal code
• Process ID and ISO 8601 UTC timestamp
• Crashing thread's TID and name
• List of all threads with TIDs and names
• Demangled C++ stack trace
• OS info (kernel version, architecture)
• Memory usage and resource limits
• Memory map (/proc/self/maps on Linux)
• Environment snapshot (if enabled)
• Registered MemSpace allocation counters

The crash log is written to:

<tempdir>/promeki-crash-<appname>-<pid>.log

The directory can be overridden via LibraryOptions::CrashLogDir or the PROMEKI_OPT_CrashLogDir environment variable.

Automatic Installation via Application

If your program uses the Application class, crash handlers are installed automatically. No extra code is needed:

#include <promeki/application.h>
 
int main(int argc, char **argv) {
        Application app(argc, argv);
 
        // Crash handlers are now active.
        // If the process crashes, a report is written automatically.
 
        return app.exec();
}

The Application constructor checks LibraryOptions::CrashHandler (default true). To disable it, set the option before constructing Application or via the environment:

// Programmatically
LibraryOptions::instance().set(LibraryOptions::CrashHandler, false);
Application app(argc, argv);
 
// Or via environment
// export PROMEKI_OPT_CrashHandler=false

Manual Installation

If your program does not use Application (e.g. a library consumer that has its own main loop), you can install the crash handler directly:

#include <promeki/crashhandler.h>
 
int main(int argc, char **argv) {
        // Install crash handlers as early as possible
        CrashHandler::install();
 
        // ... your application code ...
 
        // Optional: remove handlers before exit
        CrashHandler::uninstall();
        return 0;
}

install() is safe to call multiple times — subsequent calls refresh the snapshotted process state (hostname, thread list, MemSpace entries, etc.) without stacking signal handlers.

Refreshing the Snapshot

The crash handler snapshots process state at install() time so it can write the report from inside a signal handler without calling non-signal-safe functions. If your application registers new MemSpace entries or changes the app name after startup, call Application::refreshCrashHandler() (or just call CrashHandler::install() again) to update the snapshot:

Application app(argc, argv);
app.setAppName("my-tool");   // automatically refreshes
 
// If you register MemSpaces later:
MemSpace::registerData(...);
Application::refreshCrashHandler();

Diagnostic Traces Without Crashing

You can produce the same report a crash would generate — without actually crashing — using CrashHandler::writeTrace():

// Something unexpected happened; capture a diagnostic snapshot
if(unexpectedState) {
        CrashHandler::writeTrace("unexpected state in pipeline stage 3");
}

Each call writes to a unique file:

<tempdir>/promeki-trace-<appname>-<pid>-<seqno>.log

This is useful for capturing the state of a long-running process without stopping it.

Enabling Core Dumps

By default the crash handler does not modify the core dump resource limit. To enable core dumps, set LibraryOptions::CoreDumps to true:

// Programmatically
LibraryOptions::instance().set(LibraryOptions::CoreDumps, true);
 
// Or via environment
// export PROMEKI_OPT_CoreDumps=true

When enabled, install() raises RLIMIT_CORE to the hard limit so the kernel generates a core file on crash. Combined with the -g debug symbols from DevRelease builds, you get a core dump that GDB/LLDB can make full use of.

CrashHandler Library Options

All crash-related options can be set via code or environment variables:

Option	Env variable	Type	Default	Description
LibraryOptions::CrashHandler	PROMEKI_OPT_CrashHandler	bool	true	Install crash signal handlers
LibraryOptions::CoreDumps	PROMEKI_OPT_CoreDumps	bool	false	Raise RLIMIT_CORE for core dumps
LibraryOptions::CrashLogDir	PROMEKI_OPT_CrashLogDir	String	(empty = temp dir)	Directory for crash/trace log files
LibraryOptions::CaptureEnvironment	PROMEKI_OPT_CaptureEnvironment	bool	true	Include env vars in crash reports

---

---

Debug HTTP Server

When PROMEKI_ENABLE_HTTP is on (the default), Application can embed a lightweight HTTP debug server that exposes diagnostic information over a browser-friendly REST API and a baked-in web UI.

Quickstart: env-var activation

Set PROMEKI_DEBUG_SERVER to a host:port spec before launching your application. The constructor picks it up automatically — no code changes required:

# Bind to loopback on port 8085 (short form)
PROMEKI_DEBUG_SERVER=:8085 ./myapp
 
# Explicit host — binds to all interfaces
PROMEKI_DEBUG_SERVER=0.0.0.0:8085 ./myapp

Parse or bind failures are logged at warn level and are never fatal; the application continues without a debug server.

Then open http://localhost:8085/ in a browser — it redirects to the built-in debug UI at /promeki/debug/.

Programmatic control

Application app(argc, argv);
 
// Start on a specific port (loopback-only by default)
if(Error err = Application::startDebugServer(8085); err.isError()) {
        promekiWarn("debug server failed: %s", err.toString().cstr());
}
 
// ... later, to shut it down cleanly:
Application::stopDebugServer();
 
// Grab the server to add your own routes:
if(DebugServer *dbg = Application::debugServer()) {
        dbg->httpServer().route("/my/route", HttpMethod::Get, handler);
}

startDebugServer() returns Error::AlreadyOpen if the server is already listening. Call stopDebugServer() first to rebind.

Using DebugServer directly

Applications that own their own HttpServer can skip Application integration and cherry-pick individual module installers from <promeki/debugmodules.h>:

// Attach only the build-info and logger endpoints to your own server
installBuildInfoDebugRoutes(myServer, "/debug/api");
installLoggerDebugRoutes(myServer, "/debug/api");

Or use the full convenience wrapper on a dedicated server:

DebugServer dbg;
dbg.installDefaultModules();                 // mounts everything
if(Error err = dbg.listen(8085); err.isError()) {
        promekiWarn("debug server failed: %s", err.toString().cstr());
}
return app.exec();

API endpoints

All endpoints are mounted under /promeki/debug/api by default.

Endpoint	Method	Description
/promeki/debug/api/build	GET	Build metadata, features, platform info
/promeki/debug/api/env	GET	Full process environment as a JSON object
/promeki/debug/api/options/_schema	GET	LibraryOptions key schema (read-only)
/promeki/debug/api/memory	GET	Live MemSpace allocation counters
/promeki/debug/api/logger	GET	Logger state: level, console, debug channels
/promeki/debug/api/logger/level	PUT	Set log level — body {"level": <0-4>}
/promeki/debug/api/logger/debug/{name}	PUT	Toggle a debug channel — body {"enabled": <bool>}
/promeki/debug/api/logger/stream	WS	Stream log entries as JSON frames

Logger listener API

The debug server's WebSocket log stream is built on a general-purpose listener API exposed by Logger. Applications can install their own listeners to consume log entries programmatically:

Logger::ListenerHandle h = Logger::defaultLogger().installListener(
        [](const Logger::LogEntry &entry, const String &threadName) {
                // Called on the logger worker thread — marshal if needed
                myLogView->append(entry.msg);
        }, /*replayCount=*/50);   // replay last 50 entries first
 
// Stop receiving:
Logger::defaultLogger().removeListener(h);

removeListener() blocks until the worker thread has acknowledged removal, so it is safe to destroy state captured by the lambda immediately after it returns.

The history ring size (default 1024) controls how many entries are available for replay:

Logger::defaultLogger().setHistorySize(4096);

---

Quick Reference

Environment Variables

Variable	Purpose	Example
PROMEKI_DEBUG	Enable per-module debug logging	PROMEKI_DEBUG=ThreadPool,MediaIO
PROMEKI_DEBUG_SERVER	Start the debug HTTP server	PROMEKI_DEBUG_SERVER=:8085
PROMEKI_OPT_CrashHandler	Enable/disable crash handlers	PROMEKI_OPT_CrashHandler=false
PROMEKI_OPT_CoreDumps	Enable core dumps	PROMEKI_OPT_CoreDumps=true
PROMEKI_OPT_CrashLogDir	Override crash log directory	PROMEKI_OPT_CrashLogDir=/var/log/myapp
PROMEKI_OPT_CaptureEnvironment	Include env in crash reports	PROMEKI_OPT_CaptureEnvironment=false

Typical Debugging Workflow

# 1. Build with DevRelease (the default — just run cmake)
cmake -B build
cmake --build build
 
# 2. Run with debug logging for the subsystem you're investigating
PROMEKI_DEBUG=ThreadPool ./build/myapp
 
# 3. Optionally attach the debug HTTP UI to inspect runtime state
PROMEKI_DEBUG_SERVER=:8085 PROMEKI_DEBUG=ThreadPool ./build/myapp
# Then open http://localhost:8085/ in a browser
 
# 4. If a crash occurs, check the crash log
cat /tmp/promeki-crash-myapp-12345.log
 
# 5. For deeper investigation, enable core dumps
PROMEKI_OPT_CoreDumps=true ./build/myapp
# After crash:
gdb ./build/myapp core