1 unstable release
0.1.0 | Jun 26, 2024 |
---|
#581 in Debugging
21KB
400 lines
loggy
loggy
wraps commands and automatically tees their output to log files without modifying original programs or scripts.
Features
- Logs command output (stdout and stderr) to automatically-created files
- Passthrough mode for piping: like
tee
without the need to specify a name - High performance: gigabytes of output per second, sub-millisecond startup time
- Handles SIGHUP and closed stdout/stderr (continues after terminal or SSH disconnection)
- Wraps any command via symlinks, aliases, or Bash magic; configurable via regex patterns
Installation
From source
sudo -E "$(command -v cargo)" install --root /usr/local si-loggy
From Git
sudo -E "$(command -v cargo)" install --root /usr/local --git https://github.com/Standard-Intelligence/loggy
Usage
Manual invocation
Wrap a command with loggy
to log its output:
$ loggy echo hello world
[loggy] logging to /home/robert/logs/echo-0.log
hello world
Or, pipe a command to loggy
:
$ echo hello world | loggy
[loggy] logging to /home/robert/logs/loggy-14.log
hello world
Wrapping a command (i.e. running loggy command
or using one of the below methods) is preferable to piping to loggy
because loggy
can handle SIGHUP and closed stdout/stderr (such as from a detached terminal) on behalf of wrapped processes.
Symlinks and aliases
Symlink loggy
to each program to log:
ln -s /usr/local/bin/loggy /usr/local/bin/pip
ln -s /usr/local/bin/loggy /usr/local/bin/python
ln -s /usr/local/bin/loggy /usr/local/bin/python3
When a wrapped program is run, loggy
creates a log file in ~/logs
based on the command name and any arguments corresponding to existing files:
$ pip install -r requirements.txt
[loggy] logging to /home/user/logs/pip-requirements.txt-0.log
If the wrapped program produces no output (e.g. loggy true
), no log file is created.
Environment variable
loggy
can be temporarily disabled for a wrapped command by setting the NO_LOGGY
environment variable (to any value except 0
):
NO_LOGGY= python3 -m print-all-my-secrets
Exec hook
For loggy
to handle broad patterns that could occur in any command (such as the /sandbox
example below), it must run for every command (at which point a config file determines which commands are actually logged). Add this to your .bashrc
to run every command under loggy
:
loggy() {
if [[ -v AT_PROMPT ]]; then
unset AT_PROMPT
local t="$(type -t $@)"
if [[ "$t" == "file" ]]; then
command loggy $@
shopt -s extdebug
return 1
elif [[ "$t" == "function" ]]; then
$@
shopt -s extdebug
return 1
fi
fi
}
unset -f command_not_found_handle
trap 'loggy ${BASH_COMMAND}' DEBUG
PROMPT_COMMAND="shopt -u extdebug; ${PROMPT_COMMAND}; AT_PROMPT="
Config file
By default, loggy
creates logs for anything it wraps. You may want finer control than symlinks and NO_LOGGY
offer, e.g. to log only invocations of a certain Python module. As a last resort, loggy
loads a list of regular expressions from the first existing file out of ~/.config/loggy
and /etc/loggy
. If one of these files exists, and a command is wrapped by loggy
, it will be logged if and only if it matches one of the regular expressions from the config file.
Example configuration:
# Log `make` commands with any arguments
^make( |$)
# Log all Python scripts, but not e.g. `python -m pip`
^([^ ]*/)?(python3?|python-[0-9.]+) .*\.py\b
# Log invocations of a specific Python module
^([^ ]*/)?(python3?|python-[0-9.]+) -m ?torch\.distributed\.run
# Log any commands involving the /sandbox directory
/sandbox
We recommend using config files as a last resort, because writing regex is hard :)
Tips
If you've run loggy
and then disconnected the terminal or SSH connection, run tail -f ~/logs/example.log
in a new terminal to stream its output.
If installed, loggy
automatically uses stdbuf
to ensure wrapped commands write their output to the terminal and log file line by line instead of with full buffering. Ensure stdbuf
is installed for optimal performance.
Dependencies
~4–14MB
~180K SLoC