breakmancer

Drop a breakpoint into any shell.

Need to debug a build script, but don't have a way to SSH into the server to poke around in the build environment—or you do, but you need a way to pause the script at exactly the right point? breakmancer allows you to do just that!

Functionally, this is a reverse shell for people who are authorized to jump into a remote server. Unlike the malicious kind of reverse shell, this one aims to preserve the security of the target.

Requirements

• A development computer that is capable of listening on a public IP address (or an intranet IP if it's on the same network as the computer you're debugging.)
• A target computer where you're trying to debug a script and that you have permission to shell into. This computer must be able to reach out to the development computer over TCP.

Usage

In this example, the development computer is reachable on port 12345 at the domain name home.timmc.org. The target is a GitHub Actions workflow.

1. Start session locally: I run breakmancer listen home.timmc.org:12345 on my laptop. It prints out instructions and waits ("Now listening").
2. Set the secret on the remote server: I create a GitHub repository secret called breakmancer_callback with the secret from the command output above (sEYKNzulK6v8zE3gh23xkg).
3. Set up the breakpoint: I add the breakpoint command that was printed out above to my GitHub workflow file. It might look like this:

   name: Breakmancer Demo
   on: [push]
   jobs:
     demo:
       runs-on: ubuntu-latest
       steps:
         - uses: actions/checkout@v4
   
         - name: Do some stuff
           run: |
             date > something.txt
   
         - name: Invoke breakpoint
           env:
             BM_SETUP_SECRET: "${{ secrets.breakmancer_callback }}"
           run: |
             chmod +x ./breakmancer
             ./breakmancer break -c home.timmc.org:12345 -i TEXXICzjphp1mhzknJ/VfkHyOOhd6efImGQcrXxSjyI
   

   • The workflow fetches the securely stored secret and exports it as an environment variable.
   • Unlike the callback secret, the command line is safe to show publicly. It just contains an address and a public key.
   • I also commit the breakmancer binary to my branch.
   • I've adjusted the command to call ./breakmancer (it's not on the PATH) and I've ensured the binary is actually executable.
4. Run: I push the branch, the GitHub Action runs, and breakmancer reaches back to my laptop, which opens a shell!

   Connection from [::ffff:52.234.38.67]:61440
   Session ready. You can enter single-line commands. Use `exit` to exit.
   >> hostname
   [out] fv-az1756-422
   [exit: 0]
   >> pwd
   [out] /home/runner/work/manual-testing/manual-testing
   [exit: 0]
   >>
   

5. End: When I'm done, I use exit (or ^C or ^D) to exit and allow the workflow to continue; at the new prompt I can choose to wait for a new connection or end the program entirely.
6. Audit log: Looking at the GitHub workflow output, I see a log of what commands were run:

   Connecting to listener at 130.44.147.140:12345
   [2024-06-25T22:33:27Z] Waiting for first command...
   [2024-06-25T22:33:30Z] Running command: hostname
   [2024-06-25T22:33:39Z] Running command: pwd
   Listener asked for normal execution to resume; exiting breakpoint.
   

Tip: Once the listener has exited, the secret and command line given to the breakpoint side will no longer work and will need to be replaced. That can be annoying. So until you're sure that you're done, you may want to leave the listener running.

Build, test, and release

• Update version in Cargo.toml
• Ensure changelog has already been filled out
• cargo clippy --all-targets
• cargo test
• cargo build --release
• Do any manual testing
• cargo publish
• git tag -a vX.Y.Z -m X.Y.Z
• git push --tags
• Update the changelog version (add a new release header below the Unreleased header)
• git push

Manual testing

Some interesting things to enter:

• (yes "out" | head -n20)& (yes " err" | head -n20 >&2) to show interleaving of stdout and stderr
• echo start; sleep 10; echo stop to experiment with blocking commands
• while true; do date; sleep 0.001; done for a rapid output stream (not as fast as yes)

Limitations

• As breakmancer does not create a PTY, buffering can occur in some pipelines. This creates two problems:

  1. All of the output happens at once. In the command for x in {1..5}; do date; sleep 1; done | head we see nothing for 5 seconds, and then 5 lines of output all at once. (The output reveals that each date invocation did indeed occur 1 second apart.)
  2. If such a streaming command never terminates, the output is never sent. The command while true; do date; sleep 1; done | head will hang forever with no output.

  In those examples, dropping head from the pipeline (or piping to a different binary such as cat) allows the output to stream normally.

• See TODO.md for more bugs and wanted features.

License

Copyright 2024 Tim McCormack

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.