4 releases
0.12.2 | Oct 28, 2024 |
---|---|
0.12.1 | Oct 28, 2024 |
0.12.0 | Oct 28, 2024 |
0.11.1 | Oct 28, 2024 |
#127 in Parser implementations
331 downloads per month
23KB
276 lines
lq - low overhead yq/tq/... implementation
A lightweight and portable jq wrapper for doing arbitrary queries from YAML/TOML/JSON documents by converting to JSON and passing to jq
, then returning the result either as raw jq
output, or back into TOML or YAML.
Installation
Via cargo:
cargo install lq
or download a prebuilt from releases either manually, or via binstall:
cargo binstall lq
Note: Depends on jq
being installed.
Why / Why Not
jq compatibility
- arbitrary
jq
usage on any input format (yaml/toml/json) - same filter syntax (shells out to
jq
) - matches
jq
's cli interface (only some extra input/output format controlling flags) - supports
jq
output formatters such as-c
,-r
, and-j
(compact, raw, joined output resp) - supports jq modules on all input formats
Features
- reads multidoc yaml input, handles yaml merge keys (expanding tags)
- reads from stdin xor file (file if last arg is a file)
- output conversion shortcuts:
-y
(YAML) or-t
(TOML) - drop-in replacement to python-yq (with
alias yq=lq
) - ~1MB in binary size (for small cloud CI images / binstalled ci actions)
Limitations
- Shells out to
jq
(supports what yourjq
version supports) - Expands YAML tags (input is singleton mapped -> recursively, then merged) - so tags are not preserved in the output
- Does not preserve indentation (unsupported in serde_yaml)
- Does not support duplicate keys in the input document
- No XML/CSV support (or other more exotic formats)
Usage
YAML Input
Use as jq either via stdin:
$ lq '.[3].kind' -r < test/deploy.yaml
Service
$ lq -y '.[3].metadata' < test/deploy.yaml
labels:
app: controller
name: controller
namespace: default
or from a file arg (at the end):
$ lq '.[3].kind' -r test/deploy.yaml
$ lq -y '.[3].metadata' test/deploy.yaml
The default input format is YAML and is what the binary is named for (and the most common primary usage case).
If you want to use it as the top level yq
executable you can alias yq=lq
. This should be compatible with python-yq
, but it has some differences with the go yq.
TOML Input
Using say Cargo.toml
from this repo as input, and aliasing tq='lq --input=toml'
:
$ tq '.package.categories[]' -r < Cargo.toml
command-line-utilities
parsing
$ tq -t '.package.metadata' < Cargo.toml
[binstall]
bin-dir = "yq-{ target }/{ bin }{ format }"
pkg-url = "{ repo }/releases/download/{ version }/yq-{ target }{ archive-suffix }"
$ tq -y '.dependencies.clap' < Cargo.toml
features:
- cargo
- derive
version: 4.4.2
$ tq '.profile' -c < Cargo.toml
{"release":{"lto":true,"panic":"abort","strip":"symbols"}}
Add alias tq='lq --input=toml'
to your .bashrc
or .zshrc
(etc) to make this permanent if you find it useful.
JSON Input
If you need to convert json to another format you pass --input=json
:
$ lq --input=json '.ingredients | keys' -y < test/guacamole.json
- avocado
- coriander
- cumin
- garlic
- lime
- onions
- pepper
- salt
- tomatoes
Advanced Examples
Select with nested query and raw output:
$ lq '.spec.template.spec.containers[].image' -r < test/grafana.yaml
quay.io/kiwigrid/k8s-sidecar:1.24.6
quay.io/kiwigrid/k8s-sidecar:1.24.6
docker.io/grafana/grafana:10.1.0
Select on multidoc:
$ lq -y '.[] | select(.kind == "Deployment") | .spec.template.spec.containers[0].ports[0].containerPort' test/deploy.yaml
8000
Escaping keys with slashes etc in them:
lq -y '.updates[] | select(.["package-ecosystem"] == "cargo") | .groups' .github/dependabot.yml
Using helpers from jq
modules e.g. k.jq:
$ lq 'include "k"; .[] | gvk' -r -L$PWD/test/modules < test/deploy.yaml
v1.ServiceAccount
rbac.authorization.k8s.io/v1.ClusterRole
rbac.authorization.k8s.io/v1.ClusterRoleBinding
v1.Service
apps/v1.Deployment
Output Caveats
Output formatting such as -y
for YAML or -t
for TOML will require the output from jq
to be parseable json.
If you pass on -r
,-c
or -c
for raw/compact output, then this will generally not be parseable as json.
Debug Logs
The project respects RUST_LOG
when set, and sends these diagnostic logs to stderr:
$ RUST_LOG=debug lq '.version' test/circle.yml
2023-09-18T23:17:04.533055Z DEBUG lq: args: Args { input: Yaml, output: Jq, yaml_output: false, toml_output: false, in_place: false, jq_query: ".version", file: Some("test/circle.yml"), compact_output: false, raw_output: false, join_output: false, modules: None }
2023-09-18T23:17:04.533531Z DEBUG lq: found 1 documents
2023-09-18T23:17:04.533563Z DEBUG lq: input decoded as json: {"definitions":{"filters":{"on_every_commit":{"tags":{"only":"/.*/"}},"on_tag":{"branches":{"ignore":"/.*/"},"tags":{"only":"/v[0-9]+(\\.[0-9]+)*/"}}},"steps":[{"step":{"command":"chmod a+w . && cargo build --release","name":"Build binary"}},{"step":{"command":"rustc --version; cargo --version; rustup --version","name":"Version information"}}]},"jobs":{"build":{"docker":[{"image":"clux/muslrust:stable"}],"environment":{"IMAGE_NAME":"lq"},"resource_class":"xlarge","steps":["checkout",{"run":{"command":"rustc --version; cargo --version; rustup --version","name":"Version information"}},{"run":{"command":"chmod a+w . && cargo build --release","name":"Build binary"}},{"run":"echo versions"}]},"release":{"docker":[{"image":"clux/muslrust:stable"}],"resource_class":"xlarge","steps":["checkout",{"run":{"command":"rustc --version; cargo --version; rustup --version","name":"Version information"}},{"run":{"command":"chmod a+w . && cargo build --release","name":"Build binary"}},{"upload":{"arch":"x86_64-unknown-linux-musl","binary_name":"${IMAGE_NAME}","source":"target/x86_64-unknown-linux-musl/release/${IMAGE_NAME}","version":"${CIRCLE_TAG}"}}]}},"version":2.1,"workflows":{"my_flow":{"jobs":[{"build":{"filters":{"tags":{"only":"/.*/"}}}},{"release":{"filters":{"branches":{"ignore":"/.*/"},"tags":{"only":"/v[0-9]+(\\.[0-9]+)*/"}}}}]},"version":2}}
2023-09-18T23:17:04.533650Z DEBUG lq: jq args: [".version"]
2023-09-18T23:17:04.538606Z DEBUG lq: jq stdout: 2.1
2.1
Dependencies
~7–10MB
~175K SLoC