Hey HN friends!

I've taken `fx` back to the drawing board and completely rewritten it from the ground up. Excited to share what's new:

1. *Going Big*: `fx` now gracefully handles even the most massive JSON files.

2. *A New TUI Look*: Dive deep into your data with a revamped terminal interface—now with themes!

3. *Swift Navigation with Dig Fuzzy Search*: Feeling lost in JSON? Just type `.` and navigate with ease.

4. *Powerful Regex Search*: Scan across your entire JSON content with precision.

5. *Elegant Long String Wraps*: No more cut-offs. Your strings wrap beautifully now.

6. *All Things JSON*: Added love for comments, trailing commas, and JSON streams.

Pouring my heart and soul into this rewrite has been a journey to make `fx` faster and more powerful. If you find value in what I've crafted and want to support its future, consider sponsoring on GitHub.

Would love to hear your thoughts and feedback!

What can you do with the JSON in the TUI? Anything besides looking at it? (https://fx.wtf/getting-started#json-processing but from the TUI?)

Built something similar, but with love for all things CUE (also works for JSON/Yaml) (https://youtu.be/XNBqBWO4y08)

Yours looks to have a better object navigation, I might have to be inspired by it!

How do you like Bubbletea? We went with tcell/tview (fork) for more control of the UI & event handling

wow nice, I'll give this a try.

I've always used jq for viewing json, but it's the wrong tool for my use. I basically want to explore json interactively, not parse it specifically. So the workflow is to keep "jq"ing it, and modifying my jq query until I find what I'm looking for.

Your tool looks like it allows interactive exploration.

You may be interested in an interactive jq tool like jiq or others: https://github.com/fiatjaf/awesome-jq (do a ctrl-f for "interactive" on this page).

That page doesn't include jless[1], which seems very similar to fx and will give you a jq selector for whatever node you're on

1: https://jless.io/

This thread is a gem after gem, thank you!

Some feedback:

- Some indication of how far down you've scrolled would be useful for "situational awareness". Perhaps a percentage number in the status bar (like emacs/vim?)

- I miss not having the Home/End keys bound to the goto top/bottom actions

- The cursor behavior when pressing PageDn/PageUp throws me off.

Emacs when pressing PageDn: Cursor ends up at the top of the window.

Vim: Cursor stays at the current position.

notepad.exe: Cursor ends up in the middle of the window.

Fx: Cursor ends up at the bottom of the window.

- The . navigation is very neat with the tab completion. Probably stupid observation: If you remove the alphabet key shortcuts you wouldn't need the (non-discoverable) dot. :)

Thanks for the feedback. Will try to address it in future releases.

Yes pgdown behaviour need to be refactored. Home/end is easy to add.

> If you remove the alphabet key shortcuts you wouldn't need the (non-discoverable) dot. :)

I did not get this one.

This looks really nice, well done!

I never could get used to jq's syntax so instead I've just been grepping. But this looks to be for jq what htop is to top, which is great for exploring response data.

Hello,

Is there a commonly-used grammar for "json with comments and trailing commas" or does everyone just try to copy the jsonc from vscode?

It's basically https://json5.org/

CUE will happily take your JSON with comments and extra commas, or no commas too (lists still need them), most keys can be unquoted, and it's just a one-liner to get valid json or yaml from it

Jsonnet does too I believe. I wanted to like Cue but it only works with Go and it doesn't actually let you link the schemas you write from the "document" files, which means no automatic schema checking and no IDE support. Giving up two huge benefits of having a schema in the first place.

I'm not sure what you mean about the schema / documents thing. Can you elaborate? I've not had problems "linking" (?) data and schemas. I'd guess the lack of documentation is more to blame here.

CUE is most often used as a CLI, so no Go is needed, that's really for advanced cases. Being Go, you can compile to SO or WASM, so can use it from pretty much any language.

The lack of LSP is being actively worked on right now, I'm hoping they'll have something before the year is out

In an XML document you can specify the XML schema location with `xsi:schemaLocation`. In a JSON document you can specify the JSON Schema location with `$schema`.

In a Cue document how do you specify the location of the Cue schema? As far as I can tell you can't.

And yes I am aware that the concept of a "document" and a "schema" are basically merged in Cue, but that doesn't mean I don't want to do this.

> CUE is most often used as a CLI, so no Go is needed

Only because that's the only way you can use it.

> Being Go, you can compile to SO or WASM

Compiling to a shared library is awkward because you then need to have the Go toolchain installed. WASM might be an option ... I couldn't find out how Go actually works in WASM though given that it needs a GC and threads, and WASM currently has neither. Do they ship their own WASM GC?

Either way, it's of no practical use since nobody has actually done that. Look at the languages supported by Jsonnet for comparison: https://jsonnet.org/ref/bindings.html

Python, C, Rust (twice), Go (twice), Lua, Node, PHP, Ruby, Haskell.

If I'm loading a config file from Python I don't want to have to mess around with `subprocess.run()` to read it. It's a solvable problem... but today, it's a bit of a turn-off.

> nobody has actually done that

CUE, running in the browser: https://cuelang.org/play/?id=#cue@export@cue

I see you are not interested in CUE and the benefits it offers, despite being a young language without all the ecosystem yet

We subprocess.run because there are many problems better solved in CUE that make our other code so much simpler. I can't even tell you the 10s of thousands of lines I don't have to write, both python and other config languages require way more verbosity. There are upsides for the downsides to early adoption

> In a Cue document how do you specify the location of the Cue schema?

CUE has proper dependency management* so you specify the schema through dependencies and imports, like we do with most languages we use. JSON and Yaml are notable exceptions to this, and things like what XML and JSONNet do, referring to an http (or remote) location that may or may not serve the same content between requests, make for irreproducibility.

* or will soon, there will be an experimental version in the next cue release, and hof has had it for 2+ years now

> there will be an experimental version in the next cue release

Ah cool, I will check it out later then. Good to know people are working on the issue.

> CUE, running in the browser

No no. By "that" I mean "provide libraries for other languages that wrap it using WASM".

> I see you are not interested in CUE and the benefits it offers

That's a bad attitude. Would I have learnt how to use Cue and about these limitations because I wasn't interested? You just don't want to admit that these are valid concerns.

Subprocess.run is clearly not the ideal way to interface with a configuration language. You can do it if you have to. But you really shouldn't.

Imagine if the only JSON parser in the world was written in OCaml and every program that wanted to read JSON had to run some OCaml program to convert it into another format that they could load. Obviously not a good state of affairs.

> That's a bad attitude.

I think we were probably talking past each other, my bad for continuing it

> Subprocess.run is clearly not the ideal way to interface with a configuration language.

100% agreement, but still better than the alternatives we had for addressing a complex problem.

yea, the interfaces to other languages is part of building out that ecosystem. The language is still changing a bit, but should hopefully stabilize in 0.8 or 0.9 (0.7 is going to bring significant performance improvements)

Yaml is another interesting case where there is inconsistent support and different defaults, depending on the library or tool. Config glues our world together, it's a complex mess because of where it sits. Everything we have today is cobbled together. To me, CUE offers a really great theory and foundation if we want to do something about it as an industry. Lord knows we're all suffering in YamHell

> You just don't want to admit that these are valid concerns.

I have my complaints and frustrations for sure, and I have been direct with the CUE team (in private) about some of the more sensitive issues, there are some comments on github and in public-ish documents if you're interested, mostly around the module proposal, which is in much better shape than anything written. The YT recordings are where the latest can be learned

http://hyperrealm.github.io/libconfig/ ?

This looks like an implementation of some totally different format. Additionally, the authors of that format chose not to specify some parts of its grammar.

I am probably missing something but is interactive mode not a flag? Is it just what happens with a longer JSON automatically?

how do you use the Built-in Functions? i tried them but it gives me error every time

Why not call it Jim? JSON vim

> In fx, arguments are treated as JavaScript functions. Input data is passed sequentially through each provided function.

> echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'

Wow, that is so nice. Having to memorize JQ syntax is such a pain.

Would've been awesome if results are displayed under the commands. For example, command

  echo '{"name": "hello"}\n{"name": "world"}' | fx '.name'

would output what? My guess is hello world but it might be hello\nworld.

Having used https://jqlang.github.io/jq/, wondering your pitch on advantages, alternative use cases for using fx?

Not OP but

      echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'

Is immediately grokkable compared to jq's syntax.

I love jq, I think it's extremely powerful, but I have to load up the documentation in a side window every time I have use it.

Equivalent jq seems okay here:

    echo '{"name": "world"}' | jq '"Hello \(.name)"'

Yes and both can be expressed in js and jq. Poinjt is what JS is known by much more peopel. As fx uses Node or Deno we for example can use lodash in fx:

    fx '_.groupBy("commit.author.name")' '_.mapValues(size)'
       '_.toPairs' '_.sortBy(1)' '_.reverse' '_.take(10)' '_.fromPairs'

To do if in jq, it will take me much more time and extensive documentation reading.

What’s the original JSON? Happy to try and express this in jq

Looks like the GitHub API: https://docs.github.com/en/free-pro-team@latest/rest/commits...

I'm looking forward to your jq version. I love jq and use it all the time, but for anything of that complexity, I'd probably just use Python.

If the task was the produce an object with the top 10 commiters with username as key and value number of commits maybe something liket this:

    group_by(.commit.author.name) | map({key: first.commit.author.name, value: length}) | sort_by(.value) | reverse[:10] | from_entries

As someone who doesn't use jq, a number of questions immediately jump out at me. Why is the input string double qouted ('"input"')? What does the backslash annotate? At first it looks like an escape similar to regex, but the other bracket doesn't have it so it must mean something else? Etc. Of course I could look the answers up, but the point is I can't tell without looking it up. While the code is short it feels similar to Awk, if you know it it's trivial, if you don't know it it's hard to be sure what it does without trying it out.

jq seems shorter but I don't see how it's more intuitive - ".name" is pretty basic jq syntax but how do I know \() is syntax for wrapping selectors within strings? I've used jq for many years and tbh I've never done string templating with it, so I would need to go digging in the docs.

You can argue that's the case for any language, but it's the fundamental reason DSLs rarely capture the mainstream for on-the-fly utilities. Leveraging existing knowledge of a more heavily used syntax really helps here. Look at awk for example - it's been around forever, and despite its power and ubiquity in OS installs, you'll still see sed in many many more bash scripts because it's leaning on widely used familiar syntax rather than a DSL.

That said - the fx example has "x => Hello, ${x}!" where I would expect "x => `Hello, ${x}!`", so that does indicate to me that fx has its own gotchas lurking - this example could just be contrived to look intuitive.

Maybe the person you replied to fixed a syntax error in an edit? The backticks are required, and are currently in their comment.

After looking more at fx that seems to be the case yes! It's 100% pure js syntax so shouldn't have any gotchas.

jq is YADSL though vs JS which is ubiquitous. Same reason most people can't use awk or sed.

I know JavaScript, so I immediately know a lot of the `fx` syntax.

I always found the `jq` syntax required many round-trips to the mail and more trial and error to get the syntax right.

I know JavaScript, so I immediately know a lot of the `fx` syntax.

I always found the `jqc

jq is probably better if you don't have node or deno installed since fx can't do expressions without it

Happy jq user as well.

But is jq interactive?

ijq [1] is. I am using it almost every time I need to come up with a jq expression. However, it gets noticeably laggy with enormous JSONs.

[1]: https://sr.ht/~gpanders/ijq/

Very cool. Not a huuge deal, but for open source software, I don't totally appreciate the http call to a bitly tracker in the install script...

Just use:

    go install github.com/antonmedv/fx@latest

Pretty cool! I actually wrote something VERY similar a couple of years ago: sless[1]. It's a tool for viewing json-based structured logs. Just like your tool, you can explore into a json object. The difference is, it expects the input to have many json objects, newline separated, and it shows few keys as a preview of the object, to make looking for something in the log easier. It's not quite complete but basic browsing works. It was mainly written to learn more about Urwid[2], a library similar to Curses.

1: https://github.com/dpedu2/sless

2: https://urwid.org/

The other day, I found myself in need of visualising a tree of paths, like this:

  /one/two
  /one/three/four
  /one/three/five
  /six

I ended up converting it to a JSON with a one-off Python script, and then using fx for the viewing. It worked very well! Thank you for the tool!

(Incidentally, I’ve been writing my own viewer [0] to satisfy my original need more straightforwardly, but that’s still in very early stages.)

[0]: https://github.com/nathell/treeviewer

As in filesystem paths? Are you aware of `tree`?

https://en.wikipedia.org/wiki/Tree_(command)

Not necessarily filesystem paths (in my case, the paths correspond to RocksDB keys). And I’d like to be able to fold/unfold subtrees, search within node names, and call arbitrary commands for a given path in the tree, from within a TUI. So no, `tree` won’t cut it.

This could be done with nushell.

https://www.nushell.sh/commands/docs/from_json.html

sure can do, if you already use that shell [1], but personally I like specific tools for specific jobs such as jq [2], fx, csvq [3] etc, there's value in decoupling shells from utils (modularity, speed, innovation etc).

[1] I don't but tempted to try, like its data-types concept

[2] https://jqlang.github.io/jq/

[3] https://github.com/mithrandie/csvq

Is the output ascii diagram only? Seems like a pretty different use case

No the output is a table object, you can apply other operations to it like sort, etc. the shell displays them in this ascii art way at the end but you can also write to files in various formats.

Any comparison to jless?

fx looks to work from the command line (without viewing) to do more than just inspect, looks like you can do pretty much anything to it with a bit of JS, though not from the TUI afaict

There are many JSON viewers, I would think that inspecting with collapsable sections ought to be right in VS Code with LSP/treesitter, no extra tools needed.

What's more interesting to me is what can you do with the JSON in these tools, validate, filter, transform, chain...?

I find `gron` to be very helpful for generating jq query strings

https://github.com/tomnomnom/gron

> Built using the Go programming language for efficient performance.

Do you mean to say "efficiency" or "performance"? I dont see both working like that

It's correct as it is. Performance can be good or bad. "...for efficiency" would be correct, but if they were to write only "...for performance" it would missing a qualifier.

FYI: .wtf top domain is blocked in HaGeZi's The World's Most Abused TLDs: https://raw.githubusercontent.com/hagezi/dns-blocklists/main...

Might wanna change to something else.

This is ridiculous. I will not change my domain because of someone’s blocklist.

Used to use HaGeZi's Blocklist but kept hitting false positives like this.

Need to have a bit more of a play with it but I really like it so far. I mainly see myself using it to pipe into (like from curl or the gh cli).

One thing that isn't clear to me is how to actually use the themes you can display with fx --themes.

Will update the docs. But it is easy:

    FX_THEME=2 fx data.json

Something that would be extra awesome would be light markdown support inside fields. This would make it useful for reviewing chatbot conversations.

Looks great!

What is the website https://fx.wtf/ built-in?

Vue

it is written in Go. What does it have to do with npx/npm at all? Did I miss something about these tools?

Where do you see npm/npx mentioned?

The project supports JS scripts as arguments to process the data:

https://fx.wtf/getting-started#json-processing

It's hard to tell from the docs if there is any JS involved if you just pipe JSON data to fx. I would hope not, since it mentions being written in Go for performance.

This is what I was wondering as well. Is there an interpreter built into the go binary? or is it using my node install under the hood?

Edit: Answered my own question. I installed it into a podman container with nothing else in it, and it printed this when trying to execute js:

    Node.js or Deno is required to run fx with reducers.

strace (on Linux) would have told you that without needing to user podman

at https://fx.wtf/getting-started it says:

$ cat file.json | npx fx .field

It seems it would work since they release an npm package too: https://www.npmjs.com/package/fx

A bit surprising, I wonder if they implemented it in JS too, or somehow managed to distribute their Go program on npm.

Edit: They indeed have a separate non-interactive version in JS: https://github.com/antonmedv/fx/tree/master/npm

It is nice for PCs, but requiring Node.js limits its use on servers.

This looks really nice.

visidata anyone?

This fx rewrite is very exciting. I'll have to try it. I thought of fx as a wrapper around jq, that allowed quick iteration over building jq scripts. Sort of an Ultimate Plumber [1] but only for jq. It looks like it is now more like a JavaScript processor plus an interactive viewer.

Someone mention Visidata[2]? VisiData is also a TUI that is great on tabular data, and it can work with json. If your JSON is mostly tabular in nature, Visidata does a great job at showing that data and allowing you to explore it. A lot of json I deal with is tabular-like data. There is a great tutorial [3], that can help you get your bearings with Visidata. Once you understand those basics you might want to look at this thread [4] for what commands you can use with json.

[1] Ultimate Plumber: https://github.com/akavel/up

[2] Visidata: https://github.com/saulpw/visidata

[3] "Intro to VisiData Tutorial": https://jsvine.github.io/intro-to-visidata/ by Jeremy Singer-Vine

[4] "Is it possible to "flatten" structured data (like JSON?)": https://github.com/saulpw/visidata/discussions/1605

amazing! Does someone know something like this for yaml? (does not have to be cli)

could've used this a few years ago before jq became so second nature, cool