UI: DVC's general output format

[skshetry]

So, I am starting a discussion for having a general guideline for DVC's output. There are two schools of thoughts:

1. Human readable first, optional flags for machine-readable, and
2. Machine-readable as well as human-readable (aka grep-parseable).

It feels like we try to opt for 2, but a few commands seem to be going for human-readable first:

1. diff: cannot grep by state or path
2. status: ugly and cannot grep easily
3. exp show: Non-standard table format.
4. metrics show: Cannot grep by keys easily, etc.

Making outputs human-readable by default helps us improve and experiment with different output formats. Machine-readable formats could be through some flags (--plain, --show-json, etc.). Though, you lose the ability to grep things by default. (Humans first, Machine second).

On the other hand, grep-parseable means that the output will be grepable, so machine readable. And, we should also try to make it human-readable. We lose the ability to change output format here though, every change in the output format is going to backward-incompatible and we won't be able to improve/experiment much. (Machine first, human second?)

If I take the example of git, Git seems to be also mixed, but generally it seems to follow 1) and tries to provide more hints and current state.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   content/docs/command-reference/dag.md

no changes added to commit (use "git add" and/or "git commit -a")

Git instead provides a --short it parseable.

$ git status --short
 M content/docs/command-reference/dag.md

On the other hand, heroku's cli opts for being grep-parseable by default and provides a --json additionally.

$ heroku regions
ID         Location                 Runtime
─────────  ───────────────────────  ──────────────
eu         Europe                   Common Runtime
us         United States            Common Runtime
frankfurt  Frankfurt, Germany       Private Spaces
oregon     Oregon, United States    Private Spaces
sydney     Sydney, Australia        Private Spaces
tokyo      Tokyo, Japan             Private Spaces
virginia   Virginia, United States  Private Spaces

I'd prefer 1, as it is flexible for us, but I do see benefits of 2 for DVC. In the end, what matters is consistency (or, consistency with certain exceptions). Let's discuss and try to fix output formats and try to make it consistent.

@dberenbaum @shcheklein @dmpetrov

[dberenbaum]

Thanks, @skshetry!

What about this:

1. If it's easy to satisfy both without sacrifices, make it human readable and greppable (like heroku regions).
2. If it sacrifices readability, prefer human readable (like git status).
3. If there's a proposed usage where piping the output is likely, include a --short flag if needed (like git status --short).

I'd consider machine-readable and greppable (pipeable?) separately, since json for example may not actually be easy to parse with grep/awk/etc.

[skshetry]

@dberenbaum, I guess what you are saying is that we should prioritize human readability first (I'd prefer guidelines to be simpler). PTAL at first 4 points of the Output guideline, maybe we can adopt them here. It essentially says what you are saying.

[dberenbaum]

Hah, makes me feel good that I was so close to the guide's recommendation! Yes, my vote is to adopt what the guide says.