Naive option picker (fzf) in pure shell

 I've been a big fan of option pickers since.... forever. I've used ratmenu, ratmen, dmenu, dzen, percol, fzf, helm, vertico, consul...

For me, it's so fundamental that I need this functionality in every piece of code/script I write.

Finally I got to a working version of a very small version of it.

As always, pure shell, but with some advanced features can give you a succint function that works predictably.  The /dev/tty part was the trickiest to get working, as I didn't know how to get input from a pipe, and at the same time be able to read data from the keyboard.

jq assignment of multiple fields

Jq has become the de-facto json console tool. Apart from slicing and dicing your jsons, one can do things like assigning new fields to it, or modifying existing ones.

As I explain in my scripting field guide, changing epoch times to a readable timestamp can be done like:

echo '{"date":1643192480}' | jq '.date|=todateiso8601' #  {"date": "2022-01-26T10:22:48Z"}

 

To assign multiple fields in jq, you must interleave '|' inbetween, effectively piping the result to the next assignment.

And here's the bash snippet of the day:

join_by() { local IFS="$1"; shift; echo "$*"; }
pretty_dates() {
  jq "$(join_by "|" ${@/%/|=todateiso8601})"
} 

`curl http://..... | pretty_dates .start_date .end_date` will build the proper assignments and use jq so that we get a nicely formatted json output.

Again, for small helpers like these, a bit of advanced bash can get you very very far.

 

The explanation of the weird line is: from the default (${@}) params as an array, append every element (/%/) with |=todateiso8601). That, joined by |. And that becomes the "filter" to jq.

Ratpoison news in 2022?

I've been using the same obscure window manager since about 2003. I remember using it in 'Fedora Core 2'(released in 2004).

Lately I've thought about moving to something that behaves better with Zoom, but I really couldn't find an alternative.

 

There's https://github.com/jcs/sdorfehs , which comes from ratpoison, and it's maintained by jcs. It has the latest ratpoison patches that didn't make it to the last stable ratpoison release, and a few new features.  Well.... it turns out one of the new features breaks my workflow, so for me, this is a "nope".

I also thought about using evilwm, which is a floating wm with ratpoison shortcuts (more or less). But I do like tiling.  So maybe the combination of ratpoison and evilwm (via tmpwm) would be the sweetest spot. I remember using that config years ago, but now you have to make sure to have the latest ratpoison built from git, because tmpwm breaks on multiple screens.

Something I discovered just this week is this youtube channel from a user called root_sti. The videos are great(great music), showing neat configurations and a million scripts and tunings, using dzen2, fzf, evilwm, ratpoison, and voidlinux. And all configs are here. Lots of shellscript there :).

Maybe using sxhkd and emulating the shortcuts via wmctl... not sure

Also, a funny video just from a few minutes ago in hackernews. The first tiling window manager (1988)

A better watch: viddy

I just found this thread in HN, and I think I'm sold already on aliasing watch to this improved version of the watch command. https://news.ycombinator.com/item?id=31829343

 

Also, nice trick of aliases, where

A trailing space in value causes the next word
        to be checked for alias substitution when the alias is expanded

Read-only psql

This is how I wrote a short snippet to open read-only postgres connection using psql.  Notice the amount of non-trivial stuff in those lines (technically, only one line of code).

 

I tried to show many concepts in those short snippets I post around here. 

• For example, notice how some lines need backslashes to continue the line, but some others not. 
• Also notice how to use <() to create a temporary file, so we don't have to have a duplicate file (probably outdated). 
• The command being a "cat originalfile && echo "customline") makes it a fresh file each time, that inherits from your originalfile. 
• Finally, good old "$@" to proxy the parameters to another command.

in zsh, you can add "compdef ropsql=psql" so it also proxies the autocompletion

awk is cool (again)

Awk is lately appearing in HN more often.  I quite like awk, but having gone through Perl, bash, awk, I see people praise awk for being a POSIX compliant enhanced scripting language.  And while being true, I don't think this matters much to the commenters. They see it as an improved bash. And I think we're repeating the history.

Shortcomings that people see in bash, or awk, are solved in Perl. But people dismiss Perl for some reason, and they (we) are going to rediscover it in different forms. Perl is so optimized for scripting that it's really hard to beat. And it has a awk-like mode, so you can easily do the "/pattern/ {action}" thing.

 

Still, here are some threads on awk, with mentions to Perl.  I specially liked K (from K&R) awk.help file. That guy knows how to write!

https://news.ycombinator.com/item?id=28707463

https://earthly.dev/blog/awk-examples/

https://www.cs.princeton.edu/courses/archive/spring19/cos333/awk.help

 https://ferd.ca/awk-in-20-minutes.html

https://www.gnu.org/software/gawk/manual/gawk.pdf

http://www.cs.unibo.it/~sacerdot/doc/awk/nawkA4.pdf

https://ia803404.us.archive.org/0/items/pdfy-MgN0H1joIoDVoIC7/The_AWK_Programming_Language.pdf

Shell Tricks: ripgrep -U and $(!!)

Because learning the tools is an ongoing task, just two simple helpers for your daily terminal usage:

Finding matches in two consecutive lines. I used to do that in Perl, using multiline matches, but it involves remembering the Perl one-liner incantations....

 Well, I found that rg -U enables multiline matches, so you can do stuff like:

rg -U 'doseq.*\n.*deftest' test

To find tests wrapped in doseq.

Another cool shell trick I discovered by myself was $(!!).  

I have this helper function that greps my $HISTFILE, to find old usages of commands. I use it like ctrl-r, sort of:

rg --no-filename "$@" ~/.zhistory

I use it sometimes with fzf, and many times, when I select the commad, I realize I want to run it.  Well, there are more optimal ways to do it (fzf can run the command), but I liked discovering $(!!), that reruns your last command (the rg ..|fzf), and will run its output.

Small little grep+edit trick

This is so small, but so huge....

It's like many other finders you find in your editor of choice, but because it's such a generic and minimalistic tool, I keep up using it tenths of times a week.

Idk, this is more composable and functional and succinct than many things I find in the wild that claim to be so. 

And it is damn useful!

Also, even it's 4 lines of code, there's probably a thing or two you can probably learn from those. Here are some things I like and why I think it's High Quality Shit:

- It's short, you can inspect the words  it contains, and if you get more or less the programs it calls, it can only do one thing.

- It solves a real problem

- It has high density. All words there mean something, there's 0 boilerplate

- It is composed of smaller parts. It feels Forth-y

- The way it's composed is subtly nice (the existence of "e" function is needed to compose it that way)

- It's as configurable as the program it uses to grep. It inherits its flags

- It inherits its auto-completions

- You can learn a bunch of things from it

- It fits in your head

- It has that Iversonian "suggestivity"

- I wrote it, so it has that IKEA effect on me

Many more things like that in https://raimonster.com/scripting-field-guide/

Oh! I found https://github.com/junegunn/fzf/blob/master/ADVANCED.md#ripgrep-integration which is an even more advanced version of it. Not as cute though

A couple of jq tricks

 There's 2 jq functions I'm using a lot, and I don't see many people using: "join" and "fromjson".

The first one, is join, which joins arrays into a single string. I shared it in a HN thread about jq. That thread has some nice tricks, check it out if you do that kind of json parsing in the CLI.

The other one is "fromjson", and you use it like this:  

     echo '"{\"a\":1}"' | jq fromjson.a      # 1

A concrete practical example is in when fetching secrets from aws secretsmanager. A secret's content is a string, that is really a json.

     aws secretsmanager get-secret-value --secret-id  <secret-key> | jq '.SecretString| fromjson.key_of_the_json' -r

Taming Zoom with Ratpoison, part 2

 It's been already 2 years since the last zoom hack, and it worked fine for some time, but since January 2021, Zoom was behaving even worse on my ratpoison setup. Hijacks of popup windows left me without being able to join zoom with any sort of confidence.

A temporary solution is to use ratpoison's tmpwm, but there's another wrinkle there. In the latest official ratpoison release (1.4.9, from 2017), there's a bug that makes ratpoison not be able to go back from a tmpwm if you have multiple screens. It is fixed in the git version, but having to compile your own WM is yet another yak to shave. (Even more when using nixos). 

So I spoke to a coworker who's a StumpWM user and asked if stump had those issues.

So, it turns his hack worked, and it consists of changing the .config/zoomus.conf. Changing config files on proprietary applications reminds me of my reverse-engineer times on Windows 98.

Anyway:

sed -i 's/enableMiniWindow=true/enableMiniWindow=false/' ~/.config/zoomus.conf

Thanks, Tim and good luck!

Fast feedback loop with emacs (Lua)

Here's a long overdue post about emacs fiddling. All this belongs to my previous job time, during the past 3 years doing Lua.

One of the major test frameworks in Lua is Busted. It uses an rspec-inspired syntax and the cli accepts flags to filter the tests you want to run, by file and regular expression.

To iterate as fast as I could, I'd want to lock on a particular test, and be able to run that same test for a while. That's super useful while hacking on a feature or fixing a single test.

So the solution I came with was to use our beloved emacs, to find the current test the cursor is on, and remember it in a global variable. Of course, I want it in the most dwim-y way, so the same binding does "the right thing" most of the times.

Here's the decision making proces, that starts calling rgc-run-test (bound to f7):

rgc-run-test

If I'm in a *_spec.lua file, the process sets a global var with the current file path. The process also looks for the closest previous line to my cursor that looks like a test definition and sets another variable with the string of the test. To recognize the test definition, I match the lines against "it(", "describe(", or "it_content_types(". I use 'rx' library for that, which is pretty cool, check that out if you're doing complex regexes in elisp.

rgc-test-shell

After that, we find a buffer with a shell.

If there's none currently open, I create one, using the right parameters (so it lands on the correct directory).

An extra nicety is that I call 'highlight-regexp' with a string that will highlight any debug line.

You can enable (add-hook 'shell-mode-hook 'compilation-shell-minor-mode), making all shell buffers to run compilation-shell-minor-mode. That means that every line that looks like a path/to/source.file:line becomes clickable. That means that you can navigate to the source:line from a stacktrace.

With this, you can press F7 in a test file, and this snippet will make sure that a shell is in the right place (opening an existing one if it's already there) and will run the current test. 

But, many times, you are not just editing the test, but you're touching the code. No problem. F7 remembers the latest test you run when you were in a test file, so it will run that same test in case you're not in a test file.

When things seem to be solved, you then want to run the tests in that same file, but not restricting it to that single test, but run the whole file, to make sure you didn't mess up anything else. C-u F7 will do just that.

rgc-test-flags

Last cool thing. Sometimes I want to lock extra flags for the tests, rgc-test-flags is a quick way to overload the flags and keep them around for the next runs also.

ffap

Also, sometimes, the test errors in lua are marked like 'path/to/file @ 98'.  This makes find-file-at-point miss the line number.  But yeah, emacs. you defadvice find-file-at-point, with an extra case and off you go.

 That's it

I know this whole thing is quite hackish, and there's a lot to chew in just 50 lines, but this was so useful to me that I wanted to share before I forgot about it (changed job recently so I'm not using this snippet anymore)

It's this kind of workflows that the holistic approach of emacs allow for. And we love it so much :)

Here's the gist of it:

 

Interesting new-old takes on git

It seems there are a few things going on lately on the git subspace.

For one, ppl are talking about the mail flow vs Pull Request flow. It's interesting how people go back and forth on those: https://blog.brixit.nl/git-email-flow-versus-github-flow/

 Here's an interesting take at git flows again. Git plan focuses on writing the commit message before doing the commits. Maybe it's something to fix the issue/pr dicothomy?: https://github.com/synek/git-plan

And an experience report of using squash+merge for couple of years. That's interesting because it's what we are using at work, and even though I'm skeptical about its advantages, it's nice to see how people perceive the effects of squash+merge: https://blog.dnsimple.com/2019/01/two-years-of-squash-merge/

And a bonus of HN thread with some git helpers using fzf: https://news.ycombinator.com/item?id=26634419

a few cli tools

 New job, new tooling, and reinventing the same trick again and again :)

This time, a few things on console stuff:

• https://github.com/okbob/pspg
• https://github.com/rcoh/angle-grinder
• https://github.com/jpmens/jo
  

Shell Oneliner Compression

I'm back to Lisp. Now, Clojure. For real. But this post is about zsh

Part of my relearning of clojure is to read and watch everything clojure that crosses my path. And I discovered https://github.com/clojure-cookbook/clojure-cookbook , which seems super useful.  But I have too many open tabs and books already. So I thought I'd do a 'Tip Of The Day' like thing, and I'd pop a different page every day.

 Relevant files have .asciidoc  extension. And the ones with content start with a number. The other ones are index pages or headers/footers.

Here's a version of what I typed. and it worked (at the second try!)


random-clojure() {
   e $(ls ~/clojure-cookbook/**/*asciidoc(.) | awk -F/ '$NF ~ /^[0-9]/' | shuf -n1 | xargs realpath)
}

Quite a lot of things going on there, so I'm gonna explain the oneliner, because there's a lot of compression there.

• e. e is my shellscript that opens files in emacs, moving to the line, and massaging input. in this case, it's the same as emacs, vi, $EDITOR.
• ~/clojure-cookbook/**/*asciidoc(.). This one expands recursively files that contain 'asciidoc'. excluding directories. '**/' means 'recursively' in zsh, and the '(.)' globbing is for files only.
• awk. '-F/' makes slash a field separator. Then, we pick the latest field, and we regex match it with '/^[0-9]/', beginning with a number. Cool!
• shuf -n1 picks a line at random
• xargs realpath. As we'll be running this function from all kinds of directories, expand the file path to an absolute path. as realpath asks for a parameter, we either nest the whole thing or use the xargs trick.

 

While writing this, I noticed that the whole thing is much simpler than that:

 e $(ls ~/clojure-cookbook/**/[0-9]*asciidoc(.)|shuf -n1| xargs realpath)

A pipe inspector with tee

I've kept writing my scripting-field-guide, adding some more common pitfalls and cool tricks I've used in the last years. 

Something I started using not long time ago is the "tee >()" incantation. It's super cool to be able to branch off the output of a command to another command, and make some sort of tree. 

For now, there's this snippet I created to be able to inspect the contents of a pipe:

The default to the current timestamp isn't going to really work, as the string is evaluated only once, but I hope I'll get something more decent soon.

Another cool usage is when selecting something from a grep/fzf/dmenu and you want to do something with it like open that file, but also copy it to your clipboard.

The clearest example is my daily-zoom-selector, where I'm not just opening the room, but also copying it.

export all variables in bash

If you have a prefix, the coolest way is like the following:

export ${!KONG_*}

If you want everything:

eval $(printenv | awk -F= '{ print "export " $1 }')

pipes on steroids

I though I had blogged about that before, but I can't find it anywhere, so I'm just gonna put it here (again?).

Pipes are great. You know that, I know that, everyone knows that. Because/But they are restricted to linear, non conditional flow.

Sometimes, I'd like to have an out-of-band pipe that bypasses a command in the middle, and there's no clear way how to do it.

So here are a few links on how to use file descriptors for advanced use cases. You can use them for this, and for other smart stuff in shells.

• https://catonmat.net/bash-one-liners-explained-part-three
• https://wiki.bash-hackers.org/howto/redirection_tutorial
• http://tldp.org/LDP/abs/html/ioredirintro.html
• http://catern.com/posts/pipes.html
• https://mosermichael.github.io/jq-illustrated/dir/content.html
• https://news.ycombinator.com/item?id=21700014 ( https://www2.dmst.aueb.gr/dds/sw/dgsh/ )
• http://dongyuxuan.me/posts/pipeline.html 
• https://stackoverflow.com/questions/2990414/echo-that-outputs-to-stderr
• http://www.tldp.org/LDP/abs/html/io-redirection.html
• http://wiki.bash-hackers.org/howto/redirection_tutorial 
• https://news.ycombinator.com/item?id=22704774

disable screen blank/sleep on idle

I've finally switched to more barebones linux distros again. Now I'm on NixOS and Void.

Void is minimalist and lightweight, with a textual installation process, like vectorlinux had (using text dialogs and menus).

I just found out that the way to keep the screen from going blank after some mouse/keyboard idle time is  xset s off -dpms. That saves you from any ad-hoc command line fu when watching movies with that girl.

Do not delete tmux dead panes

I've been using tmux for about a year. tmux-fingers and the feature that I helped add of instant pasting was what triggered the move.

Even I can't exactly replicate my screen workflow in tmux, I'm more than happy with the tradeoff.

Here's a nice option I didn't know about tmux: "remain-on-exit"

By default, tmux (like screen) kills the panes (or frames, or windows, or however they are called) when the process inside them dies. And usually, that's what you want.

But there's a case when you probably don't want this: "parallell --tmux"

I use GNU parallel as much as I can. I find it an awesome tool. Very hacker friendly and composable with everything you're already doing.

So I was recently using it to build packages for multiple distros. The command was something like  "parallel ./packager.sh --tmux {} ::: alpine ubuntu".  But when the processes finish (both successfully or not), the pane disappears, and doesn't let you review and debug the outputs.

Setting "set-option -g remain-on-exit on" on tmux, leaves every pane opened for your inspection.

set -e in bash subshell

Did you ever realize that even when you use `set -e` in a bash script, anything that happens inside a  $(subshell like that) won't be executed under the `set -e` umbrella?


Here are the 2 solutions we found. None feels very solid... but hey, it's bash, what did you expect?

- https://lists.gnu.org/archive/html/bug-bash/2008-01/msg00121.html . Apparently traps work in this dynamic scope and they get access to the nesting level of the bash shell the code is called from. Kind of not unwinding the stack?

- https://unix.stackexchange.com/questions/65532/why-does-set-e-not-work-inside-subshells-with-parenthesis-followed-by-an-or .  running the subshell as a background task and immediately waiting for it, like a 1process forkjoin.  Nuts.