Postgres compression of jsonb

Postgres supports JSON data since loong time ago, but until jsonb (9.4) all solutions were somewhat limited in features.  Jsonb gives you most of what a document storage engine would give you, but...  what about performance? and space efficiency? Unfortunately I don't have real answers to those questions, but I've started doing some research, and for now, here's a nice takeaway:

First of all you should be aware of "\dt+", which gives us the size of a table and "\d+" which describes a table. The "+" suffix adds more info to the descriptions.

CREATE TABLE foo AS SELECT '{"f":true}'::jsonb FROM generate_series(1,1e6);

CREATE TABLE bar AS SELECT '{"f":true}'::text FROM generate_series(1,1e6);

\dt+ foo;

\dt+ bar;  

We see that the text field takes about half of the size of the jsonb. So if you don't need json features in a field (you're only archiving data that happens to be json), think twice when giving it the jsonb type.


Jsonb is TOAST-able, but will probably be toasted  (and compressed) when it's bigger than a page, so don't count on that if you're jsons are <4kb .="" p="">
References:

• https://dba.stackexchange.com/questions/161864/do-long-names-for-jsonb-keys-use-more-storage
• https://stackoverflow.com/questions/23120072/how-to-check-if-toast-is-working-on-a-particular-table-in-postgres
• https://postgrespro.com/list/thread-id/1849114

So, don't think spacewise it comes for free. If you're concerned about space, there's a big impact on the size of the table.

docker run top & docker exec

Here's how I test docker things:

docker container run -t ubuntu top

Then you have this top window  that shows only a single process.

In a different terminal you can then do things like "docker container exec 123456789abcdef sleep 10", and see it in the top window. It's a stupid thing, but I need instant feedback to solidify my knowledge of what's happening where.

Remapping Keys "Up to Eleven"

Remapping keys on your linux is a "normal" thing to do amongst hardcore vim or emacs users.

The most standard hack is to remap CapsLock to Escape or to Ctrl.

And with emacs' key-chord and key-seq plugins, you can have really fancy combos to do the most obvious save-buffer, m-x, switch-to-buffer...

But I just discovered (via this reddit thread) that there's this "xcape" thing that allows you to bind keys to different keycodes depending on whether a key is pressed and released on its own or it's pressed as a modifier key along with other keys.

Here's how to "Remap left and right shift, when pressed alone, to left and right parens on keyup": xcape -t 250 -e "Shift\_L=parenleft;Shift\_R=parenright" &


That's super cool. And the readme of the project has some hardcore hacks for even more crazy conditional remappings.

As a side note, but kinda related, here's a great recent article about all kinds of glue (and duct tape) for X windows. And here's another awesome post about wizardy linux usages. The guy uses dmenu as an interface to a barebones plumber and gave me an idea to improve on my home made plumber (be smart and let you trim down witn dmenu when there are multiple options)

docker-compose, multiple yml files and environment variables

Did you know that docker-compose can merge multiple yml files into one? it seems like a very cool way to override configs and have something like a linear composability.

Just for fun, try

And do a few tests using

• docker-compose -f docker-compose.yml config
• docker-compose -f docker-compose.yml -f docker-compose-over.yml config
• FOO=1 docker-compose -f docker-compose.yml config
• FOO=1 docker-compose -f docker-compose.yml -f docker-compose-over.yml config
• docker-compose -f docker-compose.yml up
• docker-compose -f docker-compose.yml -f docker-compose-over.yml up
• FOO=1 docker-compose -f docker-compose.yml up
• FOO=1 docker-compose -f docker-compose.yml -f docker-compose-over.yml up

In addition to this, you can use the .env file to add defaults to the variables, and you can take a look at a special .override.yml suffix in https://docs.docker.com/compose/extends/. Plenty of ways to configure, reconfigure, and override variables. The difficult part is to make it understandable at all for the next guy touching your setups.

Also, in your app, you can have defaults for the env variables (docker will set them to empty strings if you don't set a value for them).

Also, remember that "docker-compose run" and "docker-compose exec" accept -e to set environment variables. I still haven't tried all the possibilities with those to see exactly what do they overwrite and what is fixed. 

TIL: search-whitespace-regexp

Here's another little dwim emacs' feature that I didn't know it existed and I just noticed today when it didn't dwim'ed.

When searching for a string "foo     bar", multiple whitespaces are ignored, and strings with one whitespace between the 2 words is detected as a match anyway.

Here are some ways to fix it, or bypass it in a particular occasion.

A strange case of quoting shell vars

cat << ${FOO:-"{}"}
EOF


You got that? evaluate it in your shell. Now Put it into a file and try to "sh file.sh".  You're going to see the funny {"}.   Go figure...

1. sh doesn't understand about nested {} inside {}
2. Take into account that bash or zsh do, so none of this applies there, you have to test it on pure sh.
3. ${FOO:-"{} is what gets interpreted as part of the variable
4. as $FOO is not defined, '"{' should be outputted,
5. leaving the final "{"}
6. But nope. Because quotes inside a variable expansion do not pass through
7. So the final {"}
8. Bonus points for this not happening in bash
9. More bonus points for this being inside a makefile that calls the file as $(SHELL) file.sh. And different platforms use different $(SHELL) by default.
10. Mine (ubuntu) uses /bin/sh (which is linked to /bin/dash), others might use busybox, and MacOS uses something that does not do any of the above (zsh?)

Linux extended file attributes

I just discovered a couple of nice linux commands that seem to be quite orthogonal and suckless, and that probably allow for all sorts of nice tricks.

Extended attributes are like the -rwxrwxrwx but allow for more subtle meanings.

On the one hand we have chattr, which allows to add about 10 different attributes to files and directories with a very specific meaning:

• a - append only
• c - compressed
• d - no dump
• e - extent format
• i - immutable
• j - data journaling
• s - secure deletion
• t - no tail-merging
• u - undeletable
• A - no atime updates
• D - synchronous directory updates
• S - synchronous updates
• T - top of directory hierarchy

On the other hand, we have the command 'attr' (not nstalled by default in ubuntu, but it's just one apt-get away).

This one serves as a kind of general tag:value system for files and directories.  It feels like it's a feature that can be overused to extremes to create not only tagging systems but acls (there's already getfacl), complex permissions or just hide info in this kind of "plists" scoped by file.