Jump to content


Process Substitution


  • Please log in to reply
3 replies to this topic

#1 OFFLINE   sunrat

sunrat

    Thread Kahuna

  • Forum Moderators
  • 5,967 posts

Posted 22 July 2019 - 08:52 PM

I just read this answer on Quora which is intriguing and could be rather useful. Thanks to the author Rafal Pocztarski.

Quote

   Process Substitution (Bash Reference Manual) https://www.gnu.org/...ss-Substitution


The syntax is:

some command <(other command)

You use it like this:

cat <(ls -alp)

What it does is it runs the command in parentheses and provides its output as a file (a named pipe or an already opened file descriptor with its name passed in the form of e.g. /dev/fd/20).

With the above cat <(ls -alp) example, the result will be the same as just running ls -alp without cat or as running ls -alp | cat but it works differently.

The cat command gets an additional file descriptor (by default it gets 0 for stdin, 1 for stdout and 2 for stderr) and a special file representing this file descriptor for this process in the /dev/fd directory is passed as an argument in this place where <(&hellip;) was used. The cat command could use this number directly or it can open the special file as a normal file which will just clone this file descriptor, so the command that gets that file name doesn&rsquo;t even have to know that this file is in any way unusual.

Why it&rsquo;s useful?

E.g. you can compare two outputs of some commands using diff but without saving it in files first. For example:

ls -lp > output1.txt
ls -alp > output2.txt
diff -u output1.txt output2.txt

this can be done as one command with no saved files:

diff -u <(ls -lp) <(ls -alp)

What arguments the diff command gets can be tested by running:

echo diff -u <(ls -lp) <(ls -alp)

The result is:

diff -u /dev/fd/63 /dev/fd/62

Note that you will not be able to read those files from outside of the diff process (unless the diff process passes those file descriptors using IPC) because every process has its own set of open file descriptors so for each process the same number under /dev/fd is a different file descriptor.

registered Linux user number 324659  ||    The importance of Reading The *Fine* Manual! :D
Posted ImagePosted ImagePosted ImagePosted Image
For the things we have to learn before we can do them, we learn by doing them.

#2 OFFLINE   securitybreach

securitybreach

    CLI Phreak

  • Forum Admins
  • 24,708 posts

Posted 23 July 2019 - 05:45 PM

Very neat stuff, thanks for posting. :thumbsup:

I've used the redirect (>) for years but never used it to compare multiple things with diff.

I mostly use it for things like if I want the output to a text file to parse or generating repo lists on whenever mirrorlist gets updated:
cd /etc/pacman.d/ && rankmirrors -n 7 mirrorlist.pacnew > mirrorlist && pacman -Syy

Note: && and || are operators that combine commands. Like && is run one command and if successful, run the next command. || runs each command regardless if the first one errored out or not. I rarely use the latter.

As far as the command above, I use reflector to automate this but that was just an example.
Posted ImagePosted Image
Configs/PGP Key/comhack π

"Do you begin to see, then, what kind of world we are creating? It is the exact opposite of the stupid hedonistic Utopias that the old reformers imagined. A world of fear and treachery and torment, a world of trampling and being trampled upon, a world which will grow not less but more merciless as it refines itself. Progress in our world will be progress toward more pain." -George Orwell, 1984

#3 OFFLINE   sunrat

sunrat

    Thread Kahuna

  • Forum Moderators
  • 5,967 posts

Posted 23 July 2019 - 08:13 PM

The redirect can go the other way too, as used in the above diff example. You can also redirect a text file back to a command. I've used it in the past to mirror the installed packages on one system to another.

Get the list of packages -
dpkg --get-selections > selections.txt

Then mark the list to be installed on the other system -
dpkg --set-selections < selections.txt

This command will not install any packages, but only mark a state corresponding to every package.
For the actual installation, run the following command
apt-get -u dselect-upgrade


Note this will set the new system to exactly the same state as the old system so will remove installed packages that are not on the list.
registered Linux user number 324659  ||    The importance of Reading The *Fine* Manual! :D
Posted ImagePosted ImagePosted ImagePosted Image
For the things we have to learn before we can do them, we learn by doing them.

#4 OFFLINE   securitybreach

securitybreach

    CLI Phreak

  • Forum Admins
  • 24,708 posts

Posted 23 July 2019 - 09:43 PM

I do the same on arch:
pacman -Qqe > pkg.lst
cat pkg.lst | xargs pacman -S --needed --noconfirm

Posted ImagePosted Image
Configs/PGP Key/comhack π

"Do you begin to see, then, what kind of world we are creating? It is the exact opposite of the stupid hedonistic Utopias that the old reformers imagined. A world of fear and treachery and torment, a world of trampling and being trampled upon, a world which will grow not less but more merciless as it refines itself. Progress in our world will be progress toward more pain." -George Orwell, 1984




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users