How to extract program names from command in bash?

Question

ruby

Suppose we have a command history file like this:

sudo ls  | awk '!($NF ~ /\.[a-z]+$/)'
for i in ~/*; do echo $(locate -c -r $i) $i; done | pv | sort -nr | bcat -t
for f in *.zip; do unzip $f; done
RACK_HANDLER=falcon rails s

You need to parse these lines so that you can extract the names of the programs used (preferably not built into the shell). From the example above:

sudo ls awk
pv sort bcat
unzip
rails

I found a simple Shell::Parser module in Perl that seems to be what I need. Is there a gem with similar functionality?

Answer 1

this can be quite easily implemented on the bash itself in a few lines, if ruby ​​is needed, then you can simply take as a starting point for your implementation
the list of all programs located in the $ PATH list, or rather the description of the builtin ( builtin ) commands in the bash shell can be look with the help command, but I think it would be more correct to use type to determine the type of command
$ ls ${PATH//:/ } |awk 'NF && !/:$/'
$ help

$ type -t rg
file
$ type -t cd
builtin

$ type --help
...
-t	output a single word which is one of `alias', `keyword',
  `function', `builtin', `file' or `', if NAME is an alias,
  shell reserved word, shell function, shell builtin, disk file,
  or not found, respectively

the file type is essentially purely external programs, and builtin is internal, there may be nuances at the expense of the rest, that is, when checking through type , we leave the current utilities that return file .
As a result, we can generate a list ( file.txt ) of purely external programs from those in $PATH :

$ for i in $(ls ${PATH//:/ } |awk 'NF && !/:$/');do  && echo $i;done > file.txt

Let's say we have a test.txt file with command history:

sudo ls  | awk '!($NF ~ /\.[a-z]+$/)'
for i in ~/*; do echo $(locate -c -r $i) $i; done | pv | sort -nr | bcat -t
for f in *.zip; do unzip $f; done
RACK_HANDLER=falcon rails s

then:

$ grep -o -w -n -f file.txt test.txt
1:sudo
1:awk
2:pv
2:sort
3:zip
3:unzip

the only drawback is that each match is written from a new line, we fix it with awk:

$ grep -o -w -n -f file.txt test.txt |awk -F: '{if($1!=i){printf $2" "}else{print $2};i=$1}'
sudo awk
pv sort
zip unzip

when compared to your template:

sudo ls awk
pv sort bcat
unzip
rails

then we see that there is no ls , this is due to the fact that, for example, I have this alias based on exa - an external program, although it could just as well have been an internal program, the same ls with selected keys, but it would also not be here displayed, in general, if aliases are included in the output, then in theory they will also need to be analyzed in a separate handler.
it is also natural that jambs will sometimes occur, for example, here with zip , when analyzing a string,
for f in *.zip; do unzip $f; done
zip does not act as a program here, but it is still a word that coincided with our list of utilities.
bcat and rails did not display because these programs are not installed.
in general, if briefly, then all this is implemented in two one-liners in bash:

$ for i in $(ls ${PATH//:/ } |awk 'NF && !/:$/');do  && echo $i;done > file.txt
$ grep -o -w -n -f file.txt test.txt |awk -F: '{if($1!=i){printf $2" "}else{print $2};i=$1}'