We have a folder with more than a million files that needs sorting, but we can’t really do anything because mv outputs this message all the time
[pastacode lang=”bash” manual=”bash%3A%20%2Fbin%2Fmv%3A%20Argument%20list%20too%20long%0A” message=”bash code” highlight=”” provider=”manual”/]we using this command to move extension-less files:
[pastacode lang=”bash” manual=”mv%20–%20!(*.jpg%7C*.png%7C*.bmp)%20targetdir%2F%0A” message=”bash code” highlight=”” provider=”manual”/]- xargs is the tool for the job. That, or find with -exec … {} +. These tools run a command several times, with as many arguments as can be passed in one go.
- Both methods are easier to carry out when the variable argument list is at the end, which isn’t the case here: the final argument to mv is the destination.
- With GNU utilities (i.e. on non-embedded Linux or Cygwin), the -t option to mv is useful, to pass the destination first.
- If the file names have no whitespace nor any of \”‘, then you can simply provide the file names as input to xargs (the echo command is a bash built in, so it isn’t subject to the command line length limit):
- You can use the -0 option to xargs to use null-delimited input instead of the default quoted format.
- Alternatively, you can generate the list of file names with find. To avoid recursing into subdirectories, use -type d -prune. Since no action is specified for the listed image files, only the other files are moved.
- If you don’t have GNU utilities, you can use an intermediate shell to get the arguments in the right order. This method works on all POSIX systems.
- In zsh, you can load the mv builtin:
or if you prefer to let mv and other names keep referring to the external commands:
[pastacode lang=”bash” manual=”setopt%20extended_glob%0Azmodload%20-Fm%20zsh%2Ffiles%20b%3Azf_%5C*%0Azf_mv%20–%20%5E*.(jpg%7Cpng%7Cbmp)%20targetdir%2F%0A” message=”bash code” highlight=”” provider=”manual”/]or with ksh-style globs:
[pastacode lang=”bash” manual=”setopt%20ksh_glob%0Azmodload%20-Fm%20zsh%2Ffiles%20b%3Azf_%5C*%0Azf_mv%20–%20!(*.jpg%7C*.png%7C*.bmp)%20targetdir%2F%0A” message=”bash code” highlight=”” provider=”manual”/]Alternatively, using GNU mv and zargs:
[pastacode lang=”bash” manual=”autoload%20-U%20zargs%0Asetopt%20extended_glob%0Azargs%20–%20.%2F%5E*.(jpg%7Cpng%7Cbmp)%20–%20mv%20-t%20targetdir%2F%0A” message=”bash code” highlight=”” provider=”manual”/]- The operating system’s argument passing limit does not apply to expansions which happen within the shell interpreter.
- So in addition to using xargs or find, we can simply use a shell loop to break up the processing into individual mv commands:
- This uses only POSIX Shell Command Language features and utilities. This one-liner is clearer with indentation, with unnecessary semicolons removed:
- A solution without a catch block using “$origin”/!(*.jpg|*.png|*.bmp):
- For a multi-line script you can do the following (notice the ; before the done is dropped):
- To do a more generalized solution that moves all files, you can do the one-liner:
Which looks like this if you do indentation:
[pastacode lang=”bash” manual=”for%20file%20in%20%22%24origin%22%2F*%3B%20do%0A%20%20%20%20mv%20–%20%22%24file%22%20%22%24destination%22%0Adone%20%0A” message=”bash code” highlight=”” provider=”manual”/]- This takes every file in the origin and moves them one by one to the destination. The quotes around $file are necessary in case there are spaces or other special characters in the filenames.
- Here is an example of this method that worked perfectly