{"id":3460,"date":"2017-04-02T13:47:46","date_gmt":"2017-04-02T08:17:46","guid":{"rendered":"https:\/\/www.wikitechy.com\/technology\/?p=3460"},"modified":"2017-04-02T13:47:46","modified_gmt":"2017-04-02T08:17:46","slug":"solving-mv-argument-list-long","status":"publish","type":"post","link":"https:\/\/www.wikitechy.com\/technology\/solving-mv-argument-list-long\/","title":{"rendered":"[ Solved -3 Answers] LINUX &#8211; Solving mv : Argument list too long"},"content":{"rendered":"<p><label class=\"label label-warning\">PROBLEM:<\/label><\/p>\n<p>We have a folder with more than a million files that needs sorting, but we can\u2019t really do anything because mv outputs this message all the time<\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dbash%3A%20%2Fbin%2Fmv%3A%20Argument%20list%20too%20long%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p>we using this command to move extension-less files:<\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dmv%20\u2013%20!(*.jpg%7C*.png%7C*.bmp)%20targetdir%2F%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p><label class=\"label label-info\">SOLUTION 1:<\/label><\/p>\n<ul>\n<li>xargs is the tool for the job. That, or find with -exec \u2026 {} +. These tools run a command several times, with as many arguments as can be passed in one go.<\/li>\n<li>Both methods are easier to carry out when the variable argument list is at the end, which isn\u2019t the case here: the final argument to mv is the destination.<\/li>\n<li>With GNU utilities (i.e. on non-embedded Linux or Cygwin), the -t option to mv is useful, to pass the destination first.<\/li>\n<li>If the file names have no whitespace nor any of \\\u201d\u2018, then you can simply provide the file names as input to xargs (the echo command is a bash built in, so it isn\u2019t subject to the command line length limit):<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201decho%20!(*.jpg%7C*.png%7C*.bmp)%20%7C%20xargs%20mv%20-t%20targetdir%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n[ad type=\u201dbanner\u201d]\n<ul>\n<li>You can use the -0 option to xargs to use null-delimited input instead of the default quoted format.<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dprintf%20\u2019%25s%5C0\u2019%20!(*.jpg%7C*.png%7C*.bmp)%20%7C%20xargs%20-0%20mv%20-t%20targetdir%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p>\u00a0<\/p>\n<ul>\n<li>Alternatively, you can generate the list of file names with find. To avoid recursing into subdirectories, use -type d -prune. Since no action is specified for the listed image files, only the other files are moved.<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfind%20.%20-name%20.%20-o%20-type%20d%20-prune%20-o%20%5C%0A%20%20%20%20%20%20%20-name%20\u2019*.jpg\u2019%20-o%20-name%20\u2019*.png\u2019%20-o%20-name%20\u2019*.bmp\u2019%20-o%20%5C%0A%20%20%20%20%20%20%20-exec%20mv%20-t%20targetdir%2F%20%7B%7D%20%2B%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<ul>\n<li>If you don\u2019t have GNU utilities, you can use an intermediate shell to get the arguments in the right order. This method works on all POSIX systems.<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfind%20.%20-name%20.%20-o%20-type%20d%20-prune%20-o%20%5C%0A%20%20%20%20%20%20%20-name%20\u2019*.jpg\u2019%20-o%20-name%20\u2019*.png\u2019%20-o%20-name%20\u2019*.bmp\u2019%20-o%20%5C%0A%20%20%20%20%20%20%20-exec%20sh%20-c%20\u2019mv%20%22%24%40%22%20%22%240%22\u2019%20targetdir%2F%20%7B%7D%20%2B\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<ul>\n<li>In zsh, you can load the mv builtin:<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dsetopt%20extended_glob%0Azmodload%20zsh%2Ffiles%0Amv%20\u2013%20%5E*.(jpg%7Cpng%7Cbmp)%20targetdir%2F%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n[ad type=\u201dbanner\u201d]\n<p><b>or if you prefer to let <\/b><b>mv<\/b><b> and other names keep referring to the external commands:<\/b><\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dsetopt%20extended_glob%0Azmodload%20-Fm%20zsh%2Ffiles%20b%3Azf_%5C*%0Azf_mv%20\u2013%20%5E*.(jpg%7Cpng%7Cbmp)%20targetdir%2F%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p><b>or with <\/b><b>ksh<\/b><b>-style globs:<\/b><\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dsetopt%20ksh_glob%0Azmodload%20-Fm%20zsh%2Ffiles%20b%3Azf_%5C*%0Azf_mv%20\u2013%20!(*.jpg%7C*.png%7C*.bmp)%20targetdir%2F%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p><b>Alternatively, using <\/b><b>GNU mv <\/b><b>and <\/b><b>zargs<\/b><b>:<\/b><\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dautoload%20-U%20zargs%0Asetopt%20extended_glob%0Azargs%20\u2013%20.%2F%5E*.(jpg%7Cpng%7Cbmp)%20\u2013%20mv%20-t%20targetdir%2F%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p><label class=\"label label-info\">SOLUTION 2:<\/label><\/p>\n<ul>\n<li>The operating system\u2019s argument passing limit does not apply to expansions which happen within the shell interpreter.<\/li>\n<li>So in addition to using xargs or find, we can simply use a shell loop to break up the processing into individual mv commands:<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20x%20in%20*%3B%20do%20case%20%22%24x%22%20in%20*.jpg%7C*.png%7C*.bmp)%20%3B%3B%20*)%20mv%20\u2013%20%22%24x%22%20target%20%3B%3B%20esac%20%3B%20done%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n[ad type=\u201dbanner\u201d]\n<ul>\n<li>This uses only POSIX Shell Command Language features and utilities. This one-liner is clearer with indentation, with unnecessary semicolons removed:<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20x%20in%20*%3B%20do%0A%20%20case%20%22%24x%22%20in%0A%20%20%20%20*.jpg%7C*.png%7C*.bmp)%20%0A%20%20%20%20%20%20%20%3B%3B%20%23%20nothing%0A%20%20%20%20*)%20%23%20catch-all%20case%0A%20%20%20%20%20%20%20mv%20\u2013%20%22%24x%22%20target%0A%20%20%20%20%20%20%20%3B%3B%0A%20%20esac%0Adone%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p><label class=\"label label-info\">SOLUTION 3:<\/label><\/p>\n<ul>\n<li>A solution without a catch block using \u201c$origin\u201d\/!(*.jpg|*.png|*.bmp):<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20file%20in%20%22%24origin%22%2F!(*.jpg%7C*.png%7C*.bmp)%3B%20do%20mv%20\u2013%20%22%24file%22%20%22%24destination%22%20%3B%20done%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<ul>\n<li>For a multi-line script you can do the following (notice the ; before the done is dropped):<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20file%20in%20%22%24origin%22%2F!(*.jpg%7C*.png%7C*.bmp)%3B%20do%20%20%20%20%20%20%20%20%23%20don\u2019t%20copy%20types%20*.jpg%7C*.png%7C*.bmp%0A%20%20%20%20mv%20\u2013%20%22%24file%22%20%22%24destination%22%20%0Adone%20%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<ul>\n<li>To do a more generalized solution that moves all files, you can do the one-liner:<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20file%20in%20%22%24origin%22%2F*%3B%20do%20mv%20\u2013%20%22%24file%22%20%22%24destination%22%20%3B%20done%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n[ad type=\u201dbanner\u201d]\n<p>Which looks like this if you do indentation:<\/p>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20file%20in%20%22%24origin%22%2F*%3B%20do%0A%20%20%20%20mv%20\u2013%20%22%24file%22%20%22%24destination%22%0Adone%20%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<ul>\n<li>This takes every file in the origin and moves them one by one to the destination. The quotes around $file are necessary in case there are spaces or other special characters in the filenames.<\/li>\n<li>Here is an example of this method that worked perfectly<\/li>\n<\/ul>\n[pastacode lang=\u201dbash\u201d manual=\u201dfor%20file%20in%20%22%2FUsers%2Fwilliam%2FPictures%2Fexport_folder_111210%2F%22*.jpg%3B%20do%0A%20%20%20%20mv%20\u2013%20%22%24file%22%20%22%2FUsers%2Fwilliam%2FDesktop%2Fsouthland%2Flandingphotos%2F%22%3B%0Adone%0A\u201d message=\u201dbash code\u201d highlight=\u201d\u201d provider=\u201dmanual\u201d\/]\n<p>\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p> Solving mv : Argument list too long &#8211; Both methods are easier to carry out when the variable argument list is at the end<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1699],"tags":[6355,6354,6356,6362,6360,6357,6365,6359,6361,6363,6364,6358],"class_list":["post-3460","post","type-post","status-publish","format-standard","hentry","category-linux","tag-argument-list-too-long-ls","tag-argument-list-too-long-mv","tag-argument-list-too-long-python","tag-arugument-list-too-long-on-linux-system-python","tag-commonerrormessagesargumentlisttoolong","tag-cp-argument-list-too-long","tag-how-to-set-ulimit-values-red-hat-customer-portal","tag-move-a-large-number-of-files-in-linux","tag-resolved-binrm-argument-list-too-long-error","tag-ulimit-how-do-i-increase-the-open-files-limit-for-a-non-root-user","tag-ulimit-what-is-the-max-opened-files-limitation-on-linux","tag-unable-to-delete-file-argument-list-too-long"],"_links":{"self":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts\/3460","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/comments?post=3460"}],"version-history":[{"count":0,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/posts\/3460\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/media?parent=3460"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/categories?post=3460"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wikitechy.com\/technology\/wp-json\/wp\/v2\/tags?post=3460"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}