(One)-minute geek news

Mass renaming

It is common to generate or simply deal with a large amount of files. In such a case, organising your folders might be a tedious experience. Indeed, if renaming one file is quite easy in Bash (with the well-known mv command), mass renaming is often largely unknown.

Pure Bash approaches

The idea of mass renaming is applying a given string modification to the filename of multiple files. Example: rename all .xyz files in my ~/cp2k/test directory by adding the sub-string _old before the trailing .xyz.

This can be solved by a simple for loop: for file in ~/cp2k/test/*.xyz ; do mv $file ${file%.xyz$}_test.xyz ; done

Let us decompose this one-liner command:

• for file in ~/cp2k/test/*.xyz will simply iterate over all files whose path matches the pathname pattern ~/cp2k/test/*.xyz and assign it to the variable (called parameter in Bash) named file. Therefore, file will successively take for value the path of the selected files.
• do and done are the keywords surrounding the for loop execution block. Note the semi-colons before do and done, but not after do
• mv $file ${file%.xyz$}_test.xyz is the command that treats each individual file, so a mv command is enough here. $file refers to the value of the parameter called file (yes, Bash variables handling is unusual...). ${file%.xyz$} is a parameter substitution that has for value the $file string with the trailing .xyz removed (see the Parameter expansion section of the Bash manual)
• Bonus: Many parameter expansions exist: ${parameter/str1/str2} for example that has for value the $parameter string with the pattern str1 replaced by the string str2.
• Note: This code will break on filenames containing spaces. You should double-quote the mv command parameters in such a case. But most importantly, I can't stress this enough: NEVER USE SPACES IN FILENAMES!!!NEVER USE SPACES IN FILENAMES!!!

This method is the most general one for handling mass renaming. This is basically Bash programming, and as in all programming languages there are many different ways to handle the same problem: I used a parameter expansion, but I could have used an $(echo $file | sed 's/.xyz$//1') command as well, or even a while loop instead of a for loop: ls ~/cp2k/test/*.xyz|while read file ; do mv $file ${file%.xyz$}_test.xyz ; done. The possibilities are endless, and only limited by your imagination... and your Bash knowledge. Everything comes at a cost...

Rename

Fortunately, you do not need to be a Bash expert to perform mass renaming, since small programs were developed for that very purpose.

The rename command allow you to perform mass renaming without Bash programming expertise: rename perl_expression pathname_expression will apply the perl code perl_expression to all paths expended from pathname_expression. So as you guessed it, to master rename you must know the Perl language...

Fortunately, for simple substitution, the syntax is similar to the sed language. The above example can be solved with the command rename 's/.xyz$/_old.xyz/' ~/cp2k/test/*.xyz by replacing the trailing .xyz by the string _old.xyz to the filename of all files matching the ~/cp2k/test/*.xyz pathname pattern.

rename comes with a particularly great feature: a dry run option -n that display the renaming that your command would occur, but without executing them. A perfect way to check that your command is correct. Better safe than sorry! Example: rename -n 's/.xyz$/_old.xyz/' ~/cp2k/test/*.xyz.

rename comes also with a verbose -v option and a force -f option, just like the mv command.

Bonus:

For advanced substitution (matched value reusing), the syntax differs from sed. Example: rename 's|(.*/)([^/]*)_test_(.*)\.xyz|$1$3_test_${2}.xyz|' ~/cp2k/test/*.xyz exchange the position of the sub-strings surrounding _test_ in the files basename.

Mmv : Mass MoVe

An other cool utility for performing such renaming exist and does not require Perl nor Bash programming knowledge: mmv. Unfortunately, this awesome utility is usually not installed by default... Again, everything comes at a cost.

An example is worth a thousand words, the above example becomes mmv '~/cp2k/test/*.xyz' '~/cp2k/test/#1_test.xyz'. Simple and clear: all wildcards expansions (i.e. the *) are indexed and can be easily reused in the final names with the #1 #2 ... #42 ... syntax.

Important note: The expressions MUST be quoted. Otherwise, mmv won't be able to see the pattern (since it will be expanded by Bash first). Fortunately, mmv tries to detect such situations and warn the user, but those are to spot...

mmv does not recognize only the wildcard expansion, but all basic Bash pathname expansions: * (any string), ? (any character), [...] (a character among a defined set). But it also implements the globstar expansion (** in enabled Bash, see next Bash trick) ;.

Last but not least, mmv comes in different flavours :

• mcp (or mmv -c) performs mass copying instead of mass moving.
• mln (or mmv -l) performs mass hard linking.
• mmv -s performs mass soft linking.
• And even mad (or mmv -a) which performs mass appending!

Convinced? Go and install it already!

Optional remark: pax

pax is an other multi-tool utility that allow mass renaming with the -s/old/new/g option, using a ed-based (the EDitor inspiring sed the Stream EDitor) substitution command.

This utility is part of the POSIX standard, but yet rarely installed by default on most machines. Finally, the syntax is far less user-friendly than the other alternatives proposed (since this is clearly not the main feature of this utility designed to process archives), so it is only presented here for completeness...