Sponsored Content
Top Forums Shell Programming and Scripting Need to delete large set of files (i.e) close to 100K from a directory based on the input file Post 302688975 by prash358 on Monday 20th of August 2012 05:10:50 PM
Old 08-20-2012
Quote:
Originally Posted by Corona688
Code:
xargs echo rm < list 2> errlog

Remove the 'echo' once you've tested and are sure it really does what you want.
I have run the below command but i need to print to a file where it shows what files have been deleted?

sudo xargs rm <file

Btw all the files are in directory under "/" . But the input file is something like below (i.e) it dont have any path to the file. So i need a command which deletes these files without storing the input file inside the directory under / where the original files reside.

more file
0012301170
0000013300
0000014100
-
-
-

Thanks for the quick reply.

---------- Post updated at 02:18 PM ---------- Previous update was at 02:13 PM ----------

Corona,

Can you please suggest whether i can use the below script to delete the files?

Code:
for i in `cat file`;
do
     rm -f $i
done

If yes i want an echo statement to be printed on what files have been deleted and what file are not found in the directory?

P.S: I need to delete close to 100K files and the file anmes are listed in the input file.

Thanks

---------- Post updated at 04:10 PM ---------- Previous update was at 02:18 PM ----------

Could you please help me in deleting the large set of files without moving the input file to a directory where original files reside.

Thanks

Last edited by Franklin52; 08-21-2012 at 05:27 AM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

i want to delete a file based on existing file in a directory

hi i am having four files in a directory.like 1)sampleRej 2)exampleRej 3)samplemain 4)examplemain my requirement is i have to search for the rejected files (sampleRej,exampleRej) in a directory.if these files in that directory then i have to delete the main files... (3 Replies)
Discussion started by: srivsn
3 Replies

2. Shell Programming and Scripting

Sheel script to Delete a set of files from a given directory

I have a file <filestodelete> containing names of files to to be deleted from a directory <filesstore>. I want a script file which accptes the <filestodelete> and also the location of the directory(<filestore>) and deletes all files matching. Thanks in Advance.. (3 Replies)
Discussion started by: VardhiniVenkat
3 Replies

3. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

4. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

5. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

6. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

7. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

8. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

9. Shell Programming and Scripting

If no input then set directory to current

Hi GUys I have a script where i am counting the number of files that the user has read permissions to in a folder . They are asked for this folder at the start if they dont enter anything then i want to be able to do the search on the current directory, can you help me with that part please... (25 Replies)
Discussion started by: johnnybananas
25 Replies

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies
XARGS(1)						    BSD General Commands Manual 						  XARGS(1)

NAME
xargs -- construct argument list(s) and execute utility SYNOPSIS
xargs [-0pt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr] [-L number] [-n number [-x]] [-s size] [utility [argument ...]] DESCRIPTION
The xargs utility reads space, tab, newline and end-of-file delimited arguments from the standard input and executes the specified utility with them as arguments. The utility and any arguments specified on the command line are given to the utility upon each invocation, followed by some number of the arguments read from standard input. The utility is repeatedly executed until standard input is exhausted. Spaces, tabs and newlines may be embedded in arguments using single (`` ' '') or double (``"'') quotes or backslashes (``''). Single quotes escape all non-single quote characters, excluding newlines, up to the matching single quote. Double quotes escape all non-double quote char- acters, excluding newlines, up to the matching double quote. Any single character, including newlines, may be escaped by a backslash. The options are as follows: -0 Change xargs to expect NUL (``'') characters as separators, instead of spaces and newlines. This is expected to be used in concert with the -print0 function in find(1). -E eofstr Use eofstr as a logical EOF marker. -I replstr Execute utility for each input line, replacing one or more occurences of replstr in up to replacements (or 5 if no -R flag is speci- fied) arguments to utility with the entire line of input. The resulting arguments, after replacement is done, will not be allowed to grow beyond 255 bytes; this is implemented by concatenating as much of the argument containing replstr as possible, to the con- structed arguments to utility, up to 255 bytes. The 255 byte limit does not apply to arguments to utility which do not contain replstr, and furthermore, no replacement will be done on utility itself. Implies -x. -J replstr If this option is specified, xargs will use the data read from standard input to replace the first occurrence of replstr instead of appending that data after all other arguments. This option will not effect how many arguments will be read from input (-n), or the size of the command(s) xargs will generate (-s). The option just moves where those arguments will be placed in the command(s) that are executed. The replstr must show up as a distinct argument to xargs. It will not be recognized if, for instance, it is in the middle of a quoted string. Furthermore, only the first occurrence of the replstr will be replaced. For example, the following com- mand will copy the list of files and directories which start with an uppercase letter in the current directory to destdir: /bin/ls -1d [A-Z]* | xargs -J % cp -rp % destdir -L number Call utility for every number lines read. If EOF is reached and fewer lines have been read than number then utility will be called with the available lines. -n number Set the maximum number of arguments taken from standard input for each invocation of the utility. An invocation of utility will use less than number standard input arguments if the number of bytes accumulated (see the -s option) exceeds the specified size or there are fewer than number arguments remaining for the last invocation of utility. The current default value for number is 5000. -p Echo each command to be executed and ask the user whether it should be executed. An affirmative response, 'y' in the POSIX locale, causes the command to be executed, any other response causes it to be skipped. No commands are executed if the process is not attached to a terminal. -R replacements Specify the maximum number of arguments that -I will do replacement in. -s size Set the maximum number of bytes for the command line length provided to utility. The sum of the length of the utility name, the arguments passed to utility (including NULL terminators) and the current environment will be less than or equal to this number. The current default value for size is ARG_MAX - 4096. -t Echo the command to be executed to standard error immediately before it is executed. -x Force xargs to terminate immediately if a command line containing number arguments will not fit in the specified (or default) command line length. If no utility is specified, echo(1) is used. Undefined behavior may occur if utility reads from the standard input. The xargs utility exits immediately (without processing any further input) if a command line cannot be assembled, utility cannot be invoked, an invocation of the utility is terminated by a signal or an invocation of the utility exits with a value of 255. DIAGNOSTICS
The xargs utility exits with a value of 0 if no error occurs. If utility cannot be found, xargs exits with a value of 127, otherwise if utility cannot be executed, xargs exits with a value of 126. If any other error occurs, xargs exits with a value of 1. SEE ALSO
echo(1), find(1), execvp(3) STANDARDS
The xargs utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compliant. The -J and -R options are non-standard FreeBSD extensions which may not be available on other operating systems. HISTORY
The xargs command appeared in PWB UNIX. BUGS
If utility attempts to invoke another command such that the number of arguments or the size of the environment is increased, it risks execvp(3) failing with E2BIG. BSD
May 7, 2001 BSD
All times are GMT -4. The time now is 12:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy