Moving files from parent path to multiple child path using bash in efficient way
Hi All,
Can you please provide some pointers to move files from Base path to multiple paths in efficient way.Folder Structure is already created.
I don't want to move directory structure. I want to move files from that source path to destination path on daily basis. I would need pointer to write a bash script to make sure aaa.txt would exactly go and sit only from /Path/AdminUser/User1/1111/Reports/ to /Path/User1/1111/Reports/........
One of geek folk helped provided with inputs to start as below, It still threw error, can we also write it in an efficient way?
Last edited by Don Cragun; 05-03-2015 at 08:10 PM..
Reason: Add CODE and ICODE tags.
Does the file hierarchy you're modifying contain symbolic links matched by the patterns /Path/AdminUser/* and /Path/AdminUser/*/*? Your textual description didn't say anything about symbolic links, but your (non-working) code explicitly ignores them. If symbolic links are found, what do want to have done with them?
Are you trying to move all files in the file hierarchy rooted in /Path/AdminUser/ up one level in that hierarchy to /Path/; or are you just trying to move regular files with pathnames matching the pattern /Path/AdminUser/*/*/Reports/*.txt to the corresponding pathname after removing AdminUser/ from the source pathname?
If a directory in the target pathname does not already exist, should your script report an error and move on, should it create the missing directories, or should it silently ignore that source pathname?
Do you want to leave the (empty) source directories in place after files are moved out of them, or should emptied directories be removed?
Is the entire file hierarchy rooted in /Path in a single filesystem?
Instead of saying "It still threw error, can we also write it in an efficient way?", show us exactly what "error(s) it produces" (in CODE tags).
Please see answers for questions, which helps provide complete picture:
1. We dont have symbolic links in path, so we can ignore it.
2. I am trying to move regular files with pathnames matching the pattern /Path/AdminUser/*/*/Reports/*.txt to the corresponding pathname after removing AdminUser/ from the source pathname.
3. It can ignore that source pathname.
4. Empty source directories should be in place after files are moved out of them.
5. Yes the entire file hierarchy rooted in /Path in a single filesystem.
To improve performance can any thing like xargs play a major role
@ Chubler
Thank you. rsync works, but since in my scenario, its not just folder structure, I have to make sure security & ownership of file system isn't affected so I was inclined to use script to move files in loop from parent path to child path, with out touching directories.
You have said that a "geek" gave you a non-working script that uses bash, but you haven't said what shell(s) are available for you to use nor what operating system(s) are present on systems that will be running this script.
The following script will work with any POSIX-conforming shell on any system that has a shell that conforms to the 1992 or any later version of the POSIX Shell and Utilities standard (and many shells that support a fairly common subset of POSIX requirements) such as any version of ksh or bash:
If you have a recent ksh that expands ~(N)*.txt to an empty list if there are no files in the current directory ending with .txt or if you have a recent bash that expands [ICODE]*.txt[/CODE] to an empty list after the command shopt -s nullglob, then the 2nd for loop in the above script can be simplified as in:
or:
If the mv commands echoed by the above scripts do what you want, remove the echo shown in red to actually execute the mv commands.
All of these scripts assume that (since they move into the various source directories before expanding the list of files to be moved) that list won't exceed ARG_MAX limits (so xargs or similar utilities to combine groups of files into single invocations of mv) won't be needed. They all produce one mv command for each source directory that contains files to be moved that also have an existing target directory to receive those files. If some directories have a huge number of files to be moved with names long enough to exceed ARG_MAX, the 2nd for loop can be modified to feed the list of files to be moved to xargs, but it will run a little bit slower (and depending on what operating system you're using), may have problems with filenames containing whitespace characters and may also require special processing to specify the target directory in the appropriate place in an xargs-generated mv command.
This User Gave Thanks to Don Cragun For This Post:
The bash will trim the folder to trim folder. Within each of the folders (there may be more than 1) and the format is always the same, are several .bam and matching .bam.bai files (file structure) and the bashunder that executes and trims the .bam as expected but repeats the.bam.bai extentions... (9 Replies)
Currently I am using this laborious command
lvdisplay | awk '/LV Path/ {p=$3} /LV Name/ {n=$3} /VG Name/ {v=$3} /Block device/ {d=$3; sub(".*:", "/dev/dm-", d); printf "%s\t%s\t%s\n", p, "/dev/mapper/"v"-"n, d}'
Would like to know if there is any shorter method to get this mapping of... (2 Replies)
Hi everyone
i am very new to linux , working on bash shell.
I am trying to solve the given problem
1. Create a process and then create children using fork
2. Check the Status of the application for successful running.
3. Kill all the process(threads) except parent and first child... (2 Replies)
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
a) Some Unix tools are at $HOME/mytools directory. Make these tools accessible for use from any directory.
b)... (2 Replies)
Guys I have a big issue that I need to get fixed ASAP however I can not seem to find a way to do it. We started to use zones with Solaris 10 at work and we moved a zone from a SIT box to a DEV box. Problem is the software we have installed is looking at a /lcl/sit/apps/ path and it needs to look... (5 Replies)
Hi, I'm new here an dlearning a lot from this forum. i didnt find any solution for this in the forum.
I have already checked in folders in subversion named
HTT01,... HTT21.. and have files in each folder like below:
HTT01/HTT01_00000.hex
HTT01/HTT01_00000_fb_result.hex... (2 Replies)
Hi
I have a shell script A which calls another 10 shell scripts which run in background. How do i make the parent script wait for the child scripts complete, or in other words, i must be able to do a grep of parent script to find out if the child scripts are still running.
My Code:
... (5 Replies)
hi experts(novice people can stay away as it is no child's game),
i am developing a script which works like recycle bin of windows.
the problem i am facing is that when ever i am trying to delete a file which is situated in parent directory or parent's parent directory i am unable to... (1 Reply)
Hello All.
I am having a directory /tmp/rahul which contains many files in the format
@#home@#rahul@#programs@#script.pl
where /home/rahul/programs is the directory where the script.pl file is to be placed.
I have many files in this format.
What i want is a script which read these... (7 Replies)