Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Copy files by automatically creating directories
Top Forums Shell Programming and Scripting Copy files by automatically creating directories Post 302551586 by shoaibjameel123 on Wednesday 31st of August 2011 03:27:52 AM
Old 08-31-2011
Copy files by automatically creating directories
I am in a situation where I have some hundreds of thousands of files in a directory (name of directory: big_directory). Now, working on this directory is extremely slow.

I now plan to do this:

Copy 1000 files in each of the directories by creating the directories automatically. All files have .dat extension in the directory and have names like 1.dat, 2.dat, 3.dat until the last file number

This is what I intend to do:

1. Automatically create a directory "dir1" and copy first 1000 ".dat" files (present in big_directory) in it.

2. Automatically create a directory "dir2" and then copy the next 1000 ".dat" files (present in big_directory) in it.

3. Automatically create a directory "dir3" and then copy the next 1000 ".dat" files (present in big_directory) in it.

4. and so on until last file

I am using Linux with BASH. I know how to copy first 1000 files but not sure how to create directories automatically using a counter and copy them there.

Code:
cp `ls -ltr | grep ^- | head -1000 | tr -s " " | cut -d " " -f9` destination

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

copy multiple files in different directories

I have a report file that is generated every day by a scheduled process. Each day the file is written to a directory named .../blah_blah/Y07/MM-DD-YY/reportmmddyy.tab I want to copy all of this reports to a separate directory without having to do it one by one. However, if I try cp... (3 Replies)
Discussion started by: ken2834
3 Replies

2. Shell Programming and Scripting

Bash and Awk for creating directories and moving files

I have a security system that FTPs the camera files to my machine, however I want to sort the pictures (taken every 30s) into directories by hour. Every picture uses the following file format. yymmddhhmmsstt.jpg (where tt is the milliseconds) I am thinking the for loop is best for file... (11 Replies)
Discussion started by: Kiint
11 Replies

3. UNIX for Dummies Questions & Answers

Copy Directories excluding files

Hi guys, I want to copy folder and sub folders only. I don't want the files. If i use cp -r command it will copy entirely with files. Could any one suggest me. Thanks in advance (1 Reply)
Discussion started by: karthik82
1 Replies

4. Shell Programming and Scripting

Copying a files from a filter list and creating their associated parent directories

Hello all, I'm trying to copy all files within a specified directory to another location based on a find filter of mtime -1 (Solaris OS). The issue that I'm having is that in the destination directory, I want to retain the source directory structure while copying over only the files that have... (4 Replies)
Discussion started by: hunter55
4 Replies

5. UNIX for Dummies Questions & Answers

how can i copy those files into other directories have the same name

how can i copy those files into other directories have the same name but different in the end i have files in directory called test: 10_10_asdadfsdfad.txt 10_10_11_asdawqefwkjasd.txt 10_10_11_12_asdafjjhoqwd.txt i want to put them in exist directory thart i have on my system i have... (1 Reply)
Discussion started by: t17
1 Replies

6. Solaris

How to extract files from a tar file without creating the directories?

Hello all. I have a tar file that contains a number of files that are stored in different directories. If I extract this tar file with -xvf , the directories get created. Is there a way to extract all of the files into one directory without creating the directories stored in the tar file. (9 Replies)
Discussion started by: gkb
9 Replies

7. Shell Programming and Scripting

Creating timestamped directories/files in expect

Hi, I am looking for the syntax to provide a timestamped log_file in expect. Basically I want the equivalent of: /outputs/`date +%d%m%y`/session`date +%H%M` But scripted in expect so it can be handed over to the logfile function. I have tried playing around with the timestamp... (1 Reply)
Discussion started by: krypton
1 Replies

8. Shell Programming and Scripting

Archiving and moving files into directories, creating directories, etc.

how can i move "dataName".sql.gz into a folder called 'database' and then move "$fileName".tar.gz * .htaccess into a folder called 'www' with the entire gzipped file being "$fileName".tar.gz? Is this doable or overly complex. so mydemo--2015-03-23-1500.tar.gz > database -... (5 Replies)
Discussion started by: wyclef
5 Replies

9. Shell Programming and Scripting

Need BASH Script Help to Move Files While Creating Directories

I've got this script to loop through all folders and move files that are more than 2 years old. I'm using the install command because it creates the necessary directories on the destination path and then I remove the source. I'd like to change the script to use the mv command since it is much... (4 Replies)
Discussion started by: consultant
4 Replies

10. Shell Programming and Scripting

Copy files in respective directories

Hi Guys, I need to copy the files to respective directories based on name of the file. My script is something like below con=$1 for file in `cat $con` do file_tmp=$(ls -t1 $path| grep -i $file | head -n 1) echo $file_tmp if then cp $path$file_tmp $DIR/ap if then... (16 Replies)
Discussion started by: Master_Mind
16 Replies

LEARN ABOUT DEBIAN

pymcaroitool

pymcaroitool(1) 					 PyMca X-Ray Fluorescence Toolkit					   pymcaroitool(1)

NAME

       pymcaroitool - PyMca region-of-interest imaging X application

SYNOPSIS

       pymcaroitool [OPTIONS]... [FILE(S)]

DESCRIPTION

       Start the graphical user interface of the PyMca X-Ray Fluorescence Toolkit region-of-interest imaging tool.

       This  tool  is  best  suited  for handling datasets that can be represented by three-dimensional arrays. Typical cases are stacks of images
       (first dimension is image number) or 2D maps of 1D spectra (last dimension is spectrum channel number).

       It allows to display maps of particular regions of the spectra or spectra of a particular region of the map.

       A system of plugins allow to extend the capabilities of this tool. Plugins for multivariate analysis are already built in.

       If FILE is given, it will be opened in the program provided its format is supported.

EXAMPLES

       pymcaroitool

       Start the program with a file browser to select the input files.

       pymcaroitool file_0001.edf

       Tries to open the file named file_0001.edf and all indexed files of the form file_????.edf where ???? is a number.

       pymcaroitool --imagestack=1 file_0001.edf

       Tries to open the file named file_0001.edf and all indexed files of the form file_????.edf where ???? is a number as a set of images.

       pymcaroitool uncompressed_tiff_file_0001.tif

       Tries to open a series of uncompressed TIFF files as an image stack.

       pymcaroitool --begin=100 --end=200 --filepattern=file_%05d.edf

       Start the program loading the single indexed files from file_00100.edf to file_00200.edf

       pymcaroitool --begin=10,100 --end=20,200 --filepattern=row%d_col%03d.dat

       Load the double indexed files from row10_col100.dat, row10_col101.dat, ...  to row20_col00199.dat, row20_col00200.dat

CAVEATS

       If files f_000.xxx and f_001.xxx are present in the same directory, the program will always try to load both of them  unless  a	cumbersome
       way using a file pattern is used.

SEE ALSO

       HDF5, h5py

ESRF
								    March 2012							   pymcaroitool(1)
All times are GMT -4. The time now is 05:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy