Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Insert a line in a file by deleting that line from another file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Insert a line in a file by deleting that line from another file
# 1  
Old 11-15-2012
IBM Insert a line in a file by deleting that line from another file
Hi all,

I have a huge file(size more that 5GB).
I want to do some manupulation with the records and write to another file.
As the size of the file is huge and there is a space constraint in that directory, I want to delete that record from first file after writing it in to second file.
Example:-
Code:
cat first.dat | while read line
do
echo $line| .........    >> second.dat
# Here after writing to second file i want to delete the same record from first.dat
done

can anyone please help me on this???

thanks in adv....
Last edited by Scott; 11-15-2012 at 05:58 PM.. Reason: Code tags, again
# 2  
Old 11-15-2012
You can use sed to match that line and delete it from first.dat:-
Code:
sed -i "/$line/,1d" first.dat

But the line pattern has to be unique or you will end up deleting similar matching lines.

---------- Post updated at 15:23 ---------- Previous update was at 15:19 ----------

Another option is to use a COUNTER variable which you can increment for each line read and use the COUNTER variable to delete that particular line.
Code:
COUNTER=1
cat first.dat | while read line
do
   echo $line| .........    >> second.dat
   sed -i "${COUNTER}d" first.dat
   COUNTER=$( expr $COUNTER + 1 )
done

# 3  
Old 11-15-2012
As i am working in AIX, there is no "-i" option in sed
# 4  
Old 11-16-2012
first of all, reading the 5GB file using cat and while is not advisiable.

you need to choose awk,perl or pyton to process the file. in what conidtion, you want to remove the lines from the source file ?

give some input data and expected output.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Insert content of file before the first occurrence of a line starts with a pattern in another file

Hi all, I'm new to scripting.. facing some problems while inserting content of a file into another file... I want to insert content of a file (file2) into file1, before first occurrence of "line starts with pattern" in file1 file1 ====== working on linux its unix world working on... (14 Replies)
Discussion started by: Jagadeesh Kumar
14 Replies

2. Shell Programming and Scripting

Insert content of a file to another file at a line number which is given by third file

Hi friends, here is my problem. I have three files like this.. cat file1.txt ======= unix is best unix is best linux is best unix is best linux is best linux is best unix is best unix is best cat file2.txt ======== Windows performs better Mac OS performs better Windows... (4 Replies)
Discussion started by: Jagadeesh Kumar
4 Replies

3. UNIX for Beginners Questions & Answers

Insert a line of text on nth line of a file

Hi All, I am using UNix Sun OS sun4u sparc SUNW,SPARC-Enterprise My intention is to insert a line of text after 13th line of every file inside a particular directory. While trying to do it for a single file , i am using sed sed '3 i this is the 4th line' filename sed: command garbled: 3... (5 Replies)
Discussion started by: gotamp
5 Replies

4. Shell Programming and Scripting

How to read a text file line by line and insert into a database table?

I have a test file that I want to read and insert only certain lines into the the table based on a filter. 1. Rread the log file 12 Hours back Getdate() -12 Hours 2. Extract the following information on for lines that say "DUMP is complete" A. Date B. Database Name C.... (2 Replies)
Discussion started by: JolietJake
2 Replies

5. Shell Programming and Scripting

Insert a new line before every 5th line in a file

Hi, I need to insert a new line containing the string "QUERY" above every 5 lines. The below piece of code inserts a new line after every 5th line awk '{print $0} !(NR%5) {print "QUERY"}' sed 'n;n;n;n;G;' --> I do not know how to give "QUERY" string here But I need to insert it before... (4 Replies)
Discussion started by: royalibrahim
4 Replies

6. Shell Programming and Scripting

Insert a complete file into another file after certain fixed line

Hi Friends, I am writing a shell script where it is required to insert a file say file1 into file2 after certain number of fixed lines in file2.. Please help me how this could be done.. Please suggest me any useful links i need to go through to achieve this.... Thanks in advance .. (2 Replies)
Discussion started by: amr89
2 Replies

7. UNIX for Dummies Questions & Answers

Deleting the first line of .gz file

Hi All, I have too many .gz files (test.gz). Task is to remove first line of each file. Can I do it without unzipping the files? Your help is appreciated. (4 Replies)
Discussion started by: Chulamakuri
4 Replies

8. Shell Programming and Scripting

How to insert text into first line of the file and middle of the file?

Script 1 Pre-requisites Create a file with x amount of lines in it, the content of your choice. Write a script that takes two arguments. The first being a line of text, the second being your newly created file. The script should take the first argument and insert it into the very top (the... (3 Replies)
Discussion started by: ali hussain
3 Replies

9. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies
Login or Register to Ask a Question