Help with awk/shell script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with awk/shell script
# 8  
Old 07-23-2007
Quote:
Originally Posted by inditopgun
I would like to write a shell script which will read a list of filenames(with absolute paths) from a file and then determine the just filename component (without the absolute path and the file extension) and the file extension (if any)

The example list of filenames are as below :

/home/mark.williams/a.java
/usr/local/bin/perl
/usr/local/bin/gtar
/usr/local/packages/abcd.efgh.ijkl.mnop.xml

So the filenames may or may not have extensions and also, the filenames might have special characters like ".", "_" embedded in them.

You can do it entirely within the shell; there is no need for external commands:

Code:
while IFS= read -r path
do
  file=${path##*/}
  ext=${path##*.}
  printf "PATH:  %s\nFILE: %s\n EXT: %s\n\n" "$path" "$file" "$ext"
done < FILENAME

###

# 9  
Old 07-23-2007
Quote:
Originally Posted by inditopgun
[...]
My problem is getting the file extension (especially returning null when there is no extenson per se e.g. /usr/local/bin/perl) That's the part which I am struggling with.
[...]
Do you mean something like this?
(this is zsh)

Code:
$ while read;do print "name: \"$REPLY:t:r\"  extension: \"${${REPLY:e}:-null}\"";done<infile
name: "a"  extension: "java"
name: "perl"  extension: "null"
name: "gtar"  extension: "null"
name: "abcd,efgh_ijkl.mnop"  extension: "xml"


Last edited by radoulov; 07-23-2007 at 05:06 PM.. Reason: spelling corrected :)
# 10  
Old 07-23-2007
Thanks!!!!

Thanks guys for your quick replies! Here is a summary of my testing the various solutions proposed by various people above :

cfajohnson - For some reason your solution returned PATH value for EXT when filename didn't have extension (e.g. /usr/local/bin/perl)

radoulov - Sorry couldn't test your solution because I am using sh and ksh only on our boxes. So can't really tell whether it works

Shell Life - Your solution worked like charm and handled all possible filetypes I have in my test environment

thanks again for helping me out! appreciate it.

So here is the solution I landed up using in my script :

#################################################

#!/bin/sh

for cfilename in `cat filelist.txt | sort -u`
do
filename_w_ext=`basename $cfilename`
filename=`echo $filename_w_ext | sed 's;\(.*\)\..*;\1;'`
if [ ${#filename_w_ext} -eq ${#filename} ]; then
file_extension=''
else
file_extension=`echo $filename_w_ext | sed 's/.*\.//'`
fi
echo $filename_w_ext
echo $filename
echo $file_extension
done
# 11  
Old 07-23-2007
Quote:
Originally Posted by inditopgun
Thanks guys for your quick replies! Here is a summary of my testing the various solutions proposed by various people above :

cfajohnson - For some reason your solution returned PATH value for EXT when filename didn't have extension (e.g. /usr/local/bin/perl)

Which is why I followed it up with a correction.

Quote:
Code:
#!/bin/sh

for cfilename in `cat filelist.txt | sort -u`


That will fail if any of the lines contain spaces.

And cat is unnecessary:

Code:
for cfilename in `sort -u filelist.txt`

If you want the file sorted, and duplicates removed, use:

Code:
sort -u filelist.txt | while IFS= read -r path
do
  file=${path##*/}
  case $file in
    *.*) ext=${path##*.} ;;
    *) ext= ;;
  esac
  printf "PATH:  %s\nFILE: %s\n EXT: %s\n\n" "$path" "$file" "$ext"
done



###

This script will be orders of magnitude faster than one which calls external commands (three of them!) for every line of the file.

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to call and sort awk script and output

I'm trying to create a shell script that takes a awk script that I wrote and a filename as an argument. I was able to get that done but I'm having trouble figuring out how to keep the header of the output at the top but sort the rest of the rows alphabetically. This is what I have now but it is... (1 Reply)
Discussion started by: Eric7giants
1 Replies

2. UNIX for Dummies Questions & Answers

Passing shell script parameter value to awk command in side the script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff |... (1 Reply)
Discussion started by: Sarita Behera
1 Replies

3. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

4. Post Here to Contact Site Administrators and Moderators

Unable to pass shell script parameter value to awk command in side the same script

Variable I have in my shell script diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk -F'~' ''$2 == "$id"' {print $0}' > $new I could see value of $id is not passing to the awk... (0 Replies)
Discussion started by: Ashunayak
0 Replies

5. Shell Programming and Scripting

Calling shell script within awk script throws error

I am getting the following error while passing parameter to a shell script called within awk script. Any idea what's causing this issue and how to ix it ? Thanks sh: -c: line 0: syntax error near unexpected token `newline' sh: -c: line 0: `./billdatecalc.sh ... (10 Replies)
Discussion started by: Sudhakar333
10 Replies

6. Shell Programming and Scripting

Call shell script function from awk script

hi everyone i am trying to do this bash> cat abc.sh deepak() { echo Deepak } deepak bash>./abc.sh Deepak so it is giving me write simply i created a func and it worked now i modified it like this way bash> cat abc.sh (2 Replies)
Discussion started by: aishsimplesweet
2 Replies

7. Shell Programming and Scripting

want to pass parameters to awk script from shell script

Hello, I have this awk script that I want to execute by passing parameters through a shell script. I'm a little confused. This awk script removes duplicates from an input file. Ok, so I have a .sh file called rem_dups.sh #!/usr/bin/sh... (4 Replies)
Discussion started by: script_op2a
4 Replies

8. Shell Programming and Scripting

create a shell script that calls another script and and an awk script

Hi guys I have a shell script that executes sql statemets and sends the output to a file.the script takes in parameters executes sql and sends the result to an output file. #!/bin/sh echo " $2 $3 $4 $5 $6 $7 isql -w400 -U$2 -S$5 -P$3 << xxx use $4 go print"**Changes to the table... (0 Replies)
Discussion started by: magikminox
0 Replies

9. UNIX for Advanced & Expert Users

AWK instead of Shell script

I've a list file, which has some file names. ex: list file "list_file" will have data_file1.txt data_file2.txt data_file3.txt : : data_filen.txt Above files will have the below layout: Header1 Header2 *TM* Data record 1 Datarecord 2 Datarecord n *TM* (19 Replies)
Discussion started by: Lokesha
19 Replies

10. Shell Programming and Scripting

using shell script or awk......

suppose my one file has >1|lol|hai i am cdfd dafvshbejndjnudbnjdbyebjcb bdhbdjndjndmdkmdkldhddjndj dbdhbndjndjmdklm sjskndknsknsknks sbsjksnknsknknklsl >2|lol|hai ia m cdfd123 jbsjbjbjbjbjbjbjbjbjbj jkbkkkkkkkkkkkkk khkkkkkkkkkkkkk scbjbjbakbqlblabkla >3lol|hai ia m cdfd3435... (2 Replies)
Discussion started by: cdfd123
2 Replies
Login or Register to Ask a Question