awk division without rounding


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers awk division without rounding
# 1  
Old 04-22-2020
awk division without rounding

Hello to all,
I'm working on a file that contains a series of decimal values with different precision.
I need to divide these numbers by 100 and I don't need to round.

Code:
awk '{val = $1 / 100; print val}' input_file.txt

Input file:
Code:
0.123456789012
0.123456789012345
0.12345678901234567

output:
Code:
0.00123457
0.00123457
0.00123457

I would like not to round the result and get the following output:
Code:
0.00123456789012
0.00123456789012345
0.0012345678901234567

Thanks in advance

Last edited by vgersh99; 04-22-2020 at 12:02 PM.. Reason: code tags, please!
# 2  
Old 04-22-2020
Decimal numbers represented by a binary system are ALWAYS rounded / subject to rounding. For example, your first number, given as 0.123456789012, is internally approximated / stored as 0.123456789011999995553381381796, which itself is a rounded value already. You may end up less than or greater than your target value, or, in rare cases, exactly on target (0.5, or 0.25, for example).

So, your request is a bit diffucult to fulfill. You can try awk's OFMT variable set to a fixed output field length, but, surprise, additional undesired decimal places occur:
Code:
awk '{val = $1 / 100; print val}' OFMT="%.20f" file
0.00123456789012000006
0.00123456789012344999
0.00123456789012345671


If that is unsatisfying, you can resort to formatted print and a tricky taylored field length computation:

Code:
awk '{val = $1 / 100; printf "%.*f\n", length($1), val}' file
0.00123456789012
0.00123456789012345
0.0012345678901234567

This works as there are two additional places needed for the division by 100, but length ($1) already counts two characters: the leading 0 and dot. If you have more significant leading digits, additional measures / steps need to be taken.
This User Gave Thanks to RudiC For This Post:
# 3  
Old 04-23-2020
Thanks, that's what I was looking for, but how do I put the value into a variable instead of printing it with printf?
# 4  
Old 04-23-2020
Hi.

This might possibly be done for the range of numbers noted by parsing the numbers, inserting 2 zeros, and so avoiding any arithmetic whatsoever -- just a thought ... cheers, drl
# 5  
Old 04-23-2020
Quote:
Originally Posted by charlesep
Thanks, that's what I was looking for, but how do I put the value into a variable instead of printing it with printf?
the paradigm for the ksh/bash-like shells is:
Code:
myVar="$(executeMyCommandHere)"

# 6  
Old 04-23-2020
Quote:
Originally Posted by vgersh99
the paradigm for the ksh/bash-like shells is:
Code:
myVar="$(executeMyCommandHere)"

I need to make the assignment in awk code.
I found another post where they recommend using
Code:
variable = sprintf (x, y)

Thanks anyway for the reply Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[awk] rounding a float number?

Heyas Trying to calculate the total size of a file by reading its bitrate. Code snippet: fs_expected() { # # Returns the expected filesize in bytes # pr_str() { ff=$(cat $TMP.info) d="${ff#*bitrate: }" echo "${d%%,*}" | $AWK '{print $1}' | head -n 1 } t_BYTERATE=$((... (9 Replies)
Discussion started by: sea
9 Replies

2. Shell Programming and Scripting

awk - Division with condition

Hi Friends, I have an input file like this cat input chr1 100 200 1 2 chr1 120 130 na 1 chr1 140 160 1 na chr1 170 180 na na chr1 190 220 0 0 chr1 220 230 nd 1 chr2 330 400 1 nd chr2 410 450 nd nd chr3 500 700 1 1 I want to calculate the division of 4th and 5th columns. But, if... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

3. Shell Programming and Scripting

awk calculation automatically rounding off the output

I have some calculation in my script which is similar to the below example . I find that sometimes when using large decimal digits, the output gets automatically rounded off and it is affecting the program. I am not able to understand what is happening here.. awk '{ a=6.32498922 a1=6.324... (5 Replies)
Discussion started by: wanderingmind16
5 Replies

4. Shell Programming and Scripting

awk & division

vmstat|awk '{print $3}'|tail -1 returns 6250511, but what I need is 24416, which is 6250511 divided by 256. Please advise. Thank you so much (2 Replies)
Discussion started by: Daniel Gate
2 Replies

5. Shell Programming and Scripting

awk, floating point and rounding

I had a person bring an interesting problem to me that appears to involve some sort of rounding inside awk. I've verified this with awk and nawk on Solaris as well as with gawk 3.1.5 on a Linux box. The original code fragment he brought me was thus: for (index=0; index < 1; index=index+.1) ... (4 Replies)
Discussion started by: mmyer2
4 Replies

6. UNIX for Advanced & Expert Users

awk: division by zero

I received error "awk: division by zero" while executing the following statement. SunOS 5.10 Generic_142900-15 sun4us sparc FJSV,GPUZC-M echo 8 | awk 'END {printf ("%d\n",NR/$1 + 0.5);}' file1.lst awk: division by zero Can someone provide solution? Thanks Please use code... (11 Replies)
Discussion started by: kumar77
11 Replies

7. Shell Programming and Scripting

AWK rounding up numbers

Hi, I have managed to round up numbers by using the following command: echo "5.54" | awk '{printf "%.0f\n", $1}' result 6 How can I round up all the numbers in a column in a file and print the lines with the new calculated totals? Thanks, (3 Replies)
Discussion started by: keenboy100
3 Replies

8. Shell Programming and Scripting

awk Division and modulus

I need to read the file divide 3 column with 2nd and run a modulus of 10 and check whether the remainder is zero or not if not print the entire line. cat filename | awk '{ if ($3 / $2 % 10 != 0) print $0}' Whats wrong with it ? (4 Replies)
Discussion started by: dinjo_jo
4 Replies

9. Shell Programming and Scripting

Rounding issue with awk

Hi Friends, I am trying to round following number. 0.07435000 echo "0.07435000"|awk '{printf "%s\n",$1*100}'|awk '{printf "%.2f\n",$1}' It returns: 7.435 It should return: 7.44 Any suggestion please? Thanks, Prashant (2 Replies)
Discussion started by: ppat7046
2 Replies

10. UNIX for Dummies Questions & Answers

Annoying rounding issue in awk

Hello I am getting this very annoying issue in awk: awk '{a=12825;b=a*1.25; print b}' test 16031.2 Thing is the multiplication result is wrong... Result should be 16031.25. I think the issue only happens on bigger numbers. What can I do to get passed this? Thanks by advance (3 Replies)
Discussion started by: Indalecio
3 Replies
Login or Register to Ask a Question