Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Split Command Generating Incomplete Output Files Post 303000625 by Ariean on Sunday 16th of July 2017 10:39:59 AM
Old 07-16-2017
Split Command Generating Incomplete Output Files

Hello All,
May i please know how do i ensure my split command would NOT generate incomplete output files like below, the last lines in each file is missing some columns or last line is complete.

Code:
split -b 50GB File File_

Code:
File_aa

|551|70210203|xxxxxxx|12/22/2010 20:44:58|11/01/2010 00:00:00||||||||1.0000000000||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||AD7678||||||||||||||||||||||||||||||||||xxxxxx||||||5034||e4mdevl1|07/14/2017 15:48:02||
|551|70220223|xxxxxxx|12/22/2010 20:44:58|11/01/2010 00:00:00||||||||1.0000000000||||||||||||

File_ab

|562|71654982|xxxxxx|11/22/2011 19:01:45|10/01/2011 00:00:00||||||||1.0000000000||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||AH6734||||||||||||||||||||||||||||||||||3xxxx||||||5034||e4mdevl1|07/14/2017 15:48:02||
|562|71655059|171348415

File_ac

|573|57162801|xxxxxx|10/24/2012 15:22:12|09/01/2012 00:00:00||||||||0.9965540000||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||AB0496||||||||||||||||||||||||||||||||||xxxxx||||||5034||e4mdevl1|07/14/2017 15:48:02||
|573|57162908|4004900159|10/

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk command to split in to 2 files

Hi, I have a problem in grepping a file for 2 strings and writing them to 2 appropriate files. I need to use the awk command and read the file only once and write to the appropriate file. My file is very huge in size and it is taking a long time using cat command and grep command. Can anyone... (3 Replies)
Discussion started by: m_subra_mani
3 Replies

2. Shell Programming and Scripting

Join of files is incomplete?!

Hi folks, I am using the join command to join two files on a common field as follows: File1.txt Adsorption|H01.181.529.047 Adult|M01.060.116 Children|M01.055 File2.txt 5|Adsorption|C0001674 7|Adult|C000001 6|Children|C00002 join -i -t "|" -a 2 -1 1 -2 2 File1.txt File2.txt This... (7 Replies)
Discussion started by: s0460205
7 Replies

3. Shell Programming and Scripting

How to ignore incomplete files

On Solaris & AIX, suppose there is a directory 'dir'. Log files of size approx 1MB are continuously being deposited here by scp command. I have a script that scans this dir every 5 mins and moves away the log files that have been deposited so far. How do I design my script so that I pick up... (6 Replies)
Discussion started by: sentak
6 Replies

4. Solaris

How to ignore incomplete files

On Solaris, suppose there is a directory 'dir'. Log files of size approx 1MB are continuously being deposited here by scp command. I have a script that scans this dir every 5 mins and moves away the log files that have been deposited so far. How do I design my script so that I pick up *only*... (6 Replies)
Discussion started by: sentak
6 Replies

5. UNIX for Dummies Questions & Answers

split files into specified number of output files

Hi everyone, I have some large text files that I need to split into a specific number of files of equal size. As far as I know (and I don't really know that much :)) the split command only lets you specify the number of lines or bytes. The files are all of a different size, so the number of... (4 Replies)
Discussion started by: Migrainegirl
4 Replies

6. Shell Programming and Scripting

need help generating this output

need to check hardware error are zero iostat -en |awk '{ if ( $2 == 0 ) { print " " } else { print " Hardware errors "} } can someone please tell me whats wrong with this ---------- Post updated at 10:19 PM ---------- Previous update was at 10:16 PM ---------- iostat -en ----... (11 Replies)
Discussion started by: arch12
11 Replies

7. Shell Programming and Scripting

renaming the default output name of the split command

Hello, I would like to split a file but I dont what the subfiles to be named the way they are: ex: default name: xaa xab xac xad xae desired name: b01 b02 b03 b04 b05 I know I can rename them afterwards however I have a varialbe split lenghth, in other words I am not sure how... (2 Replies)
Discussion started by: smarones
2 Replies

8. Shell Programming and Scripting

adding file extensions to split output files

Hello, I've searched this forum and others for a solution to my problem but nothing seems just right, I'm hoping I can get some help (seems like this should be easy, and I apologize if I've missed something on the forum): I have several large .fastq DNA sequence files (~20million reads,... (2 Replies)
Discussion started by: ljk
2 Replies

9. Shell Programming and Scripting

Split: File into multiple and keeping the same 3 lines from input into all output files

The following code will split the infile into multiple files. However, I need it to insert the same first 3 lines from the original input file into each splitted file. How do I modify my script below to do so: print -n "Enter file name to split? " ; read infile if then echo "Invalid file... (4 Replies)
Discussion started by: mrn6430
4 Replies

10. Shell Programming and Scripting

Randomly selecting sequences and generating specific output files

I have two files containing hundreds of different sequences with the same Identifiers (ID-001, ID-002, etc.,), something like this: Infile1: ID-001 ATGGGAGCGGGGGCGTCTGCCTTGAGGGGAGAGAAGCTAGATACA ID-002 ATGGGAGCGGGGGCGTCTGTTTTGAGGGGAGAGAAGCTAGATACA ID-003... (18 Replies)
Discussion started by: Xterra
18 Replies
split(1)						      General Commands Manual							  split(1)

NAME
split - Splits a file into pieces SYNOPSIS
Current syntax split [-l line_count] [-a suffix_length] [file | -] [prefix] split -b n [k|m] [-a suffix_length] [file | -] [prefix] Obsolescent syntax split [-number] [-a suffix_length] [file | -] [prefix] STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: split: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Uses suffix_length letters to form the suffix portion of the file names of the split file. If -a is not specified, the default suffix length is two letters. If the sum of the prefix and the suffix arguments would create a file name exceeding NAME_MAX bytes, an error occurs. In this case, split exits with a diagnostic message and no files are created. Split a file into pieces n bytes in size. Split a file into pieces n kilobytes (1024 bytes) in size. Split a file into pieces n megabytes (1048576 bytes) in size. Specifies the number of lines in each output file. The line_count argument is an unsigned decimal integer. The default value is 1000. If the input does not end with a newline character, the partial line is included in the last output file. Specifies the number of lines in each output file. The default is 1000 lines per output file. If the input does not end with a newline character, the partial line is included in the last output file. (Obsolescent) OPERANDS
The pathname of the file to be split. If you do not specify an input file, or if you specify -, the standard input is used. DESCRIPTION
The split command reads file and writes it in number-line pieces (default 1000 lines) to a set of output files. The size of the output files can be modified by using the -b or -l options. Each output file is created with a unique suffix consisting of exactly suffix lowercase letters from the POSIX locale. The letters of the suffix are used as if they were a base-26 digit system, with the first suffix to be created consisting of all a characters, the second with b replacing the last a etc., until a name of all zs is cre- ated. By default, the names of the output files are x, followed by a two-character suffix from the character set as described above, starting with aa, ab, ac, etc., and continuing until the suffix zz, for a maximum of 676 files. The value of prefix cannot be longer than the value of NAME_MAX from <limits.h> minus two. If the number of files required is greater than the maximum allowed by the effective suffix length (such that the last allowable file would be larger than the requested size), split fails after creating the last possible file with a valid suffix. The split command will not delete the files it created with valid suffixes. If the file limit is not exceeded, the last file created contains the remainder of the input file and thus might be smaller than the requested size. EXIT STATUS
The following exit values are returned: Successful completion. An error occurred. EXAMPLES
To split a file into 1000-line segments, enter: split book This splits book into 1000-line segments named xaa, xab, xac, and so forth. To split a file into 50-line segments and specify the file name prefix, enter: split -l50 book sect This splits book into 50-line segments named sectaa, sectab, sectac, and so forth. ENVIRONMENT VARIABLES
The following environment variables affect the execution of split: Provides a default value for the internationalization variables that are unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari- ables contain an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value, overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multibyte characters in arguments and input files). Determines the locale for the format and contents of diagnostic messages written to standard error. Determines the location of message catalogues for the processing of LC_MESSAGES. SEE ALSO
Commands: bfs(1), csplit(1) Standards: standards(5) split(1)
All times are GMT -4. The time now is 12:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy