flow-merge(1) General Commands Manual flow-merge(1)NAME
flow-merge -- Merge flow files.
SYNOPSIS
flow-merge [-aghm] [-b big|little] [-C comment] [-d debug_level] [-o filename] [-z z_level] [file|directory ...]
DESCRIPTION
The flow-merge utility processes files and/or directories of files in the flow-tools format. The resulting merged data set is written to
the standard output or file specified by -o. If file is a single dash (`-') or absent, flow-merge will read from the standard input.
Unlike flow-cat, flow-merge interleaves flow records preserving the relative chronological order.
OPTIONS -a Do not ignore filenames that begin with tmp.
-b big|little
Byte order of output.
-C Comment
Add a comment.
-d debug_level
Enable debugging.
-g Sort file list by capture start time before processing.
-h Display help.
-m Disable the use of mmap().
-p Preload headers. Use to preserve meta information such as lost flows.
-o file Write to file instead of the standard out.
-z z_level
Configure compression level to z_level. 0 is disabled (no compression), 9 is highest compression.
file|directory...
Process the files and/or directory.
EXAMPLES
Merge all flow files begining with ft-v05.2001-05.01, use flow-print to display the results.
flow-merge ft-v05.2001-05-01.* | flow-print
BUGS
None known.
AUTHOR
Larry Lidz ellidz@eridu.uchicago.edu
SEE ALSO flow-tools(1)flow-merge(1)
Check Out this Related Man Page
flow-expire(1) General Commands Manual flow-expire(1)NAME
flow-expire -- Manage storage of flow file archives by expiring old data.
SYNOPSIS
flow-expire [-h] [-d debug_level] [-e expire_count] [-E expire_size] -w workdir
DESCRIPTION
The flow-expire utility will remove the oldest flow files in a directory based on either a count of files or space utilization. The direc-
tory is recursively searched for flow files. Files that do not have a flow-tools signature will be ignored. The internal timestamp is
used so backups or copies of the flow files that do not retain the original timestamp will not impact the operation of flow-expire. flow-
expire is typically used to manage storage in a distributed environment where flows are collected on a different server than they are
archived.
OPTIONS -d debug_level
Enable debugging.
-e expire_count
Retain the maximum number of files so that the total file count is less than expire_count.
-E expire_size
Retain the maximum number of files so that the total storage is less than expire_size. The letters b,K,M,G can be used as multi-
pliers, ie 16 Megabytes is 16M.
-h Display help.
-w workdir
Work in workdir.
EXAMPLES
Remove the oldest flow files in /flows/krc4 until the total storage is less then 2 Gigabytes.
flow-expire -E2Gig -w /flows/krc4
Remove the oldest flow files in /flows/krc4 until the total number of files is less then 100.
flow-expire -e100 -w /flows/krc4
BUGS
Empty directories are not removed.
AUTHOR
Mark Fullmer maf@splintered.net
SEE ALSO flow-tools(1)flow-expire(1)
hello gurus ,
i want to merge different column from two different file. file struture is below.
file 1
-------
~information is given
Name class section
A 5 b
B 7 C
D 8 A
file 2 (10 Replies)
Hi,
I have a list of 70 files in a directory and I need to merge the content of each file into one big matrix file (71 columns x 3060 rows).
Each file has the following format only two columns per file:
unique identifier1
randomtext1 randomtext1
a 5
b 3
c 6
d 3
e 2... (11 Replies)
Hi Guys,
i have two files:
fileA:
20090611 00:00:11
20090611 00:00:11
20090611 00:00:24
20090611 00:01:10
20090611 07:13:00
fileB:
20090611 00:00:01
20090611 00:00:12
20090611 00:00:24
20090611 00:01:12
20090611 09:13:00
want to make two files into a single file, but follow the... (14 Replies)
Hi,
I want to merge the two files based on the key file's columns.
The key file:
DATE~DATE
HOUSE~IN_HOUSE
CUST~IN_CUST
PRODUCT~PRODUCT
ADDRESS~CUST_ADDR
BASIS_POINTS~BASIS_POINTS
...
The other 2 files are From_file & To_file -
The From_file:
DATE|date/time|29|9 ... (9 Replies)
I have a need to merge two files on the value of an index column.
input file 1
id filePath MDL_NUMBER
1 MFCD00008104.mol MFCD00008104
2 MFCD00012849.mol MFCD00012849
3 MFCD00037597.mol MFCD00037597
4 MFCD00064558.mol MFCD00064558
5 MFCD00064559.mol MFCD00064559
input file 2
... (9 Replies)
I have two files as below
A file
/* comment for id1 */
"id1" = "x1"
/* comment for id2 */
"id2" = "x2"
/* comment for id3 */
"id3" = "x3"
B file
/* comment for id1 */
"id1" = "y1"
/* comment for id2 */
"id2" = "x2" (22 Replies)
Hi,
My requirement is,there is a directory location like:
:camp/current/
In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Hello and Good day
I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I need to merge $2,$3,$6 from first file and $6 from another files.
File1:
$1 $2 $3 $4 $5 $6... (8 Replies)
Perl script to merge cells
---------- Post updated at 12:59 AM ---------- Previous update was at 12:54 AM ----------
I am using below code to read files from a dir and print to excel.
open(my $in, '<', $file) or die "Could not open file: $!";
my $rowCount = 0;
my $colCount = 0;... (11 Replies)
Hey Guys,
I want to merge all files (Apache Tomcat Access Logs) for a particular date say "Aug 24" to be merged into a single file.
Is there any quick hack for that ?
$ ls -alrth access_log2016-08-*|grep "Aug 24"
-rw-rw-r--. 1 tomcat tomcat 16M Aug 24 00:00... (8 Replies)
I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file.
file1.txt:
1234
1234
1234
file2.txt:
aaaa
bbbb
cccc
dddd
cat file1.txt file2.txt > file3.txt
file3.txt:
1234
1234
1234aaaa
bbbb
cccc... (13 Replies)
Hi all,
i need help.
I have two csv files with a huge amount of data.
I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file.
Example
File1: (only one column)
profile_id
57036226... (11 Replies)
I have 2 big files over 4Gbs each. I'm looking for a way to print 1 file, then when that file finish printing another file proceeds to print beside it and merge the lines together. How would to cmd or code this?
from itertools import izip_longest
with open("file1") as textfile1,... (14 Replies)