flow-cat(1) General Commands Manual flow-cat(1)NAME
flow-cat -- Concatenate flow files
SYNOPSIS
flow-cat [-aghmp] [-b big|little] [-C comment] [-d debug_level] [-o filename] [-t start_time] [-T start_time] [-z z_level]
[file|directory ...]
DESCRIPTION
The flow-cat utility processes files and/or directories of files in the flow-tools format. The resulting concatenated data set is written
to the standard output or file specified by -o. If file is a single dash (`-') or absent, flow-cat will read from the standard input.
OPTIONS -a Do not ignore filenames that begin with tmp.
-b big|little
Byte order of output.
-C Comment
Add a comment.
-d debug_level
Enable debugging.
-g Sort file list by capture start time before processing.
-h Display help.
-m Disable the use of mmap().
-p Preload headers. Use to preserve meta information such as lost flows.
-o file Write to file instead of the standard out.
-t start_time
Select flow files up to start_time. If used with -T select files between start_time and end_time.
-T end_time
Select flow files after end_time. If used with -t select files between start_time and end_time.
-z z_level
Configure compression level to z_level. 0 is disabled (no compression), 9 is highest compression.
file|directory...
Process the files and/or directory.
TIME /DATE parsing
start_time and end_time parsing is implemented with getdate.y, a commonly used function to process free-form time date specifications.
Example usage borrowed from cvs:
1 month ago
2 hours ago
400000 seconds ago
last year
last Monday
yesterday
a fortnight ago
3/31/92 10:00:07 PST
January 23, 1987 10:05pm
22:00 GMT
EXAMPLES
Concatenate all flow files begining with ft-v05.2001-05.01, use flow-print to display the results.
flow-cat ft-v05.2001-05-01.* | flow-print
Concatenate flow files in /flows/krc4, store store the output in compressed.flows at compression level 9 (best). The headers are preloaded
so various metadata such as the flow count is correct in the result. Filenames begining with tmp which are typically in-progress flow
files from flow-capture are not processed.
flow-cat -p -z9 /flows/krc4 > compressed.flows
BUGS
None known.
AUTHOR
Mark Fullmer maf@splintered.net
SEE ALSO flow-tools(1)flow-cat(1)
Check Out this Related Man Page
flow-capture(1) General Commands Manual flow-capture(1)NAME
flow-capture -- Manage storage of flow file archives by expiring old data.
SYNOPSIS
flow-capture [-hu] [-b big|little] [-C comment] [-c flow_clients] [-d debug_level] [-D daemonize] [-e expire_count] [-f fil-
ter_fname] [-F filter_definition] [-E expire_size] [-n rotations] [-N nesting_level] [-p pidfile] [-R rotate_program] [-S
stat_interval] [-t tag_fname] [-T active_def|active_def,active_def ...] [-V pdu_version] [-z z_level] -w workdir [-x xlate_fname] [-X
xlate_definition] localip/remoteip/port
DESCRIPTION
The flow-capture utility will receive and store NetFlow exports to disk. The flow files are rotated rotationstimes per day and expiration
of old flow files can be configured by number of files or total space utilization. Files are stored in workdir and can optionally be
stored in additional levels of directories. Active files created by flow-capture begin with 'tmp'. Files that are complete begin with
'ft'.
When the remoteip is configured only flows from that exporter will be processed, this is the most secure and recommended configuration.
When the localip is configured flow-capture will only process flows sent to the localip IP address. If remoteip is 0 (not configured)
flows from any source IP address are accepted. Multiple non aggregated PDU versions may be accepted at once to support Cisco's Catalyst
6500 NetFlow implementation which exports from both the supervisor and MSFC with the same IP address and same port but different export
versions. In this case the exports will be stored in the format specified by pdu_version or whichever export type is received first.
NetFlow exports are UDP and do not employ congestion control or a retransmission mechanism. If the server flow-capture is configured on is
too busy, or the network is congested or lossy NetFlow exports will be lost. An estimate of lost flows is recorded in the flow files, and
logged via syslog. Most servers will provide a count of dropped packets due to full socket buffers via the netstat utility. For example
netstat -s | grep full will provide a count of UDP packets dropped due to full socket buffers. If this is a persistent occurrence either
flow-capture will need a larger server or the compression level should be decreased with -z.
A SIGHUP signal will cause flow-capture to close the current file and create a new one.
A SIGQUIT or SIGTERM signal will cause flow-capture to close the current file and exit.
OPTIONS -b big|little
Byte order of output.
-c flow_clients
Enable flow_clients TCP clients. When libwrap is available the client must be in a permit list for the service flow-capture-
client.
-C Comment
Add a comment.
-d debug_level
Enable debugging.
-e expire_count
Retain the maximum number of files so that the total file count is less than expire_count. Defaults to 0 (do not expire).
-E expire_size
Retain the maximum number of files so that the total storage is less than expire_size. The letters b,K,M,G can be used as multi-
pliers, ie 16 Megabytes is 16M. Default to 0 (do not expire).
-f filter_fname
Filter list filename. Defaults to /etc/flow-tools/cfg/filter.
-F filter_definition
Select the active definition. Defaults to default.
-h Display help.
-n rotations
Configure the number of times flow-capture will create a new file per day. The default is 95, or every 15 minutes.
-N nesting_level
Configure the nesting level for storing flow files. The default is 0.
-3 YYYY/YYYY-MM/YYYY-MM-DD/flow-file
-2 YYYY-MM/YYYY-MM-DD/flow-file
-1 YYYY-MM-DD/flow-file
0 flow-file
1 YYYY/flow-file
2 YYYY/YYYY-MM/flow-file
3 YYYY/YYYY-MM/YYYY-MM-DD/flow-file
-p pidfile
Configure the process ID file. Use - to disable pid file creation.
-R rotate_program
Execute rotate_program with the first argument as the flow file name after rotating it.
-S stat_interval
When configured flow-capture will log a timestamped message every stat_interval minutes indicating counters such as the number of
flows received, packets processed, and lost flows.
-t tag_fname
Load tags from tag_name
-T active_def|active_def,active_def...
Use active_def as the active tag definition(s).
-u Preserve inherited umask. By default the umask will be set to 0022.
-V pdu_version
Use pdu_version format output.
1 NetFlow version 1 (No sequence numbers, AS, or mask)
5 NetFlow version 5
6 NetFlow version 6 (5+ Encapsulation size)
7 NetFlow version 7 (Catalyst switches)
8.1 NetFlow AS Aggregation
8.2 NetFlow Proto Port Aggregation
8.3 NetFlow Source Prefix Aggregation
8.4 NetFlow Destination Prefix Aggregation
8.5 NetFlow Prefix Aggregation
8.6 NetFlow Destination (Catalyst switches)
8.7 NetFlow Source Destination (Catalyst switches)
8.8 NetFlow Full Flow (Catalyst switches)
8.9 NetFlow ToS AS Aggregation
8.10 NetFlow ToS Proto Port Aggregation
8.11 NetFlow ToS Source Prefix Aggregation
8.12 NetFlow ToS Destination Prefix Aggregation
8.13 NetFlow ToS Prefix Aggregation
8.14 NetFlow ToS Prefix Port Aggregation
1005 Flow-Tools tagged version 5
-w workdir
Work in workdir.
-x xlate_fname
Translation config file name. Defaults to /etc/flow-tools/cfg/xlate.c fg
-X xlate_definition
Translation definition. Defaults to default.
-z z_level
Configure compression level to z_level. 0 is disabled (no compression), 9 is highest compression.
EXAMPLES
Receive flows from the exporter at 10.0.0.1 port 9800. Maintain 5 Gigabytes of flow files in /flows/krc4. Mask the source and destination
IP addresses contained in the flow exports with 255.255.248.0.
flow-capture -w /flows/krc4 -m 255.255.248.0 -E5G 0/10.0.0.1/9800
Receive flows from any exporter on port 9800. Do not perform any flow file space management. Store the exports in /flows/krc4. Emit a
stat log message every 5 minutes.
flow-capture -w /flows/krc4 0/0/9800 -S5
BUGS
Empty directories are not removed.
FILES
Configuration files:
Tag - /etc/flow-tools/cfg/tag.cfg.
Filter - /etc/flow-tools/cfg/filter.cfg.
Xlate - /etc/flow-tools/cfg/xlate.cfg.
AUTHOR
Mark Fullmer maf@splintered.net
SEE ALSO flow-tools(1)flow-capture(1)