11-09-2010
1. Extract the all data you need just one time, and store it in a file.
2. If you run 14 simultaneous processes it will take just one day.
3. The format of tar is an established standard. Nobody is hiding it from you.
google for 'gnu coreutils' and download the source code. In the tarball is the source for tar itself.
The problem is this:
tar stands for 'tape archive'. To get to file #99 in a tar file you have to actually completely read though files 1 -> 98 first. There is no file metadata like on a hard drive that allows direct access. The same applies once you get to the file you want. You have to read thru the file itself until you get to a given record. You cannot "fast forward" to a given record or a given file.
Modify the program to run in as many separate processes as is possible on your system without overloading it. Each process opens one tar file, gets one data set, writes the "variables" to disk or even better to a database table.
So, you now have the six values for every day. And in one small file.
Do you know how to extract a given file? What command are you using to get the grib file from a tarball?
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi, ive just been given a HP UX c180 to play with, along with a few external hd`s, can anyone recommend me a good place to start learning unix please as i havent got a clue what im doing when it comes to unix.
Either web links or book recomendations would be great.
I would like to set it up with... (1 Reply)
Discussion started by: Leviathan40
1 Replies
2. UNIX for Dummies Questions & Answers
I'm trying to write a script, named "worfo" which will read a file in and return, on the screen, the number of words in the file. Also, I am looking to do several extras with this script. I need to allow it to accept the option "-n" which will use all non-alpha characters to delimit words. Also,... (2 Replies)
Discussion started by: adawg1283
2 Replies
3. AIX
I have moved the etc/passwd file and we are now unable to get in to Unix - any suggestions? (2 Replies)
Discussion started by: vbagwell
2 Replies
4. UNIX for Dummies Questions & Answers
hi there
i am very new to the world of CRON. i would like to know from below the ground up how i can learn abouot cron. where the best tutorials lie. i have an assignment that requires i set up a cron job to download a price file. am i in control of where the price file is downloaded to? (eg to... (2 Replies)
Discussion started by: buzzby
2 Replies
5. Shell Programming and Scripting
hi, i am new with scripts and have a little problem.
i have a file with dates in the form YYYYMMDD, in average about 40 days per year.
now i shall cut those days with cdo selday from monthly files.
the script shall look up the days, forward them to the cdo operator and write the new grib... (1 Reply)
Discussion started by: jumkae
1 Replies
6. UNIX for Dummies Questions & Answers
Hi
I normally backup to tape using the following command in Solaris 9 on SUN Enterprise 3500 server:
tar cvf /dev/rmt/0n data
but after a power cut, the server went down without being shutdown since the UPS failed. Now when I issue the tar command I get the root (/) being full looks like it... (6 Replies)
Discussion started by: rahmantanko
6 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I want to untar all log files from following tarred directory hierarchy
Log_files.tar.gz/subject*.tar.gz/project*/*.log
It means there are subject1.tar.gz to subject9.tar.gz and in those tarred subect directories there are project1 - project5 directories and in those directories there... (2 Replies)
Discussion started by: rv_trojan
2 Replies
8. Shell Programming and Scripting
lllll (34 Replies)
Discussion started by: risarose87
34 Replies
9. Shell Programming and Scripting
Hello,
I have massive number of big files that needed to be counted for the total number of lines (> 100x millions) each. I want the file name attached to the count results so that they are aligned nicely matching name and counts.
I could do each file at a time, which will take hours to finish,... (8 Replies)
Discussion started by: yifangt
8 Replies
LEARN ABOUT DEBIAN
git-tar-tree
GIT-TAR-TREE(1) Git Manual GIT-TAR-TREE(1)
NAME
git-tar-tree - Create a tar archive of the files in the named tree object
SYNOPSIS
git tar-tree [--remote=<repo>] <tree-ish> [ <base> ]
DESCRIPTION
THIS COMMAND IS DEPRECATED. Use git archive with --format=tar option instead (and move the <base> argument to --prefix=base/).
Creates a tar archive containing the tree structure for the named tree. When <base> is specified it is added as a leading path to the files
in the generated tar archive.
git tar-tree behaves differently when given a tree ID versus when given a commit ID or tag ID. In the first case the current time is used
as modification time of each file in the archive. In the latter case the commit time as recorded in the referenced commit object is used
instead. Additionally the commit ID is stored in a global extended pax header. It can be extracted using git get-tar-commit-id.
OPTIONS
<tree-ish>
The tree or commit to produce tar archive for. If it is the object name of a commit object.
<base>
Leading path to the files in the resulting tar archive.
--remote=<repo>
Instead of making a tar archive from local repository, retrieve a tar archive from a remote repository.
CONFIGURATION
tar.umask
This variable can be used to restrict the permission bits of tar archive entries. The default is 0002, which turns off the world write
bit. The special value "user" indicates that the archiving user's umask will be used instead. See umask(2) for details.
EXAMPLES
git tar-tree HEAD junk | (cd /var/tmp/ && tar xf -)
Create a tar archive that contains the contents of the latest commit on the current branch, and extracts it in /var/tmp/junk directory.
git tar-tree v1.4.0 git-1.4.0 | gzip >git-1.4.0.tar.gz
Create a tarball for v1.4.0 release.
git tar-tree v1.4.0^{tree} git-1.4.0 | gzip >git-1.4.0.tar.gz
Create a tarball for v1.4.0 release, but without a global extended pax header.
git tar-tree --remote=example.com:git.git v1.4.0 >git-1.4.0.tar
Get a tarball v1.4.0 from example.com.
git tar-tree HEAD:Documentation/ git-docs > git-1.4.0-docs.tar
Put everything in the current head's Documentation/ directory into git-1.4.0-docs.tar, with the prefix git-docs/.
GIT
Part of the git(1) suite
Git 1.7.10.4 11/24/2012 GIT-TAR-TREE(1)