Paul Eggert [Wed, 11 Jan 2012 19:37:10 +0000 (11:37 -0800)]
tar: fix core dump with --index-file=bad_path
* src/tar.c (decode_options): Exit cleanly if index file won't open.
Problem reported by Khanh-Dang Nguyen Thu Lam in
<http://lists.gnu.org/archive/html/bug-tar/2012-01/msg00018.html>.
Paul Eggert [Fri, 6 Jan 2012 20:38:55 +0000 (12:38 -0800)]
tar: don't assume O_NONBLOCK is benign on regular files
On Data Migration Facility (DMF), High Peformance Storage System (HPSS),
and presumably other file systems based on hierarchical storage, opening
a regular file with O_NONBLOCK can cause later reads to fail with
errno == EAGAIN. We need the O_NONBLOCK to avoid some security races.
Work around the problem by using fcntl to clear the O_NONBLOCK
flag if I/O fails with that errno value.
Problem reported by Vitezslav Cizek in
<http://lists.gnu.org/archive/html/bug-tar/2012-01/msg00000.html>.
* src/common.h (blocking_read, blocking_write): New decls.
* src/misc.c (blocking_read, blocking_write): New functions.
* src/compare.c (process_rawdata):
* src/create.c (dump_regular_file):
* src/extract.c (extract_file):
* src/sparse.c (sparse_scan_file, sparse_extract_region):
The regression was introduced by 8f390db9. This patch implements additional
option --skip-old-files, which silently skips members which would cause
writing over existing files, and restores --keep-old-files to its traditional
behavior.
* NEWS: Update.
* configure.ac: Update.
* doc/tar.texi: Document the changes.
* src/common.h (SKIP_OLD_FILES): New old_files mode.
* src/extract.c (maybe_recoverable): Restore KEEP_OLD_FILES behavior.
Handle SKIP_OLD_FILES.
* src/tar.c: New option --skip-old-files.
* tests/extrac18.at: New file.
* tests/extrac19.at: New file.
* tests/Makefile.am: Add new test cases.
* tests/testsuite.at: Likewise.
Fix operation of --verify in conjunction with --listed-incremental
* src/common.h (clear_directory_table): New proto.
* src/incremen.c (clear_directory_table): New function.
* src/compare.c (diff_dumpdir): Take a pointer to struct
tar_stat_info as argument.
Initialize its fd.
(diff_archive): Update call to diff_dumpdir.
(verify_volume): Call clear_directory_table.
* scripts/tar-snapshot-edit: Update Perl syntax to work
correctly with more recent versions of Perl. (The original
code worked with in the v5.8 timeframe but not with
Perl v5.10.1 and later.)
Add a "-c" option to check the snapshot file for invalid
field values.
Handle NFS indicator character ("+") in version 0 and 1 files.
Preserve the original header/version line when editing version 1
or 2 files.
Tweak output formatting.
* doc/tar-snapshot-edit.texi: Update documentation.
Paul Eggert [Sat, 13 Aug 2011 17:19:07 +0000 (10:19 -0700)]
tar: --owner and --group names and numbers
The --owner and --group options now accept operands of the form
NAME:NUM, so that you can specify both symbolic name and numeric
ID for owner and group. Also, in these options, NAME no longer
needs to be present in the current host's user and group
databases; this implements Debian enhancement request 136231
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=136231> reported
by Mark W. Eichin, communicated by Thayne Harbaugh to bug-tar in
<http://lists.gnu.org/archive/html/bug-tar/2011-08/msg00001.html>.
* NEWS, doc/tar.texi (Option Summary, override): Document enhancement.
* src/common.h (group_name_option, owner_name_option): New decls.
* src/create.c (start_header): Don't assume owner and group names
are in current host database.
* src/tar.c (parse_owner_group): New function, for parsing NAME:NUM.
(parse_opt): Use it.
(decode_options): Initialize owner_name_option, group_name_option.
* tests/owner.at: New file, to test this enhancement.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Include it.
* src/tar.c (add_exclude_array): Take exclude option filags in its
second argument.
(parse_opt): Register backup_file_table with the EXCLUDE_WILDCARDS flag.
* tests/exclude07.at: New testcase.
* tests/Makefile.am (TESTSUITE_AT): Add testsuite.at.
* tests/testsuite.at: Include exclude07.at.
Paul Eggert [Tue, 7 Jun 2011 23:07:08 +0000 (16:07 -0700)]
tar: simplify birthtime calculations
* src/extract.c (same_birthtime): Remove.
(extract_link, apply_delayed_links): Go back to using timestamp_cmp
instead of same_birthtime, as this is a bit simpler. This fix relies
on an up-to-date gnulib.
Paul Eggert [Tue, 7 Jun 2011 22:16:55 +0000 (15:16 -0700)]
tar: use birthtime rather than ctime when checking identity
Without this fix, tar would sometimes mishandle the extraction of
hard links to symbolic links. Problem reported in
<http://lists.gnu.org/archive/html/bug-tar/2011-06/msg00000.html>
by Andreas Stolcke.
* src/extract.c (struct delayed_link): Change ctime member to birthtime.
(same_birthtime): New function.
(create_placeholder_file, extract_link, apply_delayed_links):
Use it to compare birthtimes (when available) instead of ctime.
Paul Eggert [Thu, 26 May 2011 19:09:52 +0000 (12:09 -0700)]
* doc/tar.texi (Old Options): Clarify distinction from short options.
Adjust other parts of the manual to use examples that parse the
same regardless of whether "-" is in front of the option clump. See
<http://lists.gnu.org/archive/html/bug-tar/2011-05/msg00022.html>.
Paul Eggert [Mon, 7 Mar 2011 23:40:32 +0000 (15:40 -0800)]
--atime-preserve=replace: fix correctness and performance bugs
reported by Eric Blake in
<http://lists.gnu.org/archive/html/bug-tar/2011-03/msg00000.html>.
* src/compare.c (diff_file): Do not restore atime of size-zero files.
* src/create.c (dump_file0): Likewise. Also, do not restore atime
when fd is zero, because that indicates a file we haven't opened.
Paul Eggert [Wed, 16 Feb 2011 21:47:00 +0000 (13:47 -0800)]
tar: if (p) free (p); -> free (p);
There is no longer (since SunOS 4) any need to guard against
free (NULL), so replace each "if (p) free (p);" with "free (p);".
From Jim Meyering in
<http://lists.gnu.org/archive/html/bug-tar/2011-01/msg00026.html>.
* src/incremen.c (scan_directory, read_directory_file): As above.
(try_purge_directory): Likewise.
* src/list.c (read_header): Likewise.
* src/misc.c (assign_string): Likewise.
Correctly store long sparse file names in PAX archives.
* src/sparse.c (pax_dump_header_1): Make sure the created header name is
shorter than NAME_FIELD_SIZE bytes.
* tests/sparse04.at: New testcase.
* tests/Makefile.am (TESTSUITE_AT): Add sparse04.at.
* tests/testsuite.at: Include sparse04.at.
* NEWS: Update.
Paul Eggert [Mon, 6 Dec 2010 22:26:57 +0000 (14:26 -0800)]
tests: make the truncate test smaller and less buggy (tiny change)
Reported by Solar Designer in
<http://lists.gnu.org/archive/html/bug-tar/2010-12/msg00003.html>.
* tests/truncate.at: Use a smaller test case, and make its
race condition less likely.
Paul Eggert [Sat, 27 Nov 2010 03:35:20 +0000 (19:35 -0800)]
tests: skip SIGPIPE-dependent tests in environments ignoring SIGPIPE
Problem reported by Sven Joachim in
<http://lists.gnu.org/archive/html/bug-tar/2010-11/msg00043.html>.
* tests/remfiles01.at: Use AT_SIGPIPE_PREREQ.
* tests/sigpipe.at: Likewise.
* tests/testsuite.at (AT_SIGPIPE_PREREQ): New macro.
Paul Eggert [Thu, 25 Nov 2010 07:07:46 +0000 (23:07 -0800)]
tar: work around NetBSD and Tru64 symlink incompatibility with POSIX
Problem reported by Bruno Haible in
<http://lists.gnu.org/archive/html/bug-gnulib/2010-11/msg00306.html>.
* src/extract.c (maybe_recoverable): Also treat EFTYPE (if defined)
and ENOTSUP like ELOOP.
Paul Eggert [Tue, 23 Nov 2010 20:57:47 +0000 (12:57 -0800)]
tar: work around FreeBSD symlink incompatibility with POSIX
* src/extract.c (maybe_recoverable): Treat EMLINK like ELOOP, for
FreeBSD. Problem reported by Christian Weisgerber in
<http://lists.gnu.org/archive/html/bug-tar/2010-11/msg00080.html>.
Paul Eggert [Tue, 23 Nov 2010 06:50:58 +0000 (22:50 -0800)]
* src/names.c: tar: fix bug with --one-file-system --listed-incremental
Problem (and idea for fix) reported by Martin Weigel
<http://lists.gnu.org/archive/html/bug-tar/2010-11/msg00071.html>.
* src/common.h (is_individual_file): Remove decl.
* src/create.c (dump_file0): Replace "is_individual_file (p)"
with "top_level".
* src/incremen.c (procdir): Replace "!is_individual_file
(name_buffer)" with "st->parent". Fix bug with --one-file-system
and --listed-incremental.
* src/names.c (individual_file_table, register_individual_file):
(is_individual_file): Remove. All uses removed.
Paul Eggert [Mon, 22 Nov 2010 23:18:10 +0000 (15:18 -0800)]
scripts: fix option parsing
Problem reported by Dennis Wydra in
<http://lists.gnu.org/archive/html/bug-tar/2010-11/msg00082.html>.
* scripts/backup.in: Accept "-l LEVEL". Be more systematic about
backslashes inside ``; it shouldn't matter for modern shells but
it might matter for older ones.
* scripts/restore.in: Likewise.
* scripts/backup.in: Adjust implementation of -t/--time to match
the new implementation of -l/--level.
Paul Eggert [Mon, 15 Nov 2010 08:07:53 +0000 (00:07 -0800)]
tar: fix --verify option, which broke in 1.24
* NEWS: Document this.
* src/compare.c (verify_volume): Decode the header before invoking
diff_archive, as diff_archive no longer does this as of the
2010-06-28 commit. Also, don't try to invoke diff_archive on a
zero block.
* tests/Makefile.am (TESTSUITE_AT): Add verify.at.
* tests/testsuite.at: Include verify.at.
* tests/verify.at: New file.
Run alternative decompression programs if the principal one is not available.
Some compression programs are able to handle various compression formats
(e.g. `gzip' can decompress files created by `compress', `xz' is able
to handle lzma, etc.) Tar tries to use such programs for decompression
if the principal decompressor cannot be started.
* src/buffer.c (compress_type): Swap ct_none and ct_tar.
(archive_compression_type): New static variable.
(zip_magic): Remove program and option fields.
(zip_program): New structure and static.
(compress_program): Remove macro.
(find_zip_program): New static function.
(first_decompress_program,next_decompress_program): New functions.
(open_compressed_archive): Set archive_compression_type instead of
use_compress_program_option.
* src/common.h (first_decompress_program)
(next_decompress_program): New functions.
(WARN_DECOMPRESS_PROGRAM): New flag.
(WARN_VERBOSE_WARNINGS): Include WARN_DECOMPRESS_PROGRAM.
* src/warning.c (warning_args): Add "decompress-program".
(warning_types): Add WARN_DECOMPRESS_PROGRAM.
* src/system.c (run_decompress_program): New function.
(sys_child_open_for_uncompress): Use run_decompress_program
instead of calling execlp directly.
Paul Eggert [Tue, 2 Nov 2010 08:04:33 +0000 (01:04 -0700)]
tests: fix some issues with signals, timestamps, "test" typo
* tests/extrac17.at: Add --warning=no-timestamp, to avoid
bogus warning due to NFS clock skew.
* tests/remfiles01.at: Discard diagnostics that some shells
generate about broken pipes.
* tests/sigpipe.at: Likewise.
* tests/remfiles01.at: Fix typo: "test $EC" was written where
"test $EC -ne 0" was intended.
Paul Eggert [Thu, 28 Oct 2010 05:25:18 +0000 (22:25 -0700)]
tar: don't cross struct member boundaries with OLDGNU_MAGIC
* src/create.c (write_gnu_long_link, start_header): Access
header->buffer + offsetof (struct posix_header, magic), instead of
header->header.magic, when reading or writing the OLDGNU_MAGIC
pattern. The code violates the C standard without this change,
and GCC warns about this if fortify checking is enabled. It's not
a bug on traditional (i.e., non-debugging) platforms, but it does
violate the C standard so it should be fixed. Problem originally
reported by John Emil Karlson in
<http://lists.gnu.org/archive/html/bug-tar/2010-04/msg00023.html>.
* src/list.c (decode_header): Likewise.
Paul Eggert [Thu, 28 Oct 2010 03:25:18 +0000 (20:25 -0700)]
tests: port to sh variants that squirrel away file descriptors
OpenBSD /bin/sh, and some other sh variants, squirrel away file
descriptors before closing them. For example, for "cat 3<&-" they
first dup file descriptor 3 to a fd that is 10 or greater, then
close 3 (because if "cat" had been a builtin command like ":" then
they would have wanted to avoid the fork and restore the fd after
":" finished); and they treat ordinary (forking) commands the same
as builtin commands. This approach fails after "ulimit -n 10".
Work around this deficiency by closing the file descriptors before
invoking ulimit. Problem reported by Christian Weisgerber in
<http://lists.gnu.org/archive/html/bug-tar/2010-10/msg00041.html>;
solution suggested by Jilles Tjoelker in
<http://article.gmane.org/gmane.comp.shells.dash/415>.
* tests/extrac11.at (scarce file descriptors): Close file
descriptors before invoking ulimit -n.
Paul Eggert [Wed, 27 Oct 2010 01:12:33 +0000 (18:12 -0700)]
tests: port to Solaris diff
* tests/extrac13.at: Don't assume that "diff -c" outputs nothing
when there are no differences. This is not true on Solaris,
where it outputs "No differences encounted".
Paul Eggert [Wed, 27 Oct 2010 00:57:44 +0000 (17:57 -0700)]
tar: fix -x --overwrite bug (no --dereference, ! O_NOFOLLOW)
This bug was discovered on Solaris 8. On older hosts lacking
O_NOFOLLOW, tar -x --overwrite (without --dereference) follows
symbolic links, causing the "extract over symlinks" test to fail.
* src/extract.c (open_output_file): If O_NOFOLLOW is needed but
does not work, check for a symlink separately.
Paul Eggert [Tue, 26 Oct 2010 22:04:15 +0000 (15:04 -0700)]
tar: don't use "((" in shell scripts
* tests/extrac11.at: Replace "((" with "( (" in shell scripts, as
"((" is not portable to the Korn shell, and POSIX 1003.1-2008 says
that "((" is not portable.
* src/list.c (transform_stat_info): New function. Split off from
decode_header.
(read_and): Call transform_stat_info right before do_something,
and after deciding if we should proceed with this member name,
so that name matching occurs before name transformation.
* tests/extrac17.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Add extrac17.at
* tests/testsuite.at: Include extrac17.at.
Paul Eggert [Tue, 26 Oct 2010 03:20:19 +0000 (20:20 -0700)]
tar: fix bug with -C and extracting directories
Problem reported by Denis Excoffier in
<http://lists.gnu.org/archive/html/bug-tar/2010-10/msg00034.html>.
* src/extract.c (extract_dir): Use mkdirat, not mkdir.
* tests/extrac16.at: New file, to test for this bug.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Include it.
This file is a placeholder. It will be replaced with the actual ChangeLog
by make dist. Run make ChangeLog if you wish to create it earlier.
* configure.ac, NEWS: Version 1.24
* bootstrap: Restore tar-specific code lost during last
sync from gnulib (241b72ffad).
* src/misc.c (chdir_do): Remove unused automatic variable.
Paul Eggert [Sat, 16 Oct 2010 05:26:14 +0000 (22:26 -0700)]
tar: use more-accurate diagnostic when intermediate mkdir fails
Without this change, if tar tried to extract a file A/B/C, noticed
that A/B didn't exist, attempted to mkdir A/B, and the mkdir
failed, it did not diagnose the mkdir failure, but simply reported
the failure to open A/B/C. This sometimes led to confusion
because it wasn't clear what tar was trying to do, in particular
that tar tried to mkdir A/B. With this patch, tar issues two
diagnostics in this case: one for A/B and the other for A/B/C.
Problem reported by Hauke Laging in
<http://lists.gnu.org/archive/html/bug-tar/2010-10/msg00020.html>.
* gnulib.modules: Remove faccessat.
* src/extract.c (make_directories): New arg INTERDIR_MADE.
Diagnose mkdir failure. Return 0 on success, nonzero on failure,
as opposed to nonzero iff some directory was created. All callers
changed. Simplify the code when mkdir fails, by checking whether
the desired file exists unless errno==EEXIST: this is more robust.
* tests/extrac15.at: New test, to check this.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Include it.
Paul Eggert [Sat, 16 Oct 2010 04:52:40 +0000 (21:52 -0700)]
tests: port to NFS file servers with clock skew
Several of the tests assumed that a newly created file cannot
have a time stamp dated in the future. This assumption is not
true when files are served by a remote host whose clock is
slightly in advance of ours. Fix the problems that I observed
when running "make check" a couple of times on such a server.
Paul Eggert [Mon, 11 Oct 2010 23:21:41 +0000 (16:21 -0700)]
gnulib: sync from latest gnulib, notably bootstrap and parse-datetime
* bootstrap: Sync from gnulib.
* doc/.gitignore: Rename getdate.texi to parse-datetime.texi.
* doc/Makefile.am (tar_TEXINFOS): Likewise.
* doc/tar.texi (Top): Adjust to renaming of getdate to parse-datetime.
* gnulib.modules: Likewise. Also, remove ftruncate (now obsolete).
And add inttostr (we missed this dependency).
* src/tar.c: Include parse-datetime.h, not getdate.h. All calls
to get_date replaced with parse_datetime.
Paul Eggert [Fri, 24 Sep 2010 20:13:25 +0000 (13:13 -0700)]
tar: -x -C symlink fix
* src/misc.c (chdir_do): Don't use O_NOFOLLOW when opening the
argument to -C. This is for consistency with "tar -c -C FOO", and
matches the new documentation.
* tests/extrac14.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Include it.
Paul Eggert [Fri, 24 Sep 2010 02:41:05 +0000 (19:41 -0700)]
tar: --dereference consistency
This closes another race condition, that occurs when overwriting a
symlink with a regular file.
* NEWS (--dereference consistency): New section.
* doc/tar.texi (Option Summary): Describe new --deference behavior.
(dereference): Likewise. Remove discussion that I didn't follow,
even before --dereference was changed.
* src/common.h (deref_stat, set_file_atime): Adjust signatures.
* src/compare.c (diff_file, diff_multivol): Respect open_read_flags
instead of rolling our own flags. This implements the new behavior
for --dereference.
(diff_file, diff_dumpdir): Likewise, for fstatat_flags.
* src/create.c: Adjust to set_file_atime signature change.
* src/extract.c (mark_after_links, file_newer_p, extract_dir):
Likewise.
* src/incremen.c (try_purge_directory): Likewise.
* src/misc.c (maybe_backup_file): Likewise.
* src/extract.c (file_newer_p): New arg STP. All callers changed.
(maybe_recoverable): New arg REGULAR. All callers changed.
Handle the case of overwriting a symlink with a regular file,
when --overwrite is specified but --dereference is not.
(open_output_file): Add O_CLOEXEC, O_NOCTTY, O_NONBLOCK for
consistency with file creation. Add O_NOFOLLOW if
overwriting_old_files && ! dereference_option.
* src/incremen.c (update_parent_directory): Use fstat, not fstatat;
there's less to go wrong.
* src/misc.c (deref_stat): Remove DEREF arg. All callers changed.
Instead, use fstatat_flags.
(set_file_atime): Remove ATFLAG arg. All callers changed.
Instead, use fstatat_flags.
* src/names.c, src/update.c: Adjust to deref_stat signature change.
* src/tar.c (get_date_or_file): Use stat, not deref_stat, as this
is not a file to be archived.
* tests/Makefile.am (TESTSUITE_AT): Add extrac13.at.
* tests/extrac13.at: New file.
* tests/testsuite.at: Include it.
Paul Eggert [Mon, 20 Sep 2010 18:34:33 +0000 (11:34 -0700)]
tar: do not crash with --listed-incremental
Problem reported by Frantisek Hanzlik in
<https://bugzilla.redhat.com/635318> via Kamil Dudka in
<http://lists.gnu.org/archive/html/bug-tar/2010-09/msg00066.html>.
I don't understand this code either, but Sergey can take a look at
this patch, and perhaps install a better one, when he has the time.
* src/incremen.c (append_incremental_renames): Don't actually append
anything to DIR if DIR is null.
Paul Eggert [Sun, 19 Sep 2010 06:37:45 +0000 (23:37 -0700)]
tar: prefer openat-style functions
This change replaces traditional functions like 'open' with the
POSIX.1-2008 functions like 'openat'. Mostly this is an internal
refactoring change, in preparation for further changes to close
some races.
* gnulib.modules: Add faccessat, linkat, mkfifoat, renameat, symlinkat.
Remove save-cwd.
* src/Makefile.am (tar_LDADD): Add $(LIB_EACCESS).
* tests/Makefile.am (LDADD): Likewise.
* src/common.h (chdir_fd): New extern var.
* src/compare.c (diff_file, diff_multivol): Use openat instead of open.
* src/create.c (create_archive, restore_parent_fd): Likewise.
* src/extract.c (create_placeholder_file): Likewise.
* src/names.c (collect_and_sort_names): Likewise.
* src/update.c (append_file): Likewise.
* src/compare.c (diff_symlink): Use readlinkat instead of readlink.
* src/compare.c (diff_file): Use chdir_fd instead of AT_FDCWD.
* src/create.c (subfile_open, dump_file0): Likewise.
* src/extract.c (fd_chmod, fd_chown, fd_stat, set_stat):
(repair_delayed_set_stat, apply_nonancestor_delayed_set_stat):
Likewise.
* src/extract.c (mark_after_links, file_newer_p, extract_dir):
(extract_link, apply_delayed_links):
Use fstatat rather than stat or lstat.
* src/misc.c (maybe_backup_file, deref_stat): Likewise.
* src/extract.c (make_directories): Use mkdirat rather than mkdir.
Use faccessat rather than access. This fixes a minor permissions
bug when tar is running setuid (who would want to do that?!).
(open_output_file): Use openat rather than open.
In the process, this removes support for Masscomp's O_CTG files,
which aren't compatible with openat's signature. Masscomp! Wow!
That's a blast from the past. As far as I know, that operating
system hasn't been supported for more than 20 years.
(extract_link, apply_delayed_links):
Use linkat rather than link.
(extract_symlink, apply_delayed_links):
Use symlinkat rather than symlink.
(extract_node): Use mknodat rather than mknod.
(extract_fifo): Use mkfifoat rather than mkfifo.
(apply_delayed_links): Use unlinkat rather than unlink or rmdir.
* src/misc.c (safer_rmdir, remove_any_file): Likewise.
* src/unlink.c (flush_deferred_unlinks): Likewise.
* src/extract.c (rename_directory): Use renameat rather than rename.
* src/misc.c (maybe_backup_file, undo_last_backup): Likewise.
* src/misc.c: Don't include <save-cwd.h>; no longer needed now
that we're using openat etc.
(struct wd): Add member fd. Remove members err and fd. All uses
changed.
(CHDIR_CACHE_SIZE): New constant.
(wdcache, wdcache_count, chdir_fd): New vars.
(chdir_do): Use openat rather than save_cwd. Keep the cache up
to date. This code won't scale well, but is good enough for now.
* src/update.c (update_archive): Use openat + fdopendir +
streamsavedir rather than savedir.
This file is a placeholder. It will be replaced with the actual ChangeLog
by make dist. Run make ChangeLog if you wish to create it earlier.
Paul Eggert [Fri, 17 Sep 2010 19:25:55 +0000 (12:25 -0700)]
tar: extract permissions for . last
* src/common.h (must_be_dot_or_slash): New decl.
* src/extract.c (mark_after_links): New function, taking code
that used to be in create_placeholder_file.
(create_placeholder_file): Use it.
(delay_set_stat): Always delay setting status for . and /.
* src/misc.c (must_be_dot_or_slash): Now extern.
* tests/extrac12.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Likewise.
Paul Eggert [Fri, 17 Sep 2010 17:43:35 +0000 (10:43 -0700)]
tar: extract symlink attributes, close some symlink-related races
* NEWS: Describe symlink-extraction improvements.
* src/extract.c (enum permstatus): Remove.
(fchmod, fchown): Define dummy replacement macros if the system
does not supply them.
(implemented): New function.
(struct delayed_set_stat): Remove members invert_permissions,
permstatus. They were too confusing, and tried to do too much
in too-little space. Instead, add members current_mode,
current_mode_mask, interdir, atflag. All users changed.
(struct delayed_link): Add members mode, atime, mtime, to support
platforms such as BSD where symlinks have these attributes.
All users changed.
(fd_chmod): Renamed from fdchmod. New argument atflag. Check for
operation not supported at run-time, not at configure-time. Put
fd argument first. All callers changed.
(fd_chown): Likewise, renaming from fdchown.
(fd_stat): Likewise, renaming from fdstat.
(set_mode): Remove args stat_info, cur_info, invert_permissions,
permstatus. Add args mode, mode_mask, current_mode, current_mode_mask,
atflag. All callers changed. Close some races. Use an easier-to
understand method for computing permissions. Work around POSIX
incompatibility in Linux fchmodat. Support extraction of symlink
modes, if the OS allows it.
(set_stat): Remove args cur_info, invert_permissions, permstatus.
Add args current_mode, current_mode_mask, interdir, atflag.
All callers changed. Close some races. Support extraction of
attributes on symlinks, if the OS allows it.
(delay_set_stat): Remove args invert_permissions, permstatus.
Add args current_mode, current_mode_mask, mode, atflag.
The ST arg can be null now, indicating that it's an intermediate
directory. All callers changed.
(repair_delayed_set_stat, apply_nonancestor_delayed_set_stat):
Close some races.
(extract_dir): Also be paranoid if only --same-permissions, due
to semantics of setgid and setuid directories on some hosts.
This closes a race on those hosts. Simplify calculation of
delay_set_stat arguments; the old code was truly strange and
probably wrong in some border cases.
(extract_dir, extract_file, extract_node, extract_fifo): Don't rely on
unspecified behavior in mode arg of open, mknod, etc. Instead,
mask out those bits when creating the file, and add them later via
fchmod or chmodat.
(open_output_file): file_name is now const. Add arg current_mode,
current_mode_mask. All callers changed. When overwriting old files,
refuse to overwrite something that is not a regular file, since
we're extracting a regular file.
(extract_file): Remove the FIXME comment. Whatever the protection
issues were, they should be fixed now. As a result of all the other
API changes, we now use fchmod etc. rather than chmod etc., closing
some races.
(create_placeholder_file, apply_delayed_links): Record desired
mode and times for symlinks, for OSes that support that.
Paul Eggert [Thu, 16 Sep 2010 18:02:08 +0000 (11:02 -0700)]
tar: tar -x without --incremental no longer sets atime again
* src/extract.c (set_stat): Use UTIME_OMIT rather than UTIME_NOW.
The UTIME_NOW was there only to emulate the previous behavior of
using the current time, and the previous behavior was there only
because before we started assuming POSIX.1-2008 there was no
portable way to get the effect of UTIME_NOW.
Paul Eggert [Thu, 16 Sep 2010 17:16:47 +0000 (10:16 -0700)]
tar: --atime-preserve fixes for races etc.
This patch fixes a race condition in the --atime-preserve=replace
option, which might cause tar to improperly follow a symbolic link.
It also drops the use of the _FIOSATIME ioctl of Solaris 2.x
and later, which loses resolution on time stamps. Modern Solaris
systems support full-resolution time stamps in the kernel, and
it's not worth the hassle of testing this call, useful only in
no-longer-supported Solaris variants.
Also, it undoes a change I recently introduced to the --compare
option, which caused it to not follow symbolic links unless the
--dereference option was also used. Quite possibly this change is
a good idea, but the old behavior was documented and the change
should not have been installed casually.
* configure.ac: Don't check for stropts.h and sys/filio.h.
* gnulib.modules: Add futimens, utimensat. Remove futimens.
* src/common.h (fd_utimensat): New decl.
* src/compare.c (diff_file, diff_multivol):
Don't use open_read_flags: those are for --create only.
* src/create.c (dump_file0): Adjust to set_file_atime changes.
Pass fstatat_flags to set_file_atime, so that symbolic links are
not followed inadvertantly.
* src/extract.c: Don't include utimens.h.
(set_stat): Use fd_utimensat ant UTIME_NOW rather than fdutimens.
* src/misc.c: Don't include utimens.h, stropts.h, sys/filio.h.
(fd_utimensat): New function.
(set_file_atime): Use it. New arg atflag, controlling symlink
handling. All callers changed.
Paul Eggert [Tue, 14 Sep 2010 20:33:21 +0000 (13:33 -0700)]
* configure.ac: tar: close some race conditions when extracting
* configure.ac: Check for fchmod and fchown. Don't check for utimes.
* src/extract.c (fdchmod, fdchown, fdstat): New functions.
(set_mode, set_stat): New arg FD. All callers changed.
This avoids some race conditions between closing a regular file
and setting its metadata, and it's a bit faster.
Paul Eggert [Mon, 13 Sep 2010 20:12:54 +0000 (13:12 -0700)]
tar: don't worry about fdopendir closing its argument
* NEWS: Don't mention dirfd; no longer needed.
* gnulib.modules: Remove dirfd.
* src/create.c (get_directory_entries): Remove the code dealing
with dirfd failures, as the new fdopendir replacement doesn't
close its argument so we don't need to call dirfd. See
<http://lists.gnu.org/archive/html/bug-gnulib/2010-09/msg00208.html>
and gnulib commit 970c9038e4cca46e1b037ae0a6d574dfae6a7327.
Paul Eggert [Sun, 12 Sep 2010 21:26:31 +0000 (14:26 -0700)]
tar: live within system-supplied limits on file descriptors
* NEWS: Note the change. Mention dirfd and fdopendir.
* gnulib.modules: Add dirfd and fdopendir. The code was already
using fdopendir; dirfd is a new need.
* src/common.h (open_searchdir_flags, get_directory_entries):
(subfile_open, restore_parent_fd, tar_stat_close): New decls.
(check_exclusion_tags): Adjust signature to match code change.
* src/create.c (IMPOSTOR_ERRNO): New constant.
(check_exclusion_tags): First arg is now a struct tar_stat_info
const *, not an fd. All callers changed.
(dump_regular_file, dump_file0): A zero fd represents an unused
slot, so play it safe if the fd member is zero here. A negative
fd represents the negation of an errno value, so play it safe and
do not assign -1 to fd merely because an open fails.
(open_failure_recover, get_directory_entries, restore_parent_fd):
(subfile_open): New functions. These help to recover from file
descriptor exhaustion.
(dump_dir, dump_file0): Use them.
(dump_file0): Use tar_stat_close instead of rolling our own close.
* src/incremen.c (scan_directory): Use get_directory_entries,
subfile_open, etc., to recover from file descriptor exhaustion.
* src/names.c (add_hierarchy_to_namelist): Likewise.
(collect_and_sort_names): A negative fd represents the negation
of an errno value, so play it safe and do not assign -1 to fd.
* src/tar.c (decode_options): Set open_searchdir_flags.
Add O_CLOEXEC to all the open flags.
(tar_stat_close): New function, which knows how to deal with
new convention for directory streams and file descriptors.
Diagnose 'close' failures.
(tar_stat_destroy): Use it.
* src/tar.h (struct tar_stat_info): New member dirstream.
fd now has the negative of an errno value, not merely -1, if
the file could not be opened, so that failures to reopen directories
are better-diagnosed later.
* tests/Makefile.am (TESTSUITE_AT): Add extrac11.at.
* tests/testsuite.at: Likewise.
* tests/extrac11.at: New file.
Paul Eggert [Wed, 8 Sep 2010 20:40:10 +0000 (13:40 -0700)]
tar: improve documentation of reliability and security issues
* doc/tar.texi (Reliability and security, Reliability):
(Permissions problems, Data corruption and repair, Race conditions):
(Security, Privacy, Integrity, Live untrusted data):
(Security rules of thumb): New nodes.
Paul Eggert [Mon, 6 Sep 2010 20:38:44 +0000 (13:38 -0700)]
tar: more reliable directory traversal when creating archives
* NEWS: Document this.
* gnulib.modules: Add openat, readlinkat.
* src/common.h (open_read_flags, fstatat_flags): New global variables.
(cachedir_file_p, dump_file, check_exclusion_tags, scan_directory):
Adjust to new signatures, described below.
(name_fill_directory): Remove.
* src/compare.c (diff_file, diff_multivol): Use open_read_flags.
* src/create.c (struct exclusion_tag): Exclusion predicates now take
a file descriptor, not a file name.
(add_exclusion_tag): Likewise. All uses changed.
(cachedir_file_p): Likewise.
(check_exclusion_tags): The directory is now a file descriptor,
not a file name. All uses changed. Use openat for better traversal.
(file_dumpable_p): Arg is now a struct stat, not a struct
tar_stat_info. All uses changed. Check the arg's file types too.
(dump_dir0, dump_dir, dump_file0, dump_file): Omit top_level and
parent_device args, since st->parent tells us that now. All uses
changed.
(dump_dir): Likewise. Also, omit fd arg for similar reasons.
Apply fdsavedir to a dup of the file descriptor, since we need a
file descriptor for openat etc. as well, and fdsavedir (perhaps
unwisely) consumes its file descriptor when successful.
Do not consume st->fd when successful; this simplifies the caller.
(create_archive): Allocate a file descriptor when retraversing
a directory, during incremental dumps.
(dump_file0): Use fstatat, openat, and readlinkat for better traversal.
When opening a file, use the result of fstat on the file descriptor
rather than the fstatat on the directory entry, to avoid some race
conditions. No need to reopen the directory since we now no longer
close it. Change "did we open the file?" test from 0 <= fd to
0 < fd since fd == 0 now represents uninitialized.
(dump_file): Now accepts struct tar_stat_info describing parent,
not parent_device. Also, accept basename and fullname of entry.
All uses changed.
* src/incremen.c (update_parent_directory): Accept struct
tar_stat_info for parent, not name. All callers changed.
Use fstatat for safer directory traversal.
(procdir): Accept struct tar_stat_info, not struct stat and
dev_t, for info about directory. All callers changed.
(scan_directory): Accept struct tar_stat_info, not name,
device, and cmdline, for info about directory. All callers
changed. Do not consume the file descriptor, since caller
might need it. Use fstatat and openat for safer directory
traversal; also, use fstat after opening to double-check.
(name_fill_directory): Remove.
* src/names.c (add_hierarchy_to_namelist): Accept struct
tar_stat_info instead of device and cmdline. All callers changed.
When descending into a subdirectory, use openat and fstat for
safer directory traversal.
(collect_and_sort_names): Use open and fstat for safer directory
traversal. Set up struct tar_stat_info for callee's new API.
* src/tar.c (decode_options): Initialize open_read_flags
and fstatat_flags.
(tar_stat_destroy): Close st->fd if it is positive (not zero!).
* src/tar.h (struct tar_stat_info): New members parent, fd.
* src/update.c (update_archive): Adjust to dump_file's API change.
* tests/filerem02.at: Ignore stderr since its contents now depend
on the file system implementation.
Paul Eggert [Sun, 5 Sep 2010 06:52:34 +0000 (23:52 -0700)]
tar: remove lint discovered by Sun C compiler
* src/common.h (WARN_ALL): Don't mask with 0xffffffff; on a 32-bit
host, 0xffffffff is of type 'unsigned int', which makes WARN_ALL
an unsigned int value that is too large to fit into an int, and
the C standard says that this has undefined behavior. The mask is
not needed, so omit it.
Paul Eggert [Sat, 4 Sep 2010 07:08:42 +0000 (00:08 -0700)]
tar: restore macros that are used in some cases
* src/tar.c (LOW_DENSITY_NUM, MID_DENSITY_NUM, HIGH_DENSITY_NUM):
Restore these macros, undoing the previous change to this file.
The macros are used after all, in some cases. Sorry about that.
Paul Eggert [Fri, 3 Sep 2010 03:45:52 +0000 (20:45 -0700)]
tar: remove unused macros
* src/create.c (UINTMAX_TO_CHARS): Remove; no longer used.
* src/tar.c (LOW_DENSITY_NUM, MID_DENSITY_NUM, HIGH_DENSITY_NUM):
Likewise.
* src/incremen.c (DIR_IS_NEW): Comment out; not used.
Mark this with a FIXME, since it looks like it should be used.
Paul Eggert [Thu, 26 Aug 2010 17:22:44 +0000 (10:22 -0700)]
tar: avoid assumptions about root access and chmod -w in test cases
* tests/extrac07.at, tests/extrac09.at, tests/listed03.at: Use
AT_UNPRIVILEGED_PREREQ, since this test requires non-root
privileges.
* tests/extrac07.at: Don't use "chmod -w", as POSIX says it's not
portable to start a chmod permissions-list with "-" as it may be
confused with an option. Use "chmod a-w" instead.
Paul Eggert [Thu, 26 Aug 2010 00:09:17 +0000 (17:09 -0700)]
tar: fix bug with -C and delayed setting of metadata
* src/common.h (chdir_current): New decl.
* src/extract.c (struct delayed_set_stat, struct delayed_link):
New member change_dir.
(delay_set_stat, create_placeholder_file): Set it.
(apply_nonancestor_delayed_set_stat, apply_delayed_links): Use it.
(extract_link): Check that the links are all relative to the same
directory.
(extract_archive): Restore the current directory after
apply_nonancestor_delayed_set_stat has possibly changed it.
* src/misc.c (chdir_current): New external var; this used to
be the private static variable 'previous' inside chdir_dir.
All uses changed.
* tests/Makefile.am (TESTSUITE_AT): New test extrac10.at.
* tests/extrac10.at: New file.
* tests/testsuite.at: Include it.
Paul Eggert [Wed, 25 Aug 2010 00:28:11 +0000 (17:28 -0700)]
tar: optimize -c --sparse when file is entirely sparse
* src/sparse.c (sparse_scan_file): If the file is entirely sparse,
that is, if ST_NBLOCKS is zero, don't bother scanning for nonzero
blocks. Idea by Kit Westneat, communicated by Bernd Schubert in
<http://lists.gnu.org/archive/html/bug-tar/2010-08/msg00038.html>.
Also, omit unnecessary lseek at start of file.
Paul Eggert [Tue, 24 Aug 2010 07:07:32 +0000 (00:07 -0700)]
tar: fix 1.23 Solaris regression related to PRIV_SYS_LINKDIR
The idea was suggested by Petr Sumbera in the thread starting here:
http://lists.gnu.org/archive/html/bug-tar/2010-08/msg00000.html
* src/extract.c (set_mode): Save the errno of the chmod that
failed, for the benefit of chmod_error_details. Do not bother
retrying chmod unless the mode suggests setuid is the issue.
(extract_archive): Remove redundant call to priv_set_remove_linkdir.
* src/system.c: Include priv-set.h.
(sys_spawn_shell, sys_child_open_for_compress):
(sys_child_open_for_uncompress, sys_exec_command):
(sys_exec_info_script, sys_exec_checkpoint_script):
Invoke priv_set_restore_linkdir before execv or execlp, so that
the subprocess has the same privileges that tar originally did.
Paul Eggert [Tue, 24 Aug 2010 02:12:25 +0000 (19:12 -0700)]
tar: handle files that occur multiple times but have link count 1
This patch was inspired by the following patch that addressed a
similar problem in GNU coreutils du:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;h=efe53cc72b599979ea292754ecfe8abf7c839d22
* src/common.h (name_count): New decl.
* src/create.c (trivial_link_count): New static var.
(create_archive): Initialize it.
(dump_hard_link, file_count_links): Use it, so that files with
link count 1 are handled correctly when they are found multiple times.
* src/names.c (allocated_entries): Renamed from allocated_names,
since the identifier's name was misleading. All uses changed.
(entries): Renamed from names. All uses changed.
(scanned): Renamed from name_index. All uses changed.
(name_count): New var.
(name_add_name): Increment it.
* tests/link04.at: New file.
* tests/testsuite.at: Add it.
* tests/Makefile.am (TESTSUITE_AT): Likewise.
Paul Eggert [Fri, 20 Aug 2010 08:08:52 +0000 (01:08 -0700)]
tar: change interdir_made from int to bool
* src/extract.c (maybe_recoverable, create_placeholder_file):
Change interdir_made from int * to bool *, since the flag has just
two values 0 and 1. All uses changed. This does not affect tar's
behavior.
Paul Eggert [Thu, 19 Aug 2010 11:13:01 +0000 (04:13 -0700)]
tar: update licenses to latest versions from www.gnu.org
* COPYING: Update to latest version; this is just minor formatting.
* doc/fdl.texi: Update from GFDL 1.2 to 1.3.
* doc/tar.texi: Adjust to new format of fdl.texi. Omit trailing
white space.
Paul R. Eggert [Sun, 18 Jul 2010 19:19:18 +0000 (12:19 -0700)]
tar: no need to report getcwd error if never using the result
* src/misc.c (struct wd): Rename 'saved' to 'err', with new semantics.
(chdir_arg, chdir_do): Adjust to new semantics. Do not report an
error merely because save_cwd fails; report an error only if
save_cwd's result is needed later.
* tests/extrac09.at: New file, to test for bug that was fixed.
* tests/testsuite.at: Include it.
* tests/Makefile.am (TESTSUITE_AT): Add it.
Paul R. Eggert [Fri, 16 Jul 2010 17:25:02 +0000 (10:25 -0700)]
tar: go back to absolutifying filenames in normalize_filename for now
* src/misc.c (normalize_filename): For now, go back to making
filenames absolute, even though this causes 'tar' to fail when
getcwd fails. However, do not attempt to resolve ".." as this
does not work with symlinks. Also, do the right thing with
leading file system prefixes and on hosts where // != /.
Allow for size suffixes in -L and --record-size options.
* src/tar.c (TAR_SIZE_SUFFIXES): New define.
(parse_opt): Allow for size suffixes in arguments to
-L and --record-size options.
* NEWS, doc/tar.texi: Update.
Paul R. Eggert [Thu, 15 Jul 2010 18:24:39 +0000 (11:24 -0700)]
tar: don't crash if getcwd fails
* src/extract.c: Don't include xgetcwd.h.
(extract_dir): stat "." rather than statting getcwd's output.
* src/misc.c (normalize_filename_x): Rewrite so as not to resolve
/../, which can't be done reliably in the presence of symlinks.
Don't reject valid names such as ".".
(normalize_filename): Don't make it absolute; that way, we don't
have to invoke xgetcwd which might fail. Don't bother to realloc
at the end, since that uses time and now saves little space.
(chdir_do): Don't crash if xgetcwd fails.
* tests/Makefile.am (TESTSUITE_AT): Add listed03.at.
* tests/listed03.at: New file.
* tests/testsuite.at: Include listed03.at.
Keep a detailed map of archive members stored in the record buffer.
A separate map (bufmap) provides information for creating
multi-volume continuation headers.
* src/buffer.c (bufmap): New struct.
(bufmap_head, bufmap_tail, inhibit_map): New variables.
(mv_begin_write): New function.
(mv_begin): Rename to mv_begin_read. Rewrite using mv_begin_write.
All callers changed.
(mv_total_size): Remove.
(bufmap_locate, bufmap_free, bufmap_reset): New functions.
(_flush_write): Update bufmap.
(close_archive): Free bufmap.
(add_chunk_header): Take a bufmap argument.
(gnu_add_multi_volume_header): Likewise.
(add_multi_volume_header): Likewise.
(_gnu_flush_write): Rewrite using bufmap.
(real_s_name, real_s_totsize)
(real_s_sizeleft)
(save_name, save_totsize, save_sizeleft): Removed. All
uses updated.
(mv_size_left): Update bufmap_head.
(mv_end): Rewrite.
(multi_volume_sync): Remove.
* src/common.h (mv_begin_write): New prototype.
(mv_begin): Rename to mv_begin_read.
* src/create.c: Use mv_begin_write instead of mv_begin.
Remove calls to mv_size_left and mv_end.
* src/sparse.c: Likewise.
projects / chaz / tar / log
