Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zstd wrongly uses utime instead of utimensat on BSDs #3748

Closed
asomers opened this issue Sep 7, 2023 · 0 comments · Fixed by #3952
Closed

zstd wrongly uses utime instead of utimensat on BSDs #3748

asomers opened this issue Sep 7, 2023 · 0 comments · Fixed by #3952
Assignees

Comments

@asomers
Copy link

asomers commented Sep 7, 2023

Describe the bug
zstd and unzstd attempts to set the output file's atime and mtime to match the input file's. This behavior cannot be disabled. On platforms that support it, unzstd uses utimensat. Otherwise, it uses utime. Unfortunately, utime only sets the timestamp's seconds field, not its nanoseconds field. That matters, because zstd wrongly determines which platforms support utimensat. For the BSDs, it simply hardcodes PLATFORM_POSIX_VERSION to a number that's too old (see

# if (defined(__APPLE__) && defined(__MACH__)) || defined(__SVR4) || defined(_AIX) || defined(__hpux) /* POSIX.1-2001 (SUSv3) conformant */ \
).

Tragically, this means that the output file's mtime is almost always a fraction of a second older than the input file's. So, for example, you can't create a Makefile rule like the below. make will always execute the command.

foo: foo.zst
        unzstd -fo foo foo.zst

To Reproduce

> touch /tmp/foo
> zstd /tmp/foo
> rm /tmp/foo
> touch /tmp/foo.zst
> unzstd /tmp/foo.zst
> stat -f "%N %Fm" /tmp/foo /tmp/foo.zst
/tmp/foo 1694052370.000000000
/tmp/foo.zst 1694052370.336394000

Notice that foo has an older timestamp than foo.zst. It should be equal or greater.

Expected behavior
The output file's timestamps should always be greater than or equal to the input file's.

Desktop (please complete the following information):

  • OS: FreeBSD
  • Version: 14.0-CURRENT
  • Compiler: Clang
  • Flags: The defaults. Probably -O2
  • Other relevant hardware specs: None
  • Build system: FreeBSD's builtin build system
@felixhandte felixhandte self-assigned this Sep 7, 2023
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 12, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 12, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 12, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
felixhandte added a commit to felixhandte/zstd that referenced this issue Mar 13, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
hswong3i pushed a commit to alvistack/facebook-zstd that referenced this issue Mar 27, 2024
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
hswong3i pushed a commit to alvistack/facebook-zstd that referenced this issue Jan 5, 2025
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on facebook#3952.

Further addresses facebook#3748.

[0] https://github.com/freebsd/freebsd-src/blob/937a0055858a098027f464abf0b2b1ec5d36748f/sys/sys/unistd.h#L96

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants