Wrapper machinery should produce a tiny compiled binary

[copumpkin]

On macOS, shebangs can't point at interpreted (shebang'd) scripts, so anything we use the existing wrapper machinery on will fail if used in a shebang line on macOS. The solution would be to have the wrapper builder produce a compiled binary rather than a script.

[bjornfor]

Or can we come up with a piece of code that injects the (language specific) wrapper code directly into the program itself? Then we get rid of the argv0 thing and save one fork/exec.

[vcunat]

I wouldn't take that approach; it doesn't seem worth the risks and work included. AFAIK (in C) you can set argv0 to whatever you want.

[vcunat]

There have been several occasions where we discussed this, though I can't find them quickly. What I remember:

• sometimes the shebang is too long core packages: long interpreter lines with long nix store path prefix #10854
• many people like the wrappers to be human-readable (in some way)

IIRC there was a suggestion of compiling a tiny C file that would simply find the "wrapper script" according to some fixed relative-path scheme and execute the interpreter (bash) with the script, perhaps via parsing the shebang line. Obviously, that would would add another exec rather than save it. (Wrappers don't fork IIRC and I don't see a use for doing so.)

[BlessJah]

It seems that Apple fixed that around MacOS 10.12. Could someone test on 10.11:

cat > definitely_a_bash.sh <<EOF
#!/usr/bin/env bash
echo "Called with \$*"
bash "\${@}"
EOF

cat > script.sh <<EOF
#!./definitely_a_bash.sh
echo 'Hello world!'
EOF

chmod +x script.sh definitely_a_bash.sh
./script.sh

[Profpatsch]

I’d strongly suggest against binary wrappers, from experience I have needed to debug them in many instances, which would have been a lot harder if they were non-readable.

I do like the idea of an adapter binary for older MacOS though.

[anderslundstedt]

It seems that Apple fixed that around MacOS 10.12. Could someone test on 10.11:

cat > definitely_a_bash.sh <<EOF
#!/usr/bin/env bash
echo "Called with \$*"
bash "\${@}"
EOF

cat > script.sh <<EOF
#!./definitely_a_bash.sh
echo 'Hello world!'
EOF

chmod +x script.sh definitely_a_bash.sh
./script.sh

I am on MacOS 10.12 and this does indeed work. But not all wrappers seem to work, see for example #65351

[cstrahan]

I’d strongly suggest against binary wrappers, from experience I have needed to debug them in many instances, which would have been a lot harder if they were non-readable.

We could just include a string constant that documents what the binary is doing, and then you could open the wrapper in a text editor to check (or run it through strings).

[lavoiesl]

@anderslundstedt, that only works for me if the shell is zsh:

$ bash -c ./script.sh
Hello world!

$ zsh -c ./script.sh
Called with ./script.sh
Hello world!

[anderslundstedt]

You are correct. For my private python scripts I have solved this problem by using makeWrapper to create wrappers.

[purcell]

I hit what I think is this issue in #86881 on Catalina, but I also note that @Profpatsch landed an execline executable in #71357 which looks like it may have been related to this discussion. Any insights?

[Profpatsch]

You should be able to adapt the C wrapper for execline to arcanist. A general solution as discussed in this issue would be considerably more work I’m afraid.

…

On Tue, May 5, 2020 at 7:08 AM Steve Purcell ***@***.***> wrote: I hit what I think is this issue in #86881 <#86881> on Catalina, but I also note that @Profpatsch <https://github.com/Profpatsch> landed an execline executable in #71357 <#71357> which looks like it may have been related to this discussion. Any insights? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23018 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYB5ZQFUYNRBURDJL5LW6TRP6NLRANCNFSM4DAZC2SA> .

[FRidh]

WIP PR adding support for binary wrappers #95569.

[stale]

I marked this as stale due to inactivity. → More info

[roberth]

This came up on stackoverflow again, from a Big Sur user. https://stackoverflow.com/questions/67100831/macos-shebang-with-absolute-path-not-working

From what I understand, it should suffice to create a C wrapper that behaves like

exec "$@"

Darwin does support multiple arguments in the shebang, so we can just prefix it when necessary.

let
  shebang = if stdenv.isDarwin then "#!${exec-helper}/bin/exec-helper " else "#!";

Fixing a script for darwin will be as simple as starting with

''
  ${shebang}${pythonWithPackages .....}
''

[bergkvist]

Example with define flags and documentation that is useful for debugging.

#include <string.h>
#include <stdlib.h>

const char* DOCS = "Behaves like: \nexec \"" EXECUTABLE "\" \"$@\"\n"
"Why does this exist? On MacOS, shebangs can't point at scripts. "
"We can put this binary in the shebang instead.";

int main(int argc, char** argv) {
    char cmd[65536];
    strcpy(cmd, EXECUTABLE);
    for (int i = 1; i < argc; ++i) {
        strcat(cmd, " ");
        strcat(cmd, argv[i]);
    }
    return system(cmd);
}

$ gcc wrapper.c -o wrapper -D'EXECUTABLE="/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python"'

$ strings wrapper
Behaves like: 
exec "/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python" "$@"
Why does this exist? On MacOS, shebangs can't point at scripts. We can put this binary in the shebang instead.
/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python

[bergkvist]

Looks like we can use execv instead (which might also be slightly more efficient than system, since it replaces the process instead of creating a subprocess). Also, less code = less potential bugs.

#include <unistd.h>

const char* DOCS = "Behaves like: \nexec \"" EXECUTABLE "\" \"$@\"\n"
"Why does this exist? On MacOS, shebangs can't point at scripts. "
"We can put this binary in the shebang instead.";

int main(int argc, char** argv) {
    argv[0] = EXECUTABLE;
    return execv(EXECUTABLE, argv);
}

[bergkvist]

Consider the following shell script (/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python).

#! /nix/store/ra8yvijdfjcs5f66b99gdjn86gparrbz-bash-4.4-p23/bin/bash -e
export NIX_PYTHONPREFIX='/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env'
export NIX_PYTHONEXECUTABLE='/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python3.9'
export NIX_PYTHONPATH='/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/lib/python3.9/site-packages'
export PYTHONNOUSERSITE='true'
exec "/nix/store/7pjbbmnrch7frgyp7gz19ay0z1173c7y-python3-3.9.2/bin/python"  "$@"

If we want something that replaces the shell script, instead of pointing at it, we could do that like this in C:

#include <unistd.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    putenv("NIX_PYTHONPREFIX=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env");
    putenv("NIX_PYTHONEXECUTABLE=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python3.9");
    putenv("NIX_PYTHONPATH=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/lib/python3.9/site-packages");
    putenv("PYTHONNOUSERSITE=true");
    argv[0] = "/nix/store/7pjbbmnrch7frgyp7gz19ay0z1173c7y-python3-3.9.2/bin/python";
    return execv(argv[0], argv);
}

And it is also relatively easy to debug using the strings command:

$ strings wrapper
NIX_PYTHONPREFIX=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env
NIX_PYTHONEXECUTABLE=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/bin/python3.9
NIX_PYTHONPATH=/nix/store/i46k148mi830riq4wxh49ki8qmq0731k-python3-3.9.2-env/lib/python3.9/site-packages
PYTHONNOUSERSITE=true
/nix/store/7pjbbmnrch7frgyp7gz19ay0z1173c7y-python3-3.9.2/bin/python

[bergkvist]

Shell script for generating C-wrapper-code:

# make-c-wrapper.sh EXECUTABLE ARGS
# ARGS:
# --argv0       NAME    : set name of executed process to NAME
#                         (defaults to EXECUTABLE)
# --set         VAR VAL : add VAR with value VAL to the executable’s
#                         environment
# --set-default VAR VAL : like --set, but only adds VAR if not already set in
#                         the environment
# --unset       VAR     : remove VAR from the environment
echo "#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv) {"

# We escape C-strings by replacing \ by \\ and " by \":
sanitize_c_string() { sed 's/\\/\\\\/g' | sed 's/"/\\"/g'; };
executable=$(echo $1 | sanitize_c_string)
params=("$@")

for ((n = 1; n < ${#params[*]}; n += 1)); do
    p="${params[$n]}"
    if [[ "$p" == "--set" ]]; then
        key=$(echo "${params[$((n + 1))]}" | sanitize_c_string)
        value=$(echo "${params[$((n + 2))]}" | sanitize_c_string)
        n=$((n + 2))
        echo "    putenv(\"$key=$value\");"
    elif [[ "$p" == "--set-default" ]]; then
        key=$(echo "${params[$((n + 1))]}" | sanitize_c_string)
        value=$(echo "${params[$((n + 2))]}" | sanitize_c_string)
        n=$((n + 2))
        echo "    setenv(\"$key\", \"$value\", 0);"
    elif [[ "$p" == "--unset" ]]; then
        key=$(echo "${params[$((n + 1))]}" | sanitize_c_string)
        echo "    unsetenv(\"$key\");"
        n=$((n + 1))
    elif [[ "$p" == "--argv0" ]]; then
        argv0=$(echo "${params[$((n + 1))]}" | sanitize_c_string)
        n=$((n + 1))
    else
        # Using an error macro, we will make sure the compiler gives an understandable error message
        echo "    #error make-c-wrapper.sh did not understand argument $p"
    fi
done

echo "    argv[0] = \"${argv0:-$executable}\";"
echo "    return execv(\"$executable\", argv);\n}"

Example usage:

$ ./make-c-wrapper.sh /usr/bin/python3 --set HELLO '$WORLD"' --set-default X 1 --unset '_Y'

Output:

#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    putenv("HELLO=$WORLD\"");
    setenv("X", "1", 0);
    unsetenv("_Y");
    argv[0] = "/usr/bin/python3";
    return execv("/usr/bin/python3", argv);
}

And this could be piped into a compiler like gcc:

$ ./make-c-wrapper.sh /usr/bin/python3 --set HELLO '$WORLD"' --set-default X 1 --unset '_Y' | gcc -x c -o wrapper -
$ ./wrapper -c "import os; print('HELLO:', os.getenv('HELLO'))"
HELLO: $WORLD"

We can also get useful error messages when something goes wrong

$ ./make-c-wrapper.sh /usr/bin/python3 --set HELLO WORLD --unknown-argument | gcc -x c -o wrapper - 
<stdin>:6:6: error: make-c-wrapper.sh did not understand argument --unknown-argument
    #error make-c-wrapper.sh did not understand argument --unknown-argument
     ^
1 error generated.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixosstag.fcio.net/t/wrappers-and-hooks-do-not-invoke-wrapprogram-directly/3551/1

[stale]

I marked this as stale due to inactivity. → More info

[FRidh]

Implemented in #124556 and is being used in Python's buildEnv. In time it can become the default Nixpkgs-wide.