Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any way to deploy from desktop to phone using deploy-rs or nixus? #94

Open
bbigras opened this issue Jan 9, 2021 · 22 comments
Open

Any way to deploy from desktop to phone using deploy-rs or nixus? #94

bbigras opened this issue Jan 9, 2021 · 22 comments

Comments

@bbigras
Copy link
Contributor

bbigras commented Jan 9, 2021

https://github.com/serokell/deploy-rs
https://github.com/Infinisil/nixus

Those tools allow us to build multiples machines at once and push deploy them.

If I can build aarch64, can I just build something, nix-copy it to the phone and run an activate command on the phone?

@Gerschtli
Copy link
Collaborator

That should be possible I think. Have a look at nix-on-droid/nix-on-droid.sh on how we build and activate :)

@t184256
Copy link
Collaborator

t184256 commented Jan 10, 2021

You'll need an ssh server on the phone if you're keen on initiating that from a desktop, I guess, otherwise it's just https://github.com/t184256/nix-on-droid/wiki/Remote-building.

But once you have it, "build something, nix-copy it to the phone and run an activate command on the phone" should just work, yeah.

@bbigras
Copy link
Contributor Author

bbigras commented Jan 11, 2021

Thanks. I think I'm making some progress.

Any way to force system to aarch64-linux when running on x86_x64?

right now I'm trying to do this:

        profiles.system.path = deploy-rs.lib.aarch64-linux.activate.custom
          (
            (import (nix-on-droid + "/modules") {
              pkgs = nixpkgs.legacyPackages."aarch64-linux";
              config = nix_on_droid_config;
              home-manager-src = home-manager;
            }).activationPackage
          ) "./activate";

I think maybe I have to specify pkgs = nixpkgs.legacyPackages."aarch64-linux" in a couple of places.

current status:

❯ deploy ".#pixel2" -- --impure
🚀 ℹ️ [deploy] [INFO] Running checks for flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
warning: unknown flake output 'deploy'
🚀 ℹ️ [deploy] [INFO] Evaluating flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹ️ [deploy] [INFO] The following profiles are going to be deployed:
[pixel2.system]
user = "nix-on-droid"
ssh_user = "nix-on-droid"
path = "/nix/store/87hfvlqrb2760ns1jqz6x7b0zdfrn5cm-activatable-nix-on-droid-generation"
hostname = "pixel2"
ssh_opts = []

🚀 ℹ️ [deploy] [INFO] Building profile `system` for node `pixel2`
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹ️ [deploy] [INFO] Activating profile `system` for node `pixel2`
⭐ ℹ️ [activate] [INFO] Activating profile
/nix/var/nix/profiles/per-user/nix-on-droid/system/deploy-rs-activate: line 2:  8338 Illegal instruction     ./activate
⭐ ❌ [activate] [ERROR] The activation script resulted in a bad exit code: Some(132)
🚀 ❌ [deploy] [ERROR] Failed to deploy profile: Activating over SSH resulted in a bad exit code: Some(1)

In result, it seems my nix-on-droid-path has x86_64 stuff.

activate-rs is aarch64

EDIT: ok nix-on-droid-path is now aarch64

@bbigras
Copy link
Contributor Author

bbigras commented Jan 11, 2021

EDIT: I'm calling ./activate without the full path.

Current status:

❯ deploy ".#pixel2" -- --impure
🚀 ℹ️ [deploy] [INFO] Running checks for flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
warning: unknown flake output 'deploy'
🚀 ℹ️ [deploy] [INFO] Evaluating flake in .
warning: Git tree '/home/bbigras/nix-config' is dirty
🚀 ℹ️ [deploy] [INFO] The following profiles are going to be deployed:
[pixel2.system]
user = "nix-on-droid"
ssh_user = "nix-on-droid"
path = "/nix/store/zfyia9axhilazg3l1qw8vhi2n2raqj2d-activatable-nix-on-droid-generation"
hostname = "pixel2"
ssh_opts = []

🚀 ℹ️ [deploy] [INFO] Building profile `system` for node `pixel2`
warning: Git tree '/home/bbigras/nix-config' is dirty
copying path '/nix/store/2jjf28m0qvn9cfqrii9gzav5msv0nf0r-bash-interactive-4.4-p23-man' from 'https://cache.nixos.org'...
copying path '/nix/store/q3acm62dxjdw7rfgnz4zyiz2j619ngr3-glibc-locales-2.32-10' from 'https://cache.nixos.org'...
copying path '/nix/store/qq58n27vwpbhqbzzrzwq0gvl64nl7v74-less-563' from 'https://cache.nixos.org'...
copying path '/nix/store/4klvh9qmav5v841rlrnq9hvhz1dvw7ss-vim-8.2.1522' from 'https://cache.nixos.org'...
🚀 ℹ️ [deploy] [INFO] Activating profile `system` for node `pixel2`
⭐ ℹ️ [activate] [INFO] Activating profile
Activating linkBinSh
Activating linkUsrBinEnv
Activating installLogin
Activating installLoginInner
Activating installPackages
replacing old 'nix-on-droid-path'
installing 'nix-on-droid-path'
building '/nix/store/a4axx3d179cm9l21qhnz3fj6ms9ifqvx-user-environment.drv'...
created 528 symlinks in user environment
Activating installProotStatic
Activating setUpEtc
Removing obsolete symlink '/etc/profiles/per-user/nix-on-droid'...
Activating linkProfile
error: --set requires exactly one derivation
⭐ ❌ [activate] [ERROR] The activation script resulted in a bad exit code: Some(1)
🚀 ❌ [deploy] [ERROR] Failed to deploy profile: Activating over SSH resulted in a bad exit code: Some(1)

@bbigras
Copy link
Contributor Author

bbigras commented Jan 11, 2021

It works if I run the full path in my phone!

Screenshot_20210111-154506.png

Edit: well the "no change" might not be normal. Maybe I ran the wrong one.

Openssh doesn't seem to accept my ssh key after the switch for some reason.

@bbigras
Copy link
Contributor Author

bbigras commented Jan 11, 2021

Any ideas why the nix-on-droid user would break?

EDIT: oh for some reason the nix-on-droid's uid in /etc/passwd is 1000 when I deploy with deploy-rs and it's 10294 when I use nix-on-droid switch.

EDIT 2: I was able to work around it by hardcoding the "right uid/gid. You can see my fork for my changes.

-bash-4.4$ whoami
-whoami: cannot find name for user ID 10294

@Gerschtli
Copy link
Collaborator

The reason for the wrong uid and gid is, that these values have to be generated on the destination machine. There is a derivation built in modules/user.nix, which has to be built locally.

@Gerschtli
Copy link
Collaborator

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

@bbigras
Copy link
Contributor Author

bbigras commented Jan 12, 2021

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

Would that help with the uid thing, or it's related to the other problems I had?

The reason for the wrong uid and gid is, that these values have to be generated on the destination machine. There is a derivation built in modules/user.nix, which has to be built locally.

Could it be a good idea to have a way to optionally set those ids using the config file?

@Gerschtli
Copy link
Collaborator

Furthermore I think you need the activate script in a profile directory, because we use nix-env --set to switch between generations. Have a look at /nix/var/nix/profiles tree in our default setup.

Would that help with the uid thing, or it's related to the other problems I had?

I was refering to the comment you made "Edit: well the "no change" might not be normal. Maybe I ran the wrong one.".

Could it be a good idea to have a way to optionally set those ids using the config file?

I mean you could make these configurable, but the UX would be strange to first setup nix-on-droid the default way, gathering uid and gid, and then hardcode these values. Maybe I am just overthinking :D

Would it be possible to build everything remotely except this derivation? IIRC there aren't big dependants of this derivation, so it should be alright to build the remaining things locally on the destination phone.

@bbigras
Copy link
Contributor Author

bbigras commented Jan 12, 2021

I was refering to the comment you made "Edit: well the "no change" might not be normal. Maybe I ran the wrong one.".

Ok I think this issue is working now. Thanks. I'll take a look again if I see the message again.

I mean you could make these configurable, but the UX would be strange to first setup nix-on-droid the default way, gathering uid and gid, and then hardcode these values. Maybe I am just overthinking :D

I was thinking only people wanting to deploy from a desktop like me would have to handle this.

Would it be possible to build everything remotely except this derivation? IIRC there aren't big dependants of this derivation, so it should be alright to build the remaining things locally on the destination phone.

I have no idea how I could build it on my phone while deploying from my desktop (or from github actions).

Do you think it could be done at the activation step (on the phone)?

@Gerschtli
Copy link
Collaborator

The activation step is too late, the uid and gid need to be available to build the nix-on-droid generation. Maybe we could refactor everything, but I don't it will work.

Sounds like an exciting idea to deploy nix-on-droid config via github actions. If the uid/gid part is the only thing stopping you, you could add these two values as option if you add a note in their description, when to set these manually and when you should rather leave them emtpy and using the ids.nix derivation. I would be fine with this change.

@Gerschtli
Copy link
Collaborator

And if it works for you, it would be awesome if you could and a wiki page explaining your setup :)

@bbigras
Copy link
Contributor Author

bbigras commented Jan 12, 2021

Sounds like an exciting idea to deploy nix-on-droid config via github actions.

Yeah. It would work for people having access to an aarch64 build box though (which I don't). I should check if github actions has aarch64 nodes.

Right now my nix-config builds multiple machines (desktop and laptop) with github actions and pushes it to cachix. Then on my computer I deploy it. It just takes the time required to download and deploy.

If the uid/gid part is the only thing stopping you, you could add these two values as option if you add a note in their description, when to set these manually and when you should rather leave them emtpy and using the ids.nix derivation. I would be fine with this change.

Thanks I'll take a look.

And if it works for you, it would be awesome if you could and a wiki page explaining your setup :)

Will do! 😄

@Gerschtli
Copy link
Collaborator

Unfortunately github actions are not available for aarch64-linux AFAIK..

@bbigras
Copy link
Contributor Author

bbigras commented May 29, 2021

I'm using something like the following to build and deploy nix-on-droid with deploy-rs, but I just realized that I'm not using the nix-on-droid overlays.

Any ideas how to set the overlays?

let
pixel2 = (inputs.nix-on-droid.lib.aarch64-linux.nix-on-droid { config = ../hosts/pixel2; }).activationPackage;
in
{
{
  deploy = {
    autoRollback = true;
    magicRollback = true;
    user = "root";

    nodes = [
    {
      pixel2 = {
        hostname = "pixel2";

        # to prevent using sudo
        sshUser = "nix-on-droid";
        user = "nix-on-droid";

        profiles.nix-on-droid.path = deploy-rs.lib.aarch64-linux.activate.custom
          pixel2
          (pixel2 + "/activate");
      };
    };
];
  };
}

https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/nix/deploy.nix
https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/flake.nix
https://github.com/bbigras/nix-config/blob/7d7eb023bee13c9bfcd7f87444d2b15c7884cab8/hosts/pixel2/default.nix

@Gerschtli
Copy link
Collaborator

Unfortunately I do not know deploy-rs, but you can use the overlays directory and source all overlays via overlays/default.nix as a list.

@pinage404
Copy link
Contributor

Hello,

I did a deploy script before finding this issue

The script rely on SSH

@geoffreygarrett
Copy link

geoffreygarrett commented Sep 28, 2024

Managed deploying with deploy-rs, though I'm pretty sure the rollback isn't working exactly as intended. Feel free to add this to the wiki.


Solution for Deploying Using deploy-rs with nix-on-droid

Steps:

  1. SSH Setup

    Setup SSH on nix-on-droid if not done already. Add your desktop's SSH public key to the nix-on-droid user's ~/.ssh/authorized_keys file.

  2. UID/GID Handling

    The primary issue is ensuring the correct uid and gid for the nix-on-droid user on your phone. When building on your desktop, these might not match, leading to permission issues.

    • Find the UID and GID on your Android device:

      id nix-on-droid

      This command returns the uid and gid of the nix-on-droid user on your device.

    • Set the UID and GID explicitly in your nix-on-droid configuration:

      { 
        user.uid = <uid>;
        user.gid = <gid>;
      } # Replace <uid> and <gid> with the values from your device
  3. Set Up the deploy-rs Configuration

    The activation comes from above in this thread itself. Here's a helper to make it a little less repetitive.

    let
      activateNixOnDroid =
        configuration:
        inputs.deploy-rs.lib.aarch64-linux.activate.custom
          configuration.activationPackage
          "${configuration.activationPackage}/activate";
    in

    Here's how to configure your deploy-rs for nix-on-droid:

    deploy.nodes = {
      "pioneer" = {
        hostname = "pioneer.nixus.net"; # Replace with your device's hostname or IP (I use `dnsmaq` for local DNS)
        profiles.system = {
          sshUser = "nix-on-droid";
          user = "nix-on-droid";
          magicRollback = true;
          sshOpts = [ "-p" "8022" ]; # Adjust port if necessary (Step 1 dependent)
          path = activateNixOnDroid self.nixOnDroidConfigurations.pioneer;
        };
      };
    };

    Note: The hostname pioneer.nixus.net is specific to my setup using dnsmasq for local DNS resolution. Adjust it according to your environment.

  4. Handle Multiple Devices

    If deploying to multiple devices, define specific configurations for each, especially if they have different uid and gid values.

    nixOnDroidConfigurations = {
      "pioneer" = nix-on-droid.lib.nixOnDroidConfiguration {
        pkgs = pkgsFor "aarch64-linux";
        modules = [
          ./nix/hosts/pioneer.nix
          # Include other modules like Home Manager if needed
          { 
            user.uid = 10701;
            user.gid = 10701;
          } # Replace with the UID and GID from your device
        ];
      };
    
      "voyager" = nix-on-droid.lib.nixOnDroidConfiguration {
        pkgs = pkgsFor "aarch64-linux";
        modules = [
          ./nix/hosts/voyager.nix
          { 
            user.uid = 10403;
            user.gid = 10403;
          } # Replace with the UID and GID from your other device
        ];
      };
    };
  5. Set Up Cachix for Substitutes

    To satisfy dependencies like static-proot, ensure your Nix settings include the necessary substituters and trusted public keys. This is crucial because nix-on-droid relies on prebuilt cross-compiled binaries for proot-static, which are specified by hard-coded Nix store paths in the configuration.

    Nix Settings:

    {
      nix.settings = {
        substituters = [
          # "https://cache.nixos.org/"
          # "https://nix-community.cachix.org"
          "https://nix-on-droid.cachix.org" <----
        ];
    
        trusted-public-keys = [
          # "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
          # "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
          "nix-on-droid.cachix.org-1:56snoMJTXmE7wm+67YySRoTY64Zkivk9RT4QaKYgpkE=" # <---
        ];
      };
    } # Pretty sure you only need what's pointed out, but I kept my full Cachix config in case I'm missing something else.

    Reason for the Cache:

    The nix-on-droid configuration specifies the prootStatic binary using hardcoded Nix store paths that point to prebuilt cross-compiled binaries:

    environment.files = {
      prootStatic =
        let
          crossCompiledPaths = {
            aarch64-linux = "/nix/store/7qd99m1w65x2vgqg453nd70y60sm3kay-proot-termux-static-aarch64-unknown-linux-android-unstable-2024-05-04";
            x86_64-linux = "/nix/store/pakj3svvw84rhkzdc6211yhc2cgvc21f-proot-termux-static-x86_64-unknown-linux-android-unstable-2024-05-04";
          };
        in
        "${crossCompiledPaths.${targetSystem}}";
    };

    (Reference: nix-on-droid/modules/environment/login/default.nix#L90)

    These binaries are not built locally during deployment (I have no idea how it would be setup), so having the cache set up ensures they can be fetched from the nix-on-droid Cachix cache.

  6. Configure Overlays for nix-on-droid

    Ensure your overlays are properly set up to include nix-on-droid:

    pkgsFor = system: import nixpkgs {
      inherit system;
      config = {
        allowUnfree = true;
        overlays = [
          (import ./nix/overlays)              # Your custom overlays
          # (final: prev: { nix-on-droid = nix-on-droid.packages.${system}; }) # not necessary, I use it in one of my flake apps
        ] ++ lib.optional (custom.isAndroid system) nix-on-droid.overlays.default; 
      };
    }; # isAndroid is just checking against a list of [ "aarch64-linux" ], as I believe only that is really supported here
  7. Finalize and Deploy

    With all configurations in place, deploy using:

    deploy --targets ".#pioneer" -- --impure

    Replace pioneer with the appropriate node name from your deploy.nodes configuration. (let me know if this can be done without --impure)

    Deployment Log
    ℹ️  [deploy] [INFO] Evaluating flake in .
    trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix
    trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix
    🚀 ℹ️  [deploy] [INFO] The following profiles are going to be deployed:
    [pioneer.system]
    user = "nix-on-droid"
    ssh_user = "nix-on-droid"
    path = "/nix/store/p2jbjbr1lksk2aa26cfhxlzs0rxnbfgm-activatable-nix-on-droid-generation"
    hostname = "pioneer.nixus.net"
    ssh_opts = ["-p", "8022"]
    
    🚀 ℹ️  [deploy] [INFO] Building profile `system` for node `pioneer`
    🚀 ℹ️  [deploy] [INFO] Copying profile `system` to node `pioneer`
    🚀 ℹ️  [deploy] [INFO] Activating profile `system` for node `pioneer`
    🚀 ℹ️  [deploy] [INFO] Creating activation waiter
    ⭐ ℹ️  [activate] [INFO] Activating profile
    👀 ℹ️  [wait] [INFO] Waiting for confirmation event...
    Activating linkBinSh
    Activating linkUsrBinEnv
    Activating setPriorityHomeManagerPath
    Activating installLogin
    Activating installLoginInner
    Activating installPackages
    replacing old 'nix-on-droid-path'
    installing 'nix-on-droid-path'
    Activating installProotStatic
    Activating linkColors
    Activating linkFont
    Activating setUpEtc
    Activating sops-nix
    sops-install-secrets: Imported /data/data/com.termux.nix/files/home/.ssh/id_ed25519 as age key with fingerprint `***KEY_CENSORED***`
    Activating sshd
    Setting up OpenSSH...
    Authorized keys file already exists. Skipping...
    Setting correct permissions...
    OpenSSH setup complete.
    Activating homeManager
    Starting Home Manager activation
    Activating checkFilesChanged
    Activating checkLinkTargets
    Activating writeBoundary
    Activating installPackages
    Activating migrateGhAccounts
    Activating linkGeneration
    Cleaning up orphan links from /data/data/com.termux.nix/files/home
    No change so reusing latest profile generation 48
    Creating home file links in /data/data/com.termux.nix/files/home
    Activating onFilesChange
    Activating reloadSystemd
    User systemd daemon not running. Skipping reload.
    Activating sops-nix
    User systemd daemon not running. Probably executed on boot where no manual start/reload is needed.
    Activating linkProfile
    ⭐ ℹ️  [activate] [INFO] Activation succeeded!
    ⭐ ℹ️  [activate] [INFO] Magic rollback is enabled, setting up confirmation hook...
    👀 ℹ️  [wait] [INFO] Found canary file, done waiting!
    ⭐ ℹ️  [activate] [INFO] Waiting for confirmation event...
    🚀 ℹ️  [deploy] [INFO] Success activating, attempting to confirm activation
    🚀 ℹ️  [deploy] [INFO] Deployment confirmed.
    
    .dotfiles on  develop [$?] took 49s

    Hope this helps someone. Final note, if you're cross compiling from x86_64-linux on your nixos desktop, add this to your desktops config and switch:

    boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
  • EDIT : deploy cli use was wrong + added the deployment log
  • EDIT: Added the requirement to set both UID and GID in the configuration. While deployment was successful with just the UID, setting both UID and GID is recommended. For example, I could deploy via ssh but I couldn't log in via ssh without the correct GID. In all cases I've seen, the UID and GID for the nix-on-droid user are identical, but it's best to check and set both explicitly.

@carnotweat
Copy link

nix-on-droid.pkgs vs isAndroid

related issue & concerns

Managed deploying with deploy-rs, though I'm pretty sure the rollback isn't working exactly as intended. Feel free to add this to the wiki.

Solution for Deploying Using deploy-rs with nix-on-droid

Steps:

1. **SSH Setup**
   [Setup SSH on nix-on-droid](https://github.com/nix-community/nix-on-droid/wiki/SSH-access) if not done already. Add your desktop's SSH public key to the `nix-on-droid` user's `~/.ssh/authorized_keys` file.

2. **UID/GID Handling**
   The primary issue is ensuring the correct `uid` and `gid` for the `nix-on-droid` user on your phone. When building on your desktop, these might not match, leading to permission issues.
   
   * **Find the UID and GID on your Android device:**
     ```shell
     id nix-on-droid
     ```
     
     
         
           
         
     
           
         
     
         
       
     This command returns the `uid` and `gid` of the `nix-on-droid` user on your device.
   * **Set the UID and GID explicitly in your `nix-on-droid` configuration:**
     ```nix
     { 
       user.uid = <uid>;
       user.gid = <gid>;
     } # Replace <uid> and <gid> with the values from your device
     ```

3. **Set Up the `deploy-rs` Configuration**
   The activation comes from above in this thread itself. Here's a helper to make it a little less repetitive.
   ```nix
   let
     activateNixOnDroid =
       configuration:
       inputs.deploy-rs.lib.aarch64-linux.activate.custom
         configuration.activationPackage
         "${configuration.activationPackage}/activate";
   in
   ```
   
   
       
         
       
   
         
       
   
       
     
   Here's how to configure your `deploy-rs` for `nix-on-droid`:
   ```nix
   deploy.nodes = {
     "pioneer" = {
       hostname = "pioneer.nixus.net"; # Replace with your device's hostname or IP (I use `dnsmaq` for local DNS)
       profiles.system = {
         sshUser = "nix-on-droid";
         user = "nix-on-droid";
         magicRollback = true;
         sshOpts = [ "-p" "8022" ]; # Adjust port if necessary (Step 1 dependent)
         path = activateNixOnDroid self.nixOnDroidConfigurations.pioneer;
       };
     };
   };
   ```
   
   
       
         
       
   
         
       
   
       
     
   **Note:** The hostname `pioneer.nixus.net` is specific to my setup using `dnsmasq` for local DNS resolution. Adjust it according to your environment.

4. **Handle Multiple Devices**
   If deploying to multiple devices, define specific configurations for each, especially if they have different `uid` and `gid` values.
   ```nix
   nixOnDroidConfigurations = {
     "pioneer" = nix-on-droid.lib.nixOnDroidConfiguration {
       pkgs = pkgsFor "aarch64-linux";
       modules = [
         ./nix/hosts/pioneer.nix
         # Include other modules like Home Manager if needed
         { 
           user.uid = 10701;
           user.gid = 10701;
         } # Replace with the UID and GID from your device
       ];
     };
   
     "voyager" = nix-on-droid.lib.nixOnDroidConfiguration {
       pkgs = pkgsFor "aarch64-linux";
       modules = [
         ./nix/hosts/voyager.nix
         { 
           user.uid = 10403;
           user.gid = 10403;
         } # Replace with the UID and GID from your other device
       ];
     };
   };
   ```

5. **Set Up Cachix for Substitutes**
   To satisfy dependencies like `static-proot`, ensure your Nix settings include the necessary substituters and trusted public keys. This is crucial because `nix-on-droid` relies on prebuilt cross-compiled binaries for `proot-static`, which are specified by hard-coded Nix store paths in the configuration.
   **Nix Settings:**
   ```nix
   {
     nix.settings = {
       substituters = [
         # "https://cache.nixos.org/"
         # "https://nix-community.cachix.org"
         "https://nix-on-droid.cachix.org" <----
       ];
   
       trusted-public-keys = [
         # "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
         # "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
         "nix-on-droid.cachix.org-1:56snoMJTXmE7wm+67YySRoTY64Zkivk9RT4QaKYgpkE=" # <---
       ];
     };
   } # Pretty sure you only need what's pointed out, but I kept my full Cachix config in case I'm missing something else.
   ```
   
   
       
         
       
   
         
       
   
       
     
   **Reason for the Cache:**
   The `nix-on-droid` configuration specifies the `prootStatic` binary using hardcoded Nix store paths that point to prebuilt cross-compiled binaries:
   ```nix
   environment.files = {
     prootStatic =
       let
         crossCompiledPaths = {
           aarch64-linux = "/nix/store/7qd99m1w65x2vgqg453nd70y60sm3kay-proot-termux-static-aarch64-unknown-linux-android-unstable-2024-05-04";
           x86_64-linux = "/nix/store/pakj3svvw84rhkzdc6211yhc2cgvc21f-proot-termux-static-x86_64-unknown-linux-android-unstable-2024-05-04";
         };
       in
       "${crossCompiledPaths.${targetSystem}}";
   };
   ```
   
   
       
         
       
   
         
       
   
       
     
   _(Reference: [nix-on-droid/modules/environment/login/default.nix#L90](https://github.com/nix-community/nix-on-droid/blob/5d88ff2519e4952f8d22472b52c531bb5f1635fc/modules/environment/login/default.nix#L90))_
   These binaries are not built locally during deployment (I have no idea how it would be setup), so having the cache set up ensures they can be fetched from the `nix-on-droid` Cachix cache.

6. **Configure Overlays for `nix-on-droid`**
   Ensure your overlays are properly set up to include `nix-on-droid`:
   ```nix
   pkgsFor = system: import nixpkgs {
     inherit system;
     config = {
       allowUnfree = true;
       overlays = [
         (import ./nix/overlays)              # Your custom overlays
         # (final: prev: { nix-on-droid = nix-on-droid.packages.${system}; }) # not necessary, I use it in one of my flake apps
       ] ++ lib.optional (custom.isAndroid system) nix-on-droid.overlays.default; 
     };
   }; # isAndroid is just checking against a list of [ "aarch64-linux" ], as I believe only that is really supported here
   ```

7. **Finalize and Deploy**
   With all configurations in place, deploy using:
   ```shell
   deploy --targets ".#pioneer" -- --impure
   ```
   
   
       
         
       
   
         
       
   
       
     
   Replace `pioneer` with the appropriate node name from your `deploy.nodes` configuration. (let me know if this can be done without `--impure`)
   
      Deployment Log
   ```shell
   ℹ️  [deploy] [INFO] Evaluating flake in .
   trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix
   trace: Loading overlays: better-icons.nix shared-cargo-gptcommit.nix sops-injection-key-fetcher.nix sops-ssh-wrapper.nix
   🚀 ℹ️  [deploy] [INFO] The following profiles are going to be deployed:
   [pioneer.system]
   user = "nix-on-droid"
   ssh_user = "nix-on-droid"
   path = "/nix/store/p2jbjbr1lksk2aa26cfhxlzs0rxnbfgm-activatable-nix-on-droid-generation"
   hostname = "pioneer.nixus.net"
   ssh_opts = ["-p", "8022"]
   
   🚀 ℹ️  [deploy] [INFO] Building profile `system` for node `pioneer`
   🚀 ℹ️  [deploy] [INFO] Copying profile `system` to node `pioneer`
   🚀 ℹ️  [deploy] [INFO] Activating profile `system` for node `pioneer`
   🚀 ℹ️  [deploy] [INFO] Creating activation waiter
   ⭐ ℹ️  [activate] [INFO] Activating profile
   👀 ℹ️  [wait] [INFO] Waiting for confirmation event...
   Activating linkBinSh
   Activating linkUsrBinEnv
   Activating setPriorityHomeManagerPath
   Activating installLogin
   Activating installLoginInner
   Activating installPackages
   replacing old 'nix-on-droid-path'
   installing 'nix-on-droid-path'
   Activating installProotStatic
   Activating linkColors
   Activating linkFont
   Activating setUpEtc
   Activating sops-nix
   sops-install-secrets: Imported /data/data/com.termux.nix/files/home/.ssh/id_ed25519 as age key with fingerprint `***KEY_CENSORED***`
   Activating sshd
   Setting up OpenSSH...
   Authorized keys file already exists. Skipping...
   Setting correct permissions...
   OpenSSH setup complete.
   Activating homeManager
   Starting Home Manager activation
   Activating checkFilesChanged
   Activating checkLinkTargets
   Activating writeBoundary
   Activating installPackages
   Activating migrateGhAccounts
   Activating linkGeneration
   Cleaning up orphan links from /data/data/com.termux.nix/files/home
   No change so reusing latest profile generation 48
   Creating home file links in /data/data/com.termux.nix/files/home
   Activating onFilesChange
   Activating reloadSystemd
   User systemd daemon not running. Skipping reload.
   Activating sops-nix
   User systemd daemon not running. Probably executed on boot where no manual start/reload is needed.
   Activating linkProfile
   ⭐ ℹ️  [activate] [INFO] Activation succeeded!
   ⭐ ℹ️  [activate] [INFO] Magic rollback is enabled, setting up confirmation hook...
   👀 ℹ️  [wait] [INFO] Found canary file, done waiting!
   ⭐ ℹ️  [activate] [INFO] Waiting for confirmation event...
   🚀 ℹ️  [deploy] [INFO] Success activating, attempting to confirm activation
   🚀 ℹ️  [deploy] [INFO] Deployment confirmed.
   
   .dotfiles on  develop [$?] took 49s
   ```
   
   
       
         
       
   
         
       
   
       
     
      
   
   Hope this helps someone. Final note, if you're cross compiling from `x86_64-linux` on your nixos desktop, add this to your desktops config and switch:
   ```nix
   boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
   ```


* _EDIT_ : `deploy` cli use was wrong + added the deployment log

* _EDIT: Added the requirement to set both UID and GID in the configuration. While deployment was successful with just the UID, setting both UID and GID is recommended. For example, I could deploy via `ssh` but I couldn't log in via `ssh` without the correct `GID`. In all cases I've seen, the UID and GID for the nix-on-droid user are identical, but it's best to check and set both explicitly._

@Marlinc
Copy link

Marlinc commented Dec 29, 2024

This is working great with the instructions from @geoffreygarrett!

🚀 ℹ️ [deploy] [INFO] Running checks for flake in .
evaluation warning: The option `services.xserver.desktopManager.plasma6.enable' defined in `/nix/store/hk70f92sdwdh9xxkmw0ixm6nj0q79693-source/hosts/nixos-wsl/configuration.nix' has been renamed to `services.desktopManager.plasma6.enable'.
evaluation warning: The option `services.xserver.displayManager.sddm.enable' defined in `/nix/store/hk70f92sdwdh9xxkmw0ixm6nj0q79693-source/hosts/nixos-wsl/configuration.nix' has been renamed to `services.displayManager.sddm.enable'.
evaluation warning: The option `services.xserver.displayManager.defaultSession' defined in `/nix/store/hk70f92sdwdh9xxkmw0ixm6nj0q79693-source/hosts/nixos-wsl/configuration.nix' has been renamed to `services.displayManager.defaultSession'.
warning: unknown flake output 'nixOnDroidConfigurations'
warning: unknown flake output 'deploy'
warning: The check omitted these incompatible systems: aarch-linux, aarch64-darwin, aarch64-linux, aarch64-netbsd, armv5tel-linux, armv6l-linux, armv6l-netbsd, armv7a-darwin, armv7a-linux, armv7a-netbsd, armv7l-linux, armv7l-netbsd, i686-cygwin, i686-darwin, i686-freebsd, i686-linux, i686-netbsd, i686-openbsd, loongarch64-linux, m68k-linux, m68k-netbsd, microblaze-linux, microblazeel-linux, mips-linux, mips64-linux, mips64el-linux, mipsel-linux, mipsel-netbsd, powerpc-netbsd, powerpc64-linux, powerpc64le-linux, riscv32-linux, riscv32-netbsd, riscv64-linux, riscv64-netbsd, s390-linux, s390x-linux, x86_64-cygwin, x86_64-darwin, x86_64-freebsd, x86_64-netbsd, x86_64-openbsd, x86_64-redox, x86_64-solaris
Use '--all-systems' to check all.
🚀 ℹ️ [deploy] [INFO] Evaluating flake in .
🚀 ℹ️ [deploy] [INFO] The following profiles are going to be deployed:
[s24u.system]
user = "nix-on-droid"
ssh_user = "nix-on-droid"
path = "/nix/store/a13rc59y20vw5k95fnc2vs4xl3aj0baj-activatable-nix-on-droid-generation"
hostname = "192.168.1.56"
ssh_opts = ["-p", "8022"]

🚀 ℹ️ [deploy] [INFO] Building profile `system` for node `s24u`
🚀 ℹ️ [deploy] [INFO] Copying profile `system` to node `s24u`
🚀 ℹ️ [deploy] [INFO] Activating profile `system` for node `s24u`
🚀 ℹ️ [deploy] [INFO] Creating activation waiter
⭐ ℹ️ [activate] [INFO] Activating profile
👀 ℹ️ [wait] [INFO] Waiting for confirmation event...
Activating linkBinSh
Activating linkUsrBinEnv
Activating installLogin
Activating installLoginInner
Activating installPackages
replacing old 'nix-on-droid-path'
installing 'nix-on-droid-path'
Activating installProotStatic
Activating setUpEtc
Activating unlinkColors
Activating unlinkFont
Activating linkProfile
Activating reloadSupervisord
Reloading supervisord...
⭐ ℹ️ [activate] [INFO] Activation succeeded!
⭐ ℹ️ [activate] [INFO] Magic rollback is enabled, setting up confirmation hook...
👀 ℹ️ [wait] [INFO] Found canary file, done waiting!
⭐ ℹ️ [activate] [INFO] Waiting for confirmation event...
🚀 ℹ️ [deploy] [INFO] Success activating, attempting to confirm activation
🚀 ℹ️ [deploy] [INFO] Deployment confirmed.

@Marlinc
Copy link

Marlinc commented Dec 29, 2024

Added the instructions to the wiki and also included the configuration needed to build aarch64 from other architectures: https://github.com/nix-community/nix-on-droid/wiki/Remote-deploy-with-deploy%E2%80%90rs

I think the issue can be closed with this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants