Help poor macOS users who can't build many closures on their host #560
Comments
|
@puffnfresh is working on something that could help a lot with this |
|
@puffnfresh Do tell! |
|
He packaged HyperKit (Docker's fork of xhyve) and is making a wrapper for it to act as a background remote builder for Nix, sort of the way Docker for Mac works, except with an even more minimal OS in it that only builds Nix derivations. So basically a similar idea to the VBox build worker but fully managed by Nix, free, and with no kernel components to install (or even privileged processes). It's a work in progress but what I've seen so far has been very promising! |
|
Is it still desirable to build on the virtual machine you're deploying to? |
|
@grahamc: In NixOS/nix#1061 (comment) you noted that you were able to get distributed builds working on a multi-user Nix installation on OS X. Is that documented anywhere? |
|
I'm not sure I was able to get it working? Though, I'd definitely ping @LnL7 on that. I suspect nix-darwin helps out a lot here. |
|
@grahamc: The reason I asked is that the way we got distributed builds working on multi-user Nix on OS X required running Nix commands as the |
|
Should have mentioned another workaround: https://github.com/LnL7/nix-docker/blob/master/start-docker-nix-build-slave |
|
I am able to I haven’t had a chance to try this yet with NixOps, but @angerman’s post suggests that works as well. |
|
@Gabriel439 I wrote some basic instructions a couple days ago on how to configure the daemon without nix-darwin. The pitfall most people run into is known_hosts. https://gist.github.com/LnL7/ba2eac19e77cd6b4bb02c8de03bf5f4e |
|
I can build with nix-build on my This works: $ nix-build -E 'with import { system = "x86_64-linux"; }; hello.overrideAttrs (drv: { REBUILD = builtins.currentTime; })'
This does not work: $ nixops deploy --build-only building all machine configurations... error: assertion failed at /nix/store/7i7x9g2f6rl50cxn2xpi0yy14l5x49z7-nixpkgs-18.03pre128804.127cea92389/nixpkgs/pkgs/os-specific/linux/kernel/generic.nix:48:1 (use ‘--show-trace’ to show detailed location information) error: unable to build all machine configurations UPDATE: adding "--option system x86_64-linux" to my nixops deploy command makes the remote building work. |
|
@ledettwy Thanks, that option fixes evaluation. Everyone, this is the assertion that fails: Should this |
|
@ledettwy: Another way you can fix this problem without having to add https://nixos.org/nixos/options.html#nixpkgs.system @wmertens: Yes, that assertion is correct and should not be changed. The root cause is that NixOps is trying to build a NixOS system whose |
|
@Gabriel439 I've read that, but that doesn't work for me. I have |
|
I can confirm that setting |
|
I think this might help https://github.com/puffnfresh/linuxkit-builder |
|
At our company we use the following instructions for our macOS users with multi-user installations. They are an amalgam of what works best with remote builds today on top of nix-docker: Step 1: Install nix-docker $ git clone http://github.com/lnl7/nix-docker
$ docker run --restart always --name nix-docker -d -p 3022:22 lnl7/nix:ssh
$ chmod 600 nix-docker/ssh/insecure_rsa
$ cp nix-docker/ssh/insecure_rsa /etc/nix/docker_rsaStep 2: Add the following config inside your Step 3: Add the Step 4: Ensure that the machines file is used by Nix, by adding/ensuring the following in Step 5: Restart your nix daemon: $ sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist
$ sudo launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plistStep 6: Check you local remote builder for $ nix-shell -p wget --builders ssh://nix-docker --option system x86_64-linux
$ nix build '(with import <nixpkgs> {}; runCommand "unameNixDocker" {} "uname -a > $out")' --builders ssh://nix-docker --option system x86_64-linux
$ cat ./result
Linux 1c33a493a33e 4.9.87-linuxkit-aufs #1 SMP Wed Mar 14 15:12:16 UTC 2018 x86_64 GNU/Linux |
|
Hi, I've been trying to get this working for a few days now, but I'm having some issues that I haven't been able to get passed. Here's what I've done so far
Get the insecure private key for nix-docker Add the following to ~/.ssh/config and /var/root/.ssh/config Make sure known_hosts is updated for root
Add the following to /etc/nix/machines Add the following to /etc/nix/nix.conf
Using the trivial example from https://nixos.org/nixops/manual/ trivial.nix trivial-vbox.nix
I've tried to figure this out for a few days now without success. I also can't get 'Step 6' from Periklis's comment to work I'd appreceiate any thoughts, or if anyone can point me in the right direction. |
|
I'm having the same experience as @werbitt, and it's super frustrating. I will have to downgrade again to Nix 1.11 so I can continue with my work. |
|
@werbitt Ok, I figured it out. The option is no longer You should also not use |
|
Hmm, can anyone see where I'm going wrong with this? 🤔 This is for remote builds on a physical aarch64-linux system. Any help is greatly appreciated 🙏 Info dump
macOS host builder config services.nix-daemon.enable = true;
nix.distributedBuilds = true;
nix.buildMachines = [
{
hostName = "net1";
sshUser = "root";
sshKey = "${homeDir}/.ssh/id_rsa";
systems = [ "aarch64-linux" ];
maxJobs = 2;
}
];Target host snippet { config, pkgs, lib, domain, ... }:
let
localAddr = "10.0.0.2";
gatewayAddr = "10.0.0.1";
in {
deployment.hasFastConnection = true;
deployment.targetHost = "net1.${domain}";
nixpkgs.localSystem.system = "aarch64-linux";
... |
Poor macOS users often run into issues when trying to deploy with
nixopsbecause many packages must be built on Linux. Perhaps we can offer them some help in the documentation and potentially point them to "workarounds" like this one: https://github.com/3noch/nix-vbox-build-slaveThe text was updated successfully, but these errors were encountered: