Skip to content
This repository has been archived by the owner on Feb 27, 2024. It is now read-only.

nixpart -p generates invalid syntax for my kickstart file #11

Closed
nh2 opened this issue May 6, 2018 · 6 comments
Closed

nixpart -p generates invalid syntax for my kickstart file #11

nh2 opened this issue May 6, 2018 · 6 comments

Comments

@nh2
Copy link

nh2 commented May 6, 2018

Trying to format a Hetzner server:

root@rescue ~ # cat kickstartfile
clearpart --all --initlabel --drives=sda,sdb,sdc,sdd

part raid.1 --grow --ondisk=sda
part raid.2 --grow --ondisk=sdb

#raid "" --level=1 --device=md0 --label=ssdjournal raid.1 raid.2

part raid.3 --grow --ondisk=sdc
part raid.4 --grow --ondisk=sdd

#raid "" --level=1 --device=md1 --label=root raid.3 raid.4

%post
# --run makes mdadm not promt the user for confirmation
mdadm --create --run --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm --create --run --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1

mke2fs -O journal_dev /dev/md0
mkfs.ext4 -J device=/dev/md0 /dev/md1
echo "/dev/md1   /   ext4   data=journal,errors=remount-ro 0 1" >> /etc/fstab
%end

root@rescue ~ # nixpart -p kickstartfile 
{
  swapDevices = [
  ];
  boot.loader.grub.devices = [
    "/dev/sda"
    "/dev/sdb"
    "/dev/sdc"
    "/dev/sdd"
  ];
  fileSystems = {}{};
}

fileSystems = {}{}; isn't valid syntax, and nixops trips over this output.
@nh2
Copy link
Author

nh2 commented May 6, 2018

Verbose output:

root@rescue ~ # nixpart --verbose -p kickstartfile 
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... modprobe fcoe
modprobe: FATAL: Module fcoe not found.
Return code: 1
Running... udevadm settle --timeout=300
Return code: 0
Running... /nix/store/ys8h0cz0zkypg65xfrvm1n59pkhsa8ac-multipath-tools-0.4.9/sbin/multipath -c /dev/sda
May 06 05:01:27 | DM multipath kernel driver not loaded
Return code: 1
Running... /nix/store/ys8h0cz0zkypg65xfrvm1n59pkhsa8ac-multipath-tools-0.4.9/sbin/multipath -c /dev/sdb
May 06 05:01:27 | DM multipath kernel driver not loaded
Return code: 1
Running... /nix/store/ys8h0cz0zkypg65xfrvm1n59pkhsa8ac-multipath-tools-0.4.9/sbin/multipath -c /dev/sdc
May 06 05:01:27 | DM multipath kernel driver not loaded
Return code: 1
Running... /nix/store/ys8h0cz0zkypg65xfrvm1n59pkhsa8ac-multipath-tools-0.4.9/sbin/multipath -c /dev/sdd
May 06 05:01:28 | DM multipath kernel driver not loaded
Return code: 1
Running... dumpe2fs -h /dev/loop0
dumpe2fs 1.42.12 (29-Aug-2014)
Filesystem volume name:   <none>
Last mounted on:          /
Filesystem UUID:          40c4ea95-0ecc-4c51-9f3e-e49d8f62f160
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              160000
Block count:              643072
Reserved block count:     32153
Free blocks:              141196
Free inodes:              54745
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      93
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8000
Inode blocks per group:   500
Filesystem created:       Wed Apr 24 10:15:52 2013
Last mount time:          Tue Mar 27 14:56:30 2018
Last write time:          Tue Mar 27 14:57:50 2018
Mount count:              62
Maximum mount count:      -1
Last checked:             Mon Mar  6 13:23:27 2017
Check interval:           0 (<none>)
Lifetime writes:          14 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     28
Desired extra isize:      28
Default directory hash:   half_md4
Directory Hash Seed:      052a8a24-205b-46f4-a17c-b410f916ae6d

Return code: 0
Running... resize2fs -P /dev/loop0
resize2fs 1.42.12 (29-Aug-2014)
resize2fs: Device or resource busy while trying to open /dev/loop0
Couldn't find valid filesystem superblock.
Return code: 1
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... dd if=/dev/zero of=/dev/sdd bs=512 seek=2048 count=2048
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.0134026 s, 78.2 MB/s
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... /nix/store/1fmhjqzhzxb2rcj8miprmdjpqz3gsa46-util-linux-2.31-bin/sbin/wipefs --force -a /dev/sdd1
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... dd if=/dev/zero of=/dev/sdc bs=512 seek=2048 count=2048
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.013034 s, 80.4 MB/s
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... /nix/store/1fmhjqzhzxb2rcj8miprmdjpqz3gsa46-util-linux-2.31-bin/sbin/wipefs --force -a /dev/sdc1
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... dd if=/dev/zero of=/dev/sdb bs=512 seek=2048 count=2048
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.0115451 s, 90.8 MB/s
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... /nix/store/1fmhjqzhzxb2rcj8miprmdjpqz3gsa46-util-linux-2.31-bin/sbin/wipefs --force -a /dev/sdb1
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... dd if=/dev/zero of=/dev/sda bs=512 seek=2048 count=2048
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.0114466 s, 91.6 MB/s
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... /nix/store/1fmhjqzhzxb2rcj8miprmdjpqz3gsa46-util-linux-2.31-bin/sbin/wipefs --force -a /dev/sda1
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
Running... udevadm settle --timeout=300
Return code: 0
{
  swapDevices = [
  ];
  boot.loader.grub.devices = [
    "/dev/sda"
    "/dev/sdb"
    "/dev/sdc"
    "/dev/sdd"
  ];
  fileSystems = {}{};
}

@nh2
Copy link
Author

nh2 commented May 6, 2018 •
edited
Loading

I think one bug is in def nixify(), because

>>> nixify({})
'{}{}'

That's probably because of

        if len(data) == 0:
            out += "{}"
        if len(data) == 0 or (len(data) == 1 and
                              not isinstance(data.values()[0], dict)):
            out += "{" + mkattr(data, " ", " ", ind) + "}"

both ifs trigger. Maybe elif was intended @aszlig?

Anoter question is why it's {} at all, looks like storage is {}, maybe it doesn't run my %post commands?

@aszlig
Copy link
Member

aszlig commented May 6, 2018

@nh2: The first if probably wasn't intended, because the second one already produces the same result for empty attrsets.

@nh2
Copy link
Author

nh2 commented May 6, 2018

@aszlig Do you know if nixpart should run %pre or %post?

I'm trying to make this SSD+HDD server use ext4 with external journal (for great speedup), and for that I have to run

mke2fs -O journal_dev /dev/md0
mkfs.ext4 -J device=/dev/md0 /dev/md1

But I'm not sure where to stick it. %pre sounds like the optimal place but it doesn't seem to run.

@aszlig
Copy link
Member

aszlig commented May 15, 2018

%pre and %post are not run at all if I remember correctly.

aszlig added a commit that referenced this issue May 15, 2018
@aszlig
Fix generating invalid Nix attribute sets
69bd518 
If the data given to nixify() is a dictionary and it's empty, the result
is that we get *two* empty Nix attribute sets, like this:

fileSystems = {}{};

So either we'd need to return after appending {} to the output or just
remove the check for whether data is an empty dict.

I choose the latter as it decreases the amount of branches.

Thanks to @nh2 for reporting.

Signed-off-by: aszlig <[email protected]>
Fixes: #11
@nh2
Copy link
Author

nh2 commented Jun 28, 2018 •
edited
Loading

Closing as the fix for the invalid syntax is in 69bd518.

The lack of support for %pre and %post for my usecase can be worked around in nixops via NixOS/nixops#948.

@nh2 nh2 closed this as completed Jun 28, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aszlig @nh2