Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gadget: make reinstalls honor existing sizes of partitions #15086

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

alfonsosanchezbeato
Copy link
Member

When performing a re-install, snapd uses "size" from the gadget when re-creating partitions that it just removed. However, in the case of using min-size there is a possibility that the deleted partition size was smaller but still compatible with the installed gadget. Use the size of the deleted partition, as we want to respect the already existing layout - otherewise we might destroy partitions that are not re-created in the re-install and that could appear after the partitions using min-size.

When performing a re-install, snapd uses "size" from the gadget when
re-creating partitions that it just removed. However, in the case of
using min-size there is a possibility that the deleted partition size
was smaller but still compatible with the installed gadget. Use the
size of the deleted partition, as we want to respect the already
existing layout - otherewise we might destroy partitions that are not
re-created in the re-install and that could appear after the
partitions using min-size.
Copy link

codecov bot commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 90.24390% with 4 lines in your changes missing coverage. Please review.

Project coverage is 77.85%. Comparing base (a272aac) to head (3df0169).
Report is 7 commits behind head on master.

Files with missing lines Patch % Lines
gadget/install/partition.go 92.10% 3 Missing ⚠️
gadget/install/install.go 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #15086      +/-   ##
==========================================
- Coverage   78.07%   77.85%   -0.22%     
==========================================
  Files        1182     1156      -26     
  Lines      157743   156658    -1085     
==========================================
- Hits       123154   121967    -1187     
- Misses      26943    27110     +167     
+ Partials     7646     7581      -65     
Flag Coverage Δ
unittests 77.85% <90.24%> (-0.22%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@pedronis pedronis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we do this only if we notice that the new partition would overlap with a non-deleted partition? or are there scenarios where we don't know/that is not enough? or is too annoying?

@alfonsosanchezbeato
Copy link
Member Author

can't we do this only if we notice that the new partition would overlap with a non-deleted partition? or are there scenarios where we don't know/that is not enough? or is too annoying?

There is an unrelated problem if we do not respect the layout, which is that the partition probing does not seem to be working correctly and the newly created partitions are not detected and the installation stops. I have not digged yet into that, mainly because I think that the correct behavior is to respect the layout, on one side because of the possibility of overlaps, but we could also have the problem of trying to create partitions bigger than the available space. If we want to do something else we need additional checks and in general all the approach seems risky to me. But we can think about this for the future.

@alfonsosanchezbeato alfonsosanchezbeato added the Run nested The PR also runs tests inluded in nested suite label Feb 14, 2025
@pedronis
Copy link
Collaborator

There is an unrelated problem if we do not respect the layout, which is that the partition probing does not seem to be working correctly and the newly created partitions are not detected and the installation stops. I have not digged yet into that, mainly because I think that the correct behavior is to respect the layout, on one side because of the possibility of overlaps, but we could also have the problem of trying to create partitions bigger than the available space. If we want to do something else we need additional checks and in general all the approach seems risky to me. But we can think about this for the future.

it would be good to understand the unrelated problem even if we keep the conservative strategy fix for now.
My input is that we should explain in the code why we are doing this/the strategy and some TODO about what we could try to do instead if somebody asks for a different behavior later

Copy link

github-actions bot commented Feb 14, 2025

Fri Feb 14 21:02:10 UTC 2025
The following results are from: https://github.com/canonical/snapd/actions/runs/13335424065

Failures:

Preparing:

  • google-nested:ubuntu-20.04-64:tests/nested/manual/remodel-offline:local_snaps
  • google-nested:ubuntu-24.04-64:tests/nested/manual/hybrid-remodel
  • google-nested:ubuntu-24.04-64:tests/nested/manual/core20-boot-config-update:gadgetextra

Executing:

  • google-nested:ubuntu-22.04-64:tests/nested/core/core20-fault-inject-on-refresh:kernel_reboot_refresh_gadget_assets
  • google-nested:ubuntu-22.04-64:tests/nested/manual/remodel-min-size
  • google-nested:ubuntu-22.04-64:tests/nested/manual/recovery-system-reboot:factory_reset
  • google-nested:ubuntu-22.04-64:tests/nested/manual/muinstaller-real:partial
  • google-nested:ubuntu-24.04-64:tests/nested/manual/muinstaller-real:partial
  • google-nested:ubuntu-24.04-64:tests/nested/manual/muinstaller-real:encrypted
  • google-nested:ubuntu-24.04-64:tests/nested/manual/core20-early-config
  • google-nested:ubuntu-24.04-64:tests/nested/manual/muinstaller-real:seeded
  • google-core:ubuntu-core-18-64:tests/core/kernel-base-gadget-single-reboot
  • google-core:ubuntu-core-18-64:tests/core/kernel-base-gadget-pair-single-reboot-failover:kernel_base
  • google:ubuntu-20.04-64:tests/main/lxd:snapd_cgroup_both
  • google:ubuntu-20.04-64:tests/main/lxd:snapd_cgroup_just_inside
  • google:ubuntu-24.04-64:tests/main/interfaces-audio-playback-record
  • google:ubuntu-24.04-64:tests/main/progress

Restoring:

  • google-nested:ubuntu-22.04-64:tests/nested/manual/muinstaller-real:seeded
  • google-nested:ubuntu-22.04-64:tests/nested/manual/
  • google-nested:ubuntu-22.04-64
  • google-core:ubuntu-core-18-64:tests/core/kernel-base-gadget-single-reboot
  • google-core:ubuntu-core-18-64:tests/core/
  • google-core:ubuntu-core-18-64

@alfonsosanchezbeato
Copy link
Member Author

There is an unrelated problem if we do not respect the layout, which is that the partition probing does not seem to be working correctly and the newly created partitions are not detected and the installation stops. I have not digged yet into that, mainly because I think that the correct behavior is to respect the layout, on one side because of the possibility of overlaps, but we could also have the problem of trying to create partitions bigger than the available space. If we want to do something else we need additional checks and in general all the approach seems risky to me. But we can think about this for the future.

it would be good to understand the unrelated problem even if we keep the conservative strategy fix for now. My input is that we should explain in the code why we are doing this/the strategy and some TODO about what we could try to do instead if somebody asks for a different behavior later

I've added comments about this now. I think that an additional consideration is that if there is an installer (say, muinstaller for UC) the installer can decide on any size in a [min-size, size] range, and if we force using size we could be overriding an installer decision.

Copy link
Collaborator

@pedronis pedronis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, small confusion

@@ -70,7 +70,7 @@ type CreateOptions struct {
// OnDiskStructure, as it is meant to be used externally (i.e. by
// muinstaller).
func CreateMissingPartitions(dv *gadget.OnDiskVolume, gv *gadget.Volume, opts *CreateOptions) ([]*gadget.OnDiskAndGadgetStructurePair, error) {
dgpairs, err := createMissingPartitions(dv, gv, opts)
dgpairs, err := createMissingPartitions(dv, gv, opts, map[int]StructOffsetSize{})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we don't write to it this can be nil afaict

if opts == nil {
opts = &CreateOptions{}
}
if deletedOffsetSize == nil {
deletedOffsetSize = map[int]StructOffsetSize{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Comment on lines 320 to 336
deletedIndexes := make(map[int]bool, 3)
deletedOffsetSize := make(map[int]StructOffsetSize, 3)
for i, s := range dl.Structure {
if wasCreatedDuringInstall(gv, s) {
yamlIdx := indexIfCreatedDuringInstall(gv, s)
if yamlIdx >= 0 {
logger.Noticef("partition %s was created during previous install", s.Node)
sfdiskIndexes = append(sfdiskIndexes, strconv.Itoa(i+1))
deletedOffsetSize[yamlIdx] = StructOffsetSize{
StartOffset: s.StartOffset,
Size: s.Size,
}
deletedIndexes[i] = true
}
}
if len(sfdiskIndexes) == 0 {
return nil
return deletedOffsetSize, nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire setup is very confusing. We are using 3 different indices, and there are absoultely no description as to why each one is used.

Would it make more sense for YamlIndex to just be a part of the OnDiskStructure (i.e dl.Structure members in this case) - so yamlIndex wouldn't have to be returned from indexIfCreatedDuringInstall, and we could avoid that change?

Also from a readability perspective this change to removeCreatedPartitions that it returns a seemingly unrelated map of StructOffsetSize, without any documentation as to why this is returned seems like a menace.

To me, it seems the information about which partitions that should be removed, should be encapsulated in it's own function, return one/list of structures with all information removeCreatedPartitions and whatever deletedOffsetSize has to perform the delete and then keep that function simple.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Run nested The PR also runs tests inluded in nested suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants