Bootstrap PP from a SnapshotRepository snapshot #2288
Conversation
pcholakov
changed the title
| task_center() | ||
| .spawn_blocking_fn_unmanaged("unpack-snapshot", Some(partition_id), move || { | ||
| let mut tarball = tar::Archive::new(snapshot_reader); | ||
| for file in tarball.entries()? { | ||
| let mut file = file?; | ||
| trace!("Unpacking snapshot file: {:?}", file.header().path()?); | ||
| file.unpack_in(&snapshot_dir)?; | ||
| } | ||
| Ok::<(), anyhow::Error>(()) | ||
| }) | ||
| .await??; |
There was a problem hiding this comment.
I initially tried to use spawn_blocking_unmanaged here but got the following error:
"Cannot start a runtime from within a runtime. This happens because a function (like
block_on) attempted to block the current thread while the thread is being used to drive asynchronous tasks."
This is not a problem in the earlier use in this file - on line 136, we spawn a future which also uses the tar crate to bundle up a local directory on the upload path, and that works just fine. The only difference here appears to be the use of SyncIoBridge to make the AsyncRead byte stream of the download into a Read for tar's Archive to consume.
I was also able to get this to work using plain vanilla tokio::task::spawn_blocking. It seems like the TaskCenter implementation using block_on is part of this but I'm not entirely sure what's going on.
restate/crates/core/src/task_center.rs
Lines 754 to 756 in c825714
No description provided.