Default shouldUseMultipart setting for AWS S3 plugin doesn't always work correctly for really large files

[denisahearn]

Initial checklist

• I understand this is a bug report and questions should be posted in the Community Forum
• I searched issues and couldn’t find anything (or linked relevant results below)

Link to runnable example

No response

Steps to reproduce

@uppy/aws-s3: 4.1.0
@uppy/core: 4.2.0
@uppy/dashboard: 4.1.0

Attached below is a .zip file containing a small sample web app that can be used to recreate the issue. There is a README.md in the .zip file that explains how to build and run the app. Please pay attention to /src/uppy.js, which is the JavaScript code that instantiates Uppy and configures the AwsS3 plugin.

uppy-multi-part-upload-failure.zip

If I configure the plugin as shown below, using the default shouldUseMultipart, there are times that Uppy will incorrectly determine it should use a single-part upload for files that exceed 5 GB in size. Unfortunately this behavior is not consistent. I've see incorrect behavior for a 12 GB and a 27 GB file, but I've seen the correct behavior for an 18 GB file. I put a 6GB file at https://s3.us-west-2.amazonaws.com/data.sentera.com/Uppy/test-file-6gb.txt that you can download and use with the sample app to consistently demonstrate the issue.

uppy.use(AwsS3, {
  id: `uppy-aws-s3-${id}`,
  endpoint: '/'
})

When I override the shouldUseMultipart setting with my own function that does exactly what is documented here, then Uppy correctly determines when to use a multi-part upload.

uppy.use(AwsS3, {
  id: `uppy-aws-s3-${id}`,
  endpoint: '/',
  shouldUseMultipart(file) {
    // Use multipart only for files larger than 100MiB.
    return file.size > 100 * 2 ** 20
  }
})

Expected behavior

When selecting a file larger than 100 MB, I expect the default behavior of the shouldUseMultipart setting of the AWS S3 plugin to determine that files less than or equal to 100 MB in size should be uploaded using a single chunk, and files greater than 100 MB in size will be uploaded using multiple chunks.

Actual behavior

For some really large files (larger than 5 GB), the default shouldUseMultipart setting of the AWS S3 plugin sometimes incorrectly determines that the file can be uploaded in a single chunk, which fails because AWS S3 doesn't permit the chunk size to exceed 5 GB in size.