Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use MonotonicTimelineInterpolator in video transcoder #4257

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

jmartinesp
Copy link
Member

@jmartinesp jmartinesp commented Feb 10, 2025

Content

This should ensure timestamps are monotonically increasing, as the transcoder library needs.

Motivation and context

Screenshots / GIFs

Tests

  • Step 1
  • Step 2
  • Step ...

Tested devices

  • Physical
  • Emulator
  • OS version(s):

Checklist

  • Changes have been tested on an Android device or Android emulator with API 24
  • UI change has been tested on both light and dark themes
  • Accessibility has been taken into account. See https://github.com/element-hq/element-x-android/blob/develop/CONTRIBUTING.md#accessibility
  • Pull request is based on the develop branch
  • Pull request title will be used in the release note, it clearly define what will change for the user
  • Pull request includes screenshots or videos if containing UI changes
  • You've made a self review of your PR

This should ensure timestamps are motononically increasing, as the transcoder library needs.
Copy link
Contributor

github-actions bot commented Feb 10, 2025

📱 Scan the QR code below to install the build (arm64 only) for this PR.
QR code
If you can't scan the QR code you can install the build via this link: https://i.diawi.com/BsVX3e

Copy link

codecov bot commented Feb 10, 2025

Codecov Report

Attention: Patch coverage is 0% with 53 lines in your changes missing coverage. Please review.

Project coverage is 80.03%. Comparing base (de673c9) to head (67839c4).
Report is 8 commits behind head on develop.

Files with missing lines Patch % Lines
...roid/libraries/mediaupload/impl/VideoCompressor.kt 0.00% 46 Missing ⚠️
...aries/mediaupload/impl/AndroidMediaPreProcessor.kt 0.00% 7 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4257      +/-   ##
===========================================
- Coverage    80.07%   80.03%   -0.05%     
===========================================
  Files         2048     2048              
  Lines        54357    54385      +28     
  Branches      6633     6646      +13     
===========================================
  Hits         43525    43525              
- Misses        8555     8583      +28     
  Partials      2277     2277              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

 Allow uploading the original file if it can't be transcoded.
Copy link
Member

@bmarty bmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a first comment, I do not see any MonotonicTimelineInterpolator in the changes. What am I missing?

@jmartinesp
Copy link
Member Author

The title is a bit outdated, this PR was just added so we could test solutions to fix an issue when trying to transcode videos before sending them (the linked one above) in some Android versions (MIUI for Android 12?).

The original error said:

java.lang.IllegalStateException: Timestamps must be monotonically increasing: 0, 0

So I tried to fix that by using an interpolator that ensures the video timestamps always go forward, to see if that could solve the issue (it doesn't). This is either a problem with the codecs in the OS shipped by problematic vendors or an issue with the lib we're using to simplify the transcoding process.

So I guess our options are either allowing videos to be sent as is if transcoding fails (either as a file or as the actual media) or using a different lib or transcoder for this. I'm waiting for a decision from @mxandreas , but the latest commit tries to implement the first solution.

@mxandreas
Copy link
Contributor

videos to be sent as is if transcoding fails (either as a file or as the actual media)

If "as is" means that the video size/quality will not be affected, but we can still remove the sensitive pieces (like geolocation tags), then I think that's fine. However, I would send them as "file" to have some indication that the transcoding was not applied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants