Media transcoding and modifying, remodel and roll out!

Posted by Andrew Lewis – Software program Engineer, Android Media Options

The creation of user-generated content material is on the rise, and customers are searching for extra methods to personalize and add uniqueness to their creations. These creations are then shared to an unlimited community of gadgets, every with its personal capabilities. The Jetpack Media3 1.0 launch consists of new performance within the Transformer module for changing media information between codecs, or transcoding, and making use of modifying operations. For instance, you’ll be able to trim a clip from an extended piece of media and apply results to the video monitor to share over social media, or transcode media right into a extra environment friendly codec for add to a server.

The general purpose of Transformer is to offer a straightforward to make use of, dependable and performant API for transcoding and modifying media, together with help for customizing performance, following the identical API design rules to ExoPlayer. The library is supported on gadgets operating Android 5.0 Lollipop (API 21) onwards and consists of device-specific optimizations, giving builders a powerful basis to construct on. This submit provides an introduction to the brand new performance and describes a number of the many options we’re planning for upcoming releases!

Getting Began

Most operations with Transformer will observe the identical common sample:

  1. Configure a TransformationRequest with settings like your required output format
  2. Create a Transformer and move it your TransformationRequest
  3. Apply extra results and edits
  4. Connect a listener to react to completion occasions
  5. Begin the transformation

In fact, relying in your desired transformations, you could not want each step. Here is an instance of transcoding an enter video to the H.265/HEVC video format and eradicating the audio monitor.

val transformationRequest = TransformationRequest.Builder().setVideoMimeType(MimeTypes.VIDEO_H265).construct()

val transformer = Transformer.Builder(context)

val inputMediaItem = MediaItem.fromUri("path_to_input_file")
transformer.startTransformation(inputMediaItem, outputPath)

Throughout transformation you may get progress updates with Transformer.getProgress. When the transformation completes the listener is notified in its onTransformationCompleted or onTransformationError callback, and you’ll course of the output media as wanted.

Try our documentation to find out about additional capabilities within the Transformer APIs. You can even discover particulars about utilizing Transformer to precisely convert 10-bit HDR content material to 8-bit SDR within the “Coping with shade washout” blog post to make sure your video’s colours stay as vibrant as doable within the case that your app or the gadget would not help HDR content material.

Edits, results, and extensions

Media3 features a set of core video effects for easy edits, similar to scaling, cropping, and shade filters, which you should use with Transformer. For instance, you’ll be able to create a Presentation impact to scale the enter to 480p decision whereas sustaining the unique side ratio, and apply it with setVideoEffects:


You can even chain a number of results to create extra complicated outcomes. This instance converts the enter video to grayscale and rotates it by 30 levels:


It is also doable to increase Transformer’s performance by implementing customized results that construct on current ones. Right here is an instance of subclassing MatrixTransformation, the place we begin zoomed in by 2 instances, then zoom out step by step because the body presentation time will increase:

val zoomOutEffect = MatrixTransformation { presentationTimeUs ->
val transformationMatrix = Matrix()
val scale = 2 - min(1f, presentationTimeUs / 1_000_000f)
transformationMatrix.postScale( scale, scale)

Here is a display recording that exhibits this impact being utilized within the Transformer demo app:

moving image showing what subclassing matrix transformation looks like in the Transformer demo app

For much more superior use circumstances, you’ll be able to wrap your individual OpenGL code or different processing libraries in a customized GL texture processor and plug these into Transformer as customized results. See the demo app for some examples of customized results. The README additionally has directions for making an attempt a demo of MediaPipe integration with Transformer.

Coming quickly

Transformer is actively below improvement however prepared to make use of, so please give it a attempt to share your suggestions! The Media3 development branch features a sneak peek into a number of new options constructing on the 1.0 launch described right here, together with help for tone-mapping HDR movies to SDR utilizing OpenGL, previewing video results utilizing ExoPlayer.setVideoEffects, and customized audio processing. We’re additionally engaged on help for modifying a number of movies in additional versatile compositions, with export from Transformer and playback by ExoPlayer, making Media3 an end-to-end answer for reworking media.

We hope you may discover Transformer an easy-to-use and highly effective software for implementing incredible media modifying experiences on Android! You may ship us function requests and bug stories within the Media3 GitHub issue tracker, and observe this weblog to get updates on new options. Keep tuned for our upcoming talk “Prime quality Android media experiences” at Google I/O.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button