Tutorial - Using VapourSynth vs AVISynth+ for QTGMC: Round One

PLEASE NOTE: This tutorial is now significantly out of date. I've provided it only for historical reference, and am currently working on a followup. The instructions below WILL NOT WORK with current versions of VapourSynth.

Since I've had a few commenters on my videos ask about VapourSynth, I figured it was time to give it a look. For those who don't know, VapourSynth is a python-based video processing scripting system similar to AVISynth, and can actually use AVISynth plugins. My main interest in VS is that it has a native QTGMC port, and before AVISynth+ 64-bit got stable, VS has been a preferred method among some users for faster/more stable conversions.

It took me about a morning to get everything I needed and set up a sample script for QTGMC conversion. I decided to try to keep the setup as bloat-free as possible by using "portable" versions of the apps involved.

Here was my ultimate workflow. I won't try to explain everything since I'm not totally fluent in Python, but adapting the following settings should allow you to get it work:


BIG DISCLAIMER: This process may not work, may crash, or do other things to your system. Virus scanning all files is strongly encouraged, but not a 100% guarantee of safety.

You have been warned. 

If you're on a deadline (and using Premiere Pro, After Effects, or Final Cut Pro) probably your best best is to use a paid plugin like FieldsKit. And no, they aren't paying me to say that.

Also, this tutorial is for Windows 10. Most of the steps work for other OSes, but I won't cover the differences here.

Here's a video version of the tutorial:

First, grabbed the embeddable version of Python 3.7.X:
https://www.python.org/downloads/
(Click on the name of the latest version of Python 3.7, then scroll down to find the embeddable version.)

Then, downloaded the portable VapourSynth:
https://github.com/vapoursynth/vapoursynth/releases
(I'm using the 64-bit portable version.)

Then, grabbed VapourSynth Editor (VSEdit):
https://bitbucket.org/mystery_keeper/vapoursynth-editor/downloads/

Extracted the Python archive to a directory. Extract both VapourSynth and VSEdit to the same directory, in that order.

Now, for the needed plugins and VapourSynth Python modules:

FFmpegSource:
https://github.com/FFMS/ffms2/releases

havsfunc
https://github.com/HomeOfVapourSynthEvolution/havsfunc/releases
(The source code is what you want here.)

mvsfunc
https://github.com/HomeOfVapourSynthEvolution/mvsfunc/releases

adjust
https://github.com/dubhater/vapoursynth-adjust/releases

nnedi3_resample
https://github.com/mawen1250/VapourSynth-script
(Click on the "Clone or download button" and select "Download ZIP")

I also grabbed fmtconv for colorspace conversion:
https://github.com/EleonoreMizo/fmtconv/releases

Then, the VapourSynth versions of the needed QTGMC prerequisites:
https://github.com/dubhater/vapoursynth-mvtools/releases

https://github.com/dubhater/vapoursynth-nnedi3/releases
(You'll also need nnedi3_weights.bin from here. Left-click on the link, don't right-click/save)

Scanned all the above files for viruses.

Extracted the .py files in the main directory. Extracted the (64-bit) .dll files to the vapoursynth64/plugins directory

Opened VSEdit. Made the following initial script:

import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='F:\directory\\input movie.mov')
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540, matrix_in_s='709')
clip.set_output()

Note that the final backslash before the input file name needs to be "escaped" with another backslash.

With the above settings, the source colorspace will be preserved, but the color matrix will be shifted to REC.709 on resize. To change both the output colorspace and color matrix, use:

clip = core.resize.Spline36(clip, 720, 540, format=vs.YUV422P10, matrix_in_s='709')

If you're coming from a source file with a non-recognized colorspace, you can use:

clip = core.fmtc.resample (clip=clip, css="420")
clip = core.fmtc.bitdepth (clip=clip, bits=8)

right after the ffms2 command.

Update: As UniveralAl1 mentions in a comment on the tutorial video, it may be possible to skip this by just converting once to YUV422p10 before QTGMC. The resulting script might look something like this:

import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='E:\Archive\\input video.avi')
clip = vs.core.resize.Point(clip, format = vs.YUV422P10)
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540)
clip.set_output()

Please try this first rather than the final script below.

All of the above ended up being necessary for the video file I selected for testing due to it using 4:1:1 chroma subsampling, so here is my final script:

import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='E:\Archive\\input video.avi')
clip = core.fmtc.resample (clip=clip, css="420")
clip = core.fmtc.bitdepth (clip=clip, bits=8)
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540, format=vs.YUV422P10, matrix_in_s='709')
clip.set_output()

Saving this script gives you a .vpy file.

To render it out, I used a combination of vspipe and FFMPEG as per the documentation. However, VapourSynth does not handle audio and video at the same time, so I had to use the mapping command in FFMPEG to copy over the audio separately:

vspipe --y4m Upscale.vpy - | ffmpeg -i pipe: -i "C:\pathto\inputmovie.mov" -c:v prores -profile:v 3 -c:a copy -map 0:0 -map 1:1 "F:\Temp\output file.mov" 

Preliminary testing gives me 67 fps average for VapourSynth and 77fps average for AVS+ using the following script:

SetFilterMTMode("QTGMC", 2)
FFmpegSource2("ITVS Trailer.avi", atrack=1)
ConvertToYV12()
AssumeBFF()
QTGMC(Preset="Slower", EdiThreads=3)
Spline36Resize(720, 540)
Prefetch(10)

The only difference I noticed in the resulting files was that bright red colors bled upwards in the VS encode, and downwards in the AVS+ encode. If I figure out more, will report back.

If you have any input or suggestions, feel free to leave a comment below.

Announcement: Adobe has brought (officially licensed) ProRes encoding to Windows versions of their video apps. This is not a drill.

Adobe just announced that officially licensed ProRes encoding is now available for Premiere Pro, After Effects, etc. on Windows. I just downloaded the update.

It works.

Finally.

Remember, if you can't afford the Creative Cloud All Apps subscription, you can still export (not officially licensed) ProRes files using FFMPEG or one of the many programs that uses FFMPEG for export. Just realize that some TV channels will not accept non-official ProRes files, so don't use FFMPEG as your final export if you're submitting to a major TV channel.

Anyways, my next tutorial was going to be about DNxHD/DNxHR encoding in FFMPEG for this very reason. I will now be looking for another topic.

Tutorial - FFMPEG issues - Resizing color shift, Adobe weirdness, etc.

FFMPEG is an amazing program. I frequently use it to convert obscure video formats to something usable, export from AVISynth scripts, and swap audio tracks in and out of videos. It does have some quirks, however, and if you aren't on the lookout for them, they can change your video in ways you might not like. Here's a few of them:

BIG DISCLAIMER: This process may not work, may crash, or do other things to your system. Virus scanning all files is strongly encouraged, but not a 100% guarantee of safety.

You have been warned. 

Also, this tutorial is for Windows 10. Most of the steps work for other OSes, but I won't cover the differences here.

Here's a video version of the tutorial:

The SD to HD resizing bug

Resizing in FFMPEG is done via filters, which are specified by the -vf switch. -vf has so many filters it would be impossible to describe here, but the structure for how each is called is basically:

-vf "filter1=optiona:optionb:optionc, filter2=optiona:optionb:optionc"

Of course, you won't necessarily use that many (or even any) options depending on the filters you choose.

A simple resize and convert to ProRes 422HQ would be:

ffmpeg -i "inputvideo.mp4" -vf  "scale=1280:720:flags=spline" -c:v prores  -profile:v 3 -pix_fmt yuv422p10le -c:a pcm_s16le "outputvideo.mov"

Note how the resize comes before the codec setting. Order of commands is very important in FFMPEG, and putting things in the wrong order can result in errors or quality issues.

The above command would resize the source video to 1280x720 using the Spline resize method, similar to what I've used in AVISynth tutorials in the past. For more info on FFMPEG resizing, check out the wiki entry here:

https://trac.ffmpeg.org/wiki/Scaling

So what's the issue? Well, if you're resizing from a standard definition video (SD) to a high definition (HD) framesize, you might get a color shift... depending on the codec you use. In my testing, DNxHD/HR and H.264 have this issue, while ProRes does not. Check out the following frame grabs:

This is the original image from a DVD version of a documentary I worked on:

Yes, it's supposed to look like that. The DVD is encoded in anamorphic widescreen, when means it takes a 16x9 image and squeezes it to fit inside a 4x3 frame. Your DVD player can then un-squeeze the image back to 16x9 for widescreen TVs.

This is the video upscaled to 720p using FFMPEG:

Notice the color shift? It's subtle. It's most prominent on the lower-third graphic and the warden's face.

So, why is this happening? Basically, SD video conforms to a broadcast standard known as bt.601. HD video conforms to a very slightly different standard known as bt.709 (otherwise known as Rec.709). FFMPEG does not automatically correct for the differences between the two standards, and this results in a color/brightness shift.

Luckily, you can force the conversion by adding colormatrix=bt601:bt709 to the filters entry, like so:

ffmpeg -i "inputvideo.mp4" -vf  "scale=1280:720:flags=spline, colormatrix=bt601:bt709" -c:v dnxhd -profile:v dnxhr_hqx -pix_fmt yuv422p10le -c:a pcm_s16le "outputvideo.mxf"

and the result is this:

Viola! Image fixed.

However, don't use this filter unless you see a color shift. Otherwise, you might end up creating a color shift where one didn't exist before. Oh, and you can also convert the other direction as well if you're downscaling to SD from an HD video; just switch the options around:

colormatrix=bt709:bt601

Bottom line? Always check your exports before sending them out or dropping them in your edit timeline. You'll save yourself a lot of work later trying to manually correct them.

 

Color shifts in Adobe products

Update: the ProRes color shift I had up here before is gone in current nightly builds of FFMPEG. If you're still stuck on an older build that creates a color shift when the exported file is viewed in Premiere Pro or After Effects, use the prores_ks encoder rather than prores, and it'll correct the issue.

This glitch is a bit of a puzzler, frankly. Basically, FFMPEG's DNxHD/HR exports have a color shift, but only in After Effects and Premiere Pro.

This is really subtle, so I'm going to zoom in.

Here's the original:

And here's the DNxHD/HR encode viewed in After Effects:

If you still can't see it, use your left and right arrow keys (or tap on the pictures if you're on a mobile device) to move back and forth between the images. Basically there's a slight green/yellow shift.

Now, here's the kicker. If I import these files into DaVinci Resolve, they look identical.

So, what the heck? I'm not entirely sure. If I had to guess, I'd say that FFMPEG is generating metadata that Adobe programs don't like, but it could be something else.

Fortunately, after testing and Googling, I've figured out how to fix the color shifts.

All you have to do is change the container format to .mxf instead of .mov:

ffmpeg -i "inputvideo.mov" -c:v dnxhd -profile:v dnxhr_hqx -c:a pcm_s16le "outputfile.mxf"

And this is the result:

The only issue with the DNxHD/HR solution is that oddly enough, FFMPEG-generated .mxf files don't work in Resolve (and Assimilate Scratch Play). Resolve will refuse to load them, and Scratch Play will load but will only play the audio from the file. Adobe programs will handle them just fine. However, Resolve and Scratch Play will recognize and play back the .mov files with no color shift issues. 

So, yeah. Weird.

Tutorial - Using FFMPEG for DNxHD/DNxHR encoding, resizing, and batch encoding

Before I begin, I'd like to acknowledge Jon Hall, whose recently deceased web page taught me how to set up FFMPEG in my system path. Also, a shoutout to /u/kichigai on the Reddit /r/editors subreddit for his excellent FFMPEG for editors FAQ:

https://www.reddit.com/r/editors/wiki/ffmpeg

If you want a comprehensive look at FFMPEG for a bunch of editing tasks, the above link is the way to go. If you want a video tutorial for a comprehensive FFMPEG setup, check out my tutorial for setting up FFMPEG 32-bit (along with an alternate install method for AVISynth+).

Or, you could try setting up both the 32-bit and 64-bit versions of FFMPEG (along with AVISynth+) at the same time.


BIG DISCLAIMER: This process may not work, may crash, or do other things to your system. Virus scanning all files is strongly encouraged, but not a 100% guarantee of safety.

You have been warned. 

Also, this tutorial is for Windows 10. Most of the steps work for other OSes, but I won't cover the differences here.



Regardless, I'm going to assume for the rest of this post that you already have FFMPEG installed.

First, let's start with a quick review of an FFMPEG command. Note that the line below doesn't actually run as written; this is just to show you the basic structure of the command:

ffmpeg -i "inputvideo.mp4" -c:v videocodec -c:a audiocodec "outputvideofile.mp4"

Simple enough, right? -i for input, -c:v for video codec, -c:a for audio codec, and then the name of the output file. There's a bunch of other options you can put in amongst this, but basically, this is the way the command is structured.


DNxHD, DNxHR, ProRes and Cineform are all what are known as "intermediate codecs". This means that they're designed to be used to transcode footage from other sources into a form that's easy for video editing programs to work with while maintaining quality. So, unlike most h.264 implimentations, they focus on low CPU usage, retaining as much detail as possible,  and an ability to be re-compressed several times without significant loss in quality. They have larger bitrates than consumer video codecs, but they still represent a significant space savings over fully uncompressed video.

DNxHD and DNxHR are Avid's intermediate codecs, designed to work well with Avid Media Composer, and as competition for ProRes. They have roughly equivalent quality to ProRes, but, like Media Composer, are organized in a more complicated manner.

DNxHD

DNxHD is the first variant of Avid's intermediate codec, and focuses (as the name suggests) on high-definition resolutions. It works great, but has an incredibly unintuitive naming system for different quality levels and resolutions based on the overall bitrate of the video. If you really want to take a look at the full list, check out this Wikipedia page:

https://en.wikipedia.org/wiki/List_of_Avid_DNxHD_resolutions

In FFMPEG, you need to set the bitrate properly in order for the built-in DNxHD encoder to work. Here's an example command using a 1280x720 h.264 .mp4 file running at 29.97fps:

ffmpeg -i "inputvideo.mp4" -c:v dnxhd -b:v 110M -pix_fmt yuv422p -c:a pcm_s16le "outputvideo.mxf"

Using the list of resolutions, we find that is the correct bitrate to output an 8-bit 422 chroma subsampled 720p video at 29.97fps is 110 megabits, which is denoted by that "110M" setting in the command.

Unfortunately, the official list of bitrates isn't exact. For example, looking at the list, you might assume that if you wanted to do 10-bit instead of 8-bit, you would think that you just had to change the colorspace to yuv422p10le (like we do for ProRes) and keep the bitrate the same. However, that will give you an error. Luckily, DNxHD errors in FFMPEG will list the correct range of supported bitrates. Let's take a look (cropped to relevant section and cleaned up a bit for readability):

Frame size: 1280x720p; bitrate: 90Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 180Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 220Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 90Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 110Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 180Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 220Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 60Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 75Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 120Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 145Mbps; pixel format: yuv422p

Unfortunately, this list doesn't display framerates, but you'll find that these bitrates work for any framerate within the same frame size and pixel format. In this case, the answer for 10-bit 720p/29.97fps is 180M. So, the command for a 10-bit output would look like this:

ffmpeg -i "inputvideo.mp4" -c:v dnxhd -b:v 180M -pix_fmt yuv422p10le -c:a pcm_s16le "outputvideo.mxf"

Just for reference, here's the complete list that FFMPEG spits out:

Frame size: 1920x1080p; bitrate: 175Mbps; pixel format: yuv422p10
Frame size: 1920x1080p; bitrate: 185Mbps; pixel format: yuv422p10
Frame size: 1920x1080p; bitrate: 365Mbps; pixel format: yuv422p10
Frame size: 1920x1080p; bitrate: 440Mbps; pixel format: yuv422p10
Frame size: 1920x1080p; bitrate: 115Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 120Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 145Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 240Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 290Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 175Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 185Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 220Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 365Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 440Mbps; pixel format: yuv422p
Frame size: 1920x1080i; bitrate: 185Mbps; pixel format: yuv422p10
Frame size: 1920x1080i; bitrate: 220Mbps; pixel format: yuv422p10
Frame size: 1920x1080i; bitrate: 120Mbps; pixel format: yuv422p
Frame size: 1920x1080i; bitrate: 145Mbps; pixel format: yuv422p
Frame size: 1920x1080i; bitrate: 185Mbps; pixel format: yuv422p
Frame size: 1920x1080i; bitrate: 220Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 120Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 145Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 90Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 180Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 220Mbps; pixel format: yuv422p10
Frame size: 1280x720p; bitrate: 90Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 110Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 180Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 220Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 60Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 75Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 120Mbps; pixel format: yuv422p
Frame size: 1280x720p; bitrate: 145Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 36Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 45Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 75Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 90Mbps; pixel format: yuv422p
Frame size: 1920x1080p; bitrate: 350Mbps; pixel format: yuv444p10, gbrp10
Frame size: 1920x1080p; bitrate: 390Mbps; pixel format: yuv444p10, gbrp10
Frame size: 1920x1080p; bitrate: 440Mbps; pixel format: yuv444p10, gbrp10
Frame size: 1920x1080p; bitrate: 730Mbps; pixel format: yuv444p10, gbrp10
Frame size: 1920x1080p; bitrate: 880Mbps; pixel format: yuv444p10, gbrp10
Frame size: 960x720p; bitrate: 42Mbps; pixel format: yuv422p
Frame size: 960x720p; bitrate: 60Mbps; pixel format: yuv422p
Frame size: 960x720p; bitrate: 75Mbps; pixel format: yuv422p
Frame size: 960x720p; bitrate: 115Mbps; pixel format: yuv422p
Frame size: 1440x1080p; bitrate: 63Mbps; pixel format: yuv422p
Frame size: 1440x1080p; bitrate: 84Mbps; pixel format: yuv422p
Frame size: 1440x1080p; bitrate: 100Mbps; pixel format: yuv422p
Frame size: 1440x1080p; bitrate: 110Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 80Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 90Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 100Mbps; pixel format: yuv422p
Frame size: 1440x1080i; bitrate: 110Mbps; pixel format: yuv422p

You'll notice some 10-bit 444 color profiles in the 1920x1080p section. Don't use those unless you're coming from a 444 source, or you're wasting hard drive space.

DNxHR

So what about DNxHR? This is the more modern variant of DNxHD that supports a wider range of resolutions and color options. More importantly, it's way simpler to work with.

There's no need to specify bitrate, it will automatically use the correct one based on your input file. Unfortunately, most HR codec variants are only 8-bit, and will give an error if you try to convert to a different bit depth. The options for DNxHR are:

LB (roughly equivalent to ProRes Proxy)
SQ (roughly equivalent to ProRes LT)
HQ (roughly equivalent to ProRes 422)
HQX (roughly equivalent to ProRes 422HQ)
444 (you guessed it, roughly equivalent to ProRes 444)

Both the 444 and HQX variants use 10-bit color in HD or less, and either 10 or 12-bit color at 2K/UHD/4k resolutions. LB through HQX will always use 422 chroma subsampling, 444 will always use, well, 444 chroma subsampling. Still Confusing? You bet.

What's worse is that FFMPEG won't actually encode to 12-bit DNxHR HQX/444 or ProRes 4444XQ, it only supports 10-bit color depth.

That all said, here's a command that will give the same video quality as the first DNxHD command above, but using DNxHR:

ffmpeg -i "inputvideo.mp4" -c:v dnxhd  -profile:v dnxhr_hq -pix_fmt yuv422p -c:a pcm_s16le "outputvideo.mxf"

Notice how the codec variant is specified at the end of the profile option with an underscore before it.

If you wanted to transcode to HQX 10bit from a file with a different bitdepth/colorspace, you would do something like:

ffmpeg -i "inputvideo.mp4" -c:v dnxhd -profile:v dnxhr_hqx -pix_fmt yuv422p10le -c:a pcm_s16le "outputvideo.mxf"

To go to 10-bit 444, you would need to give it a 2K or larger resolution file and do:

ffmpeg -i "inputvideo.mp4" -c:v dnxhd -profile:v dnxhr_444 -pix_fmt rgb24 -c:a pcm_s16le "outputvideo.mxf"

As before, you generally don't want to use 444 unless you're coming from a 444 colorspace file, or a raw capture format that FFMPEG can understand. Otherwise, it's a waste of storage space, and is much slower to encode.

Oh, and if you're wondering how the FFMPEG DNxHD/HR encoder compares to the equivalent presets in Adobe Media Encoder, know that it's very close, but just slightly softer.