Tutorial - Deinterlacing and upscaling with 32-bit QTGMC revisited

Hey folks - just wanted to let you know that this tutorial is now officially outdated.  I'm leaving it up here are for up historical interest, but here's a link to a post that I will maintain from this point forward with the current instructions for setting up AVISynth and QTGMC:

https://macilatthefront.blogspot.com/2021/01/deinterlacing-with-avisynth-and-qtgmc.html

================================================

After more than a year and a lot more experience with my SD to HD workflow, I’ve made a few changes. That means it’s time for an update.

Some of the larger changes include switching to AVISynth+ instead of standard AVISynth plus AVISynth MT, dropping the “source match” and “lossless” presets from QTGMC, and updating a whole mess of plug-ins. In total, these changes result in better speed, stability, and image quality.

BIG DISCLAIMER: This process may not work, may crash, or do other things to your system. I've had an issue with intermittent audio dropouts in the resulting files, but it seems to only happen with Quicktime ProRes files from a certain source. Always check your files before submitting/uploading them.

Virus scan everything you download. It's not a 100% guarantee that you'll avoid getting a malware infection, but it's a lot better than not checking at all.

You have been warned. 

If you're on a deadline (and using Premiere Pro, After Effects, or Final Cut Pro) probably your best best is to use a paid plugin like FieldsKit. And no, they aren't paying me to say that.

Also, this tutorial is for Windows 10. Most of the steps work for previous versions of Windows, but may require slight modifications. Users of MacOS and other OSes should look elsewhere.

Here's the video version of the setup and a basic QTGMC deinterlacing workflow:

And here's a video about cropping and upscaling:

 First of all, we're going to need to get AVISynth+. You can grab it from here:

https://github.com/pinterf/AviSynthPlus/releases

Then, we'll need to get all the filters (plugins) needed:

FFMPEGSource

This is what will allow AVISynth+ to load our video files. Works with almost any container/codec combination, but read the "Known Issues" section for things to look out for. Also, don't bother with the "10-bit hack version", as we won't be working in 10-bit color.

11/6/2018 Update: looks like Jiangmin antivirus is currently flagging FFMpegSource with a warning on VirusTotal. Given that no other antivirus engines have spotted anything, I'm cautiously calling this a false positive, especially because one of the top search results for Jiangmin on Google is a post about the problems with reporting false positives to them.

QTGMC

The deinterlacing plugin of choice. Requires a whole host of additional plugins, of which you will be using the 32-bit versions (if there's a choice). For most uses, however, only the "Core Plugins and scripts" are necessary. Thaks to one anonymous commenter, I should mention that if you're trying to import YUY2 colorspace video (some pro video codecs use this) without converting, you'll also need the AnimeITVC plugin.

Update: the 64-bit version of QTGMC and related programs are now stable, and slightly faster. You can find info on how to install those versions here:

https://macilatthefront.blogspot.com/2018/10/64-bit-avisynth-and-qtgmc-update-and.html

For the SmDegrain link, go to the page and click on the tiny "download" link next to the "raw" link. When you save the file to your computer, rename the .txt extension to .avsi.

If you decide to grab the optional plugins, then for the AnimeITVC pastebin link, select the text of the link and copy it, then right-click on the link and do "Save Page As", select "All Files" in the Save As drop-down box, paste the name of the file, delete any spaces from the front of the filename, then save the file. Most people won't need this plugin, thankfully.

Whatever you do, DO NOT forget to download the 32-bit FFTW3 library as well. Without it, QTGMC will not run.

If you need to sharpen the image, I recommend LimitedSharpen (Technically "LimitedSharpenFaster"). Get the x86 version in order for it to work within a script using 32-bit QTGMC.

With all the AVISynth filters and scripts grabbed, it's time to get the supporting software:

AvsPMod

This is like an IDE for AVISynth scripts, and is pretty much essential IMO. Grab the 32-bit version for this tutorial.

FFMPEG

Update: since Zeranoe has shut down, I will be recommending compiling 32-bit FFMPEG from source using https://github.com/m-ab-s/media-autobuild_suite . Since this tutorial is getting a little long in the tooth any ways, I'll just leave this here as a placeholder until I create a newer and better version. 


7-zip

If you don't have 7-zip already installed, you won't be able to open many of the downloaded archives. The version you get here doesn't matter, so maybe just do the 64-bit one.

After virus scanning everything, install:

7-zip
AVISynth+

Just follow the default options in both cases. Then, go to the plugins directory for AVISynth+. On my system, the default installer creates a plugin folder at

C:\Program Files(x86)\AVISynth+\plugins+\

Extract all .dll and .avsi files to the plugins directory from all plugins EXCEPT the fftw-3*.dll archive. If there's a choice between x86 and x64, use the x86 version. If there's a choice between AVX, AVX2 and a version without AVX, you'll need to know what instruction sets your processor supports. CPU-Z can tell you this if you're not sure.

Now, open the fftw-3*.dll archive, then (as noted on the QTGMC page) extract the libfftw3f-3.dll file. Make a copy of it and rename it as "FFTW3.dll". Place the files "libfftw3f-3.dll" and "FFTW3.dll" in the SysWow64 folder. Don't ask me why you have to do this, I agree that is seems pointlessly tedious.

Extract the AvsPMod archive to wherever you want to run it from.

Finally, open the ffmpeg archive, navigate to the "bin" directory, and extract the ffmpeg.exe file to whatever directory your video file/files are located in.

Alternatively, you can extract the archive to somewhere on your system and then set up FFMPEG to run from any directory by adding it to your PATH variable:

1. Press the Windows and R keys.
2. Type "control sysdm.cpl,,3". 
3. Click "Run".
4. Click on "Environment Variables".
5. Select "Path" under "System variables" and click "Edit".
6. Go to where you extracted ffmpeg.exe. 
7. Select and copy the folder's path address from the address bar towards the top of the window.
8. Back in the "Edit environment variable" window, click "New" and paste in the folder path. Click OK on all the windows you opened to get here.

Now that everything's ready, let's go to the directory with your video files and make an .avs script. Right-click anywhere in the directory, select New, then AVISynth Script. Alternatively, you can just create a new text file and chance the .txt extension to .avs.

Right-click on the .avs script, choose "Open With", then "Choose Another App", then check "Always use this app to open .avs files", then scroll down the list of apps until you see "More Apps". Click on that, scroll some more until you see "Look for another app on this PC". Click on that, navigate to where you extracted AvsPmod, select avspmod.exe and click OK. Your .avs scripts will now automatically open in AvsPMod when you double-click on them.

In a few moments, you should now see your blank script open in AvsPmod.

Here's my boilerplate .avs script settings for deinterlacing:

    SetFilterMTMode ("QTGMC", 2)
    FFMPEGSource2("videofile.avi", atrack=1)

    ConvertToYV12()
    AssumeBFF()
    QTGMC(preset="Slower", EdiThreads=1)
    BilinearResize(720,540)
    Prefetch(10)


The Prefetch number should be set to 1 or 2 less than the number of "logical processors" in your system. To check what that is, go to Task Manager in Windows, select the Performance tab, then click on the CPU item. You'll see down below in the window an entry for the number of logical processors in your system. For some people, it will be the same as the number of physical cores, for others it might be double that if your CPU supports Hyperthreading. In my case, I have 12 logical processors, so I set Prefetch to 10.

On the other hand, EdiThreads is supposed to be set to half the number of physical cores in your system, and if so can give a mild speed boost. With the current workflow, I've been able to set it to 3 with no issues, but your mileage may vary. If you experience any random rendering freezes (not overall system freezes), then lower this number first.

Also, a quick note here: Amazon and some other streaming services prefer 640x480 over 720x540 for SD content. I use the latter because I consider the resulting file a new "master" file that can be used to create future derivative versions, but if you're going to send the file directly to a distributor or service, read/ask for their deliverables spec to see what resolution and framerate they want.

If you need to output 29.97fps rather than 59.94fps at this stage, then add FPSDivisor=2 to the QTGMC command like so:

    QTGMC(preset="Slower", FPSDivisor=2, EdiThreads=1)

Anyways, here's what to add if you want to upscale to HD with a center crop:

    Crop(0,68,0,-68)
    Spline64Resize(1280,720)

Note that the numbers in the crop command refer to left,top,right,bottom respectively, and the right and bottom numbers are negative, while the left and top numbers are positive. Don't ask me why AVISynth does this, it just does.

The above settings will give you as close to a 16x9 frame as you can get out of this resolution. Technically, 720x405 is true 16x9, but the crop command doesn't like odd numbers, so 720x404 is the closest you can get.

Also, this may crop off more than you want, so feel free to play around with the settings. If you want to frame a different part of the image (like the top or bottom versus the middle), then just adjust the crop numbers accordingly by adding what you subtract from one number to the other, or vice versa. For example, to use just the top part of the frame, you would do:

    Crop(0,0,0,-136)

If your initial video has noticeable compression artifacts, you might want to do BilinearResize instead of Spline64Resize to avoid accentuating said artifacts. If it's clean and you want to sharpen even more, LimitedSharpenFaster does a a good job of this.

    LimitedSharpenFaster()

It has a ton of additional features listed on the wiki page, which you can play around with if the defaults don't work for you.

Loading this script in AvsPmod will let you preview your results, and change to your preference. To preview your settings, click anywhere on the playback bar at the bottom of the window. The first time you do this, FFmpegSource will index your file, so be patient. When you're done, don't forget to save your work.

Create a new text file in the same directory and change its extension to .bat. Add the FFMPEG commands of your choice. For example, here's a command to encode an .avs script to a ProRes 422HQ Quicktime file:

    ffmpeg -i "videofile.avs" -c:v prores -profile:v 3 -pix_fmt yuv422p10le "output.mov"

If you need to convert to a different video format, search for options and further instructions on the FFMPEG Wiki.

PS: FFMPEG doesn't generate "official" ProRes files, so if it will be the last stop in your export chain, consider using a different workflow if the file is going to a major TV channel. Some channels have reportedly (at least according to editors on Reddit's /r/editors subreddit) said that FFMPEG ProRes files won't pass QC with them.