PLEASE NOTE: This tutorial is now significantly out of date. I've provided it only for historical reference, and am currently working on a followup. The instructions below WILL NOT WORK with current versions of VapourSynth.
It took me about a morning to get everything I needed and set up a sample script for QTGMC conversion. I decided to try to keep the setup as bloat-free as possible by using "portable" versions of the apps involved.
Here was my ultimate workflow. I won't try to explain everything since I'm not totally fluent in Python, but adapting the following settings should allow you to get it work:
BIG DISCLAIMER: This process may not work, may crash, or do other things to your system. Virus scanning all files is strongly encouraged, but not a 100% guarantee of safety.
You have been warned.
If you're on a deadline (and using Premiere Pro, After Effects, or Final Cut Pro) probably your best best is to use a paid plugin like FieldsKit. And no, they aren't paying me to say that.
Also, this tutorial is for Windows 10. Most of the steps work for other OSes, but I won't cover the differences here.
Here's a video version of the tutorial:
First, grabbed the embeddable version of Python 3.7.X:
https://www.python.org/downloads/
(Click on the name of the latest version of Python 3.7, then scroll down to find the embeddable version.)
Then, downloaded the portable VapourSynth:
https://github.com/vapoursynth/vapoursynth/releases
(I'm using the 64-bit portable version.)
Then, grabbed VapourSynth Editor (VSEdit):
https://bitbucket.org/mystery_keeper/vapoursynth-editor/downloads/
Extracted the Python archive to a directory. Extract both VapourSynth and VSEdit to the same directory, in that order.
Now, for the needed plugins and VapourSynth Python modules:
FFmpegSource:
https://github.com/FFMS/ffms2/releases
havsfunc
https://github.com/HomeOfVapourSynthEvolution/havsfunc/releases
(The source code is what you want here.)
mvsfunc
https://github.com/HomeOfVapourSynthEvolution/mvsfunc/releases
adjust
https://github.com/dubhater/vapoursynth-adjust/releases
nnedi3_resample
https://github.com/mawen1250/VapourSynth-script
(Click on the "Clone or download button" and select "Download ZIP")
I also grabbed fmtconv for colorspace conversion:
https://github.com/EleonoreMizo/fmtconv/releases
Then, the VapourSynth versions of the needed QTGMC prerequisites:
https://github.com/dubhater/vapoursynth-mvtools/releases
https://github.com/dubhater/vapoursynth-nnedi3/releases
(You'll also need nnedi3_weights.bin from here. Left-click on the link, don't right-click/save)
Scanned all the above files for viruses.
Extracted the .py files in the main directory. Extracted the (64-bit) .dll files to the vapoursynth64/plugins directory
Opened VSEdit. Made the following initial script:
import vapoursynth as vsNote that the final backslash before the input file name needs to be "escaped" with another backslash.
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='F:\directory\\input movie.mov')
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540, matrix_in_s='709')
clip.set_output()
With the above settings, the source colorspace will be preserved, but the color matrix will be shifted to REC.709 on resize. To change both the output colorspace and color matrix, use:
clip = core.resize.Spline36(clip, 720, 540, format=vs.YUV422P10, matrix_in_s='709')If you're coming from a source file with a non-recognized colorspace, you can use:
clip = core.fmtc.resample (clip=clip, css="420")right after the ffms2 command.
clip = core.fmtc.bitdepth (clip=clip, bits=8)
Update: As UniveralAl1 mentions in a comment on the tutorial video, it may be possible to skip this by just converting once to YUV422p10 before QTGMC. The resulting script might look something like this:
import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='E:\Archive\\input video.avi')
clip = vs.core.resize.Point(clip, format = vs.YUV422P10)
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540)
clip.set_output()
Please try this first rather than the final script below.
All of the above ended up being necessary for the video file I selected for testing due to it using 4:1:1 chroma subsampling, so here is my final script:
import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()
clip = core.ffms2.Source(source='E:\Archive\\input video.avi')
clip = core.fmtc.resample (clip=clip, css="420")
clip = core.fmtc.bitdepth (clip=clip, bits=8)
clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Spline36(clip, 720, 540, format=vs.YUV422P10, matrix_in_s='709')
clip.set_output()
Saving this script gives you a .vpy file.
To render it out, I used a combination of vspipe and FFMPEG as per the documentation. However, VapourSynth does not handle audio and video at the same time, so I had to use the mapping command in FFMPEG to copy over the audio separately:
vspipe --y4m Upscale.vpy - | ffmpeg -i pipe: -i "C:\pathto\inputmovie.mov" -c:v prores -profile:v 3 -c:a copy -map 0:0 -map 1:1 "F:\Temp\output file.mov"
Preliminary testing gives me 67 fps average for VapourSynth and 77fps average for AVS+ using the following script:
SetFilterMTMode("QTGMC", 2)
FFmpegSource2("ITVS Trailer.avi", atrack=1)
ConvertToYV12()
AssumeBFF()
QTGMC(Preset="Slower", EdiThreads=3)
Spline36Resize(720, 540)
Prefetch(10)
The only difference I noticed in the resulting files was that bright red colors bled upwards in the VS encode, and downwards in the AVS+ encode. If I figure out more, will report back.
If you have any input or suggestions, feel free to leave a comment below.
13 comments:
Very cool, thank you! I've been looking for a way for QTGMC to be more stable, I hope using VaporSynth will get me that stability.
Could you please provide samples of the bright red areas bleeding?
Just as an FYI, VapourSynth is more stable than classic AVISynth, but AVISynth+ also does this, and doesn't require Python commands or Vspipe at the end.
As to the bleeding, I don't have VapourSynth installed anymore, but the difference isn't massive. it basically just shifts drastically red parts of the image up by a couple of pixels. There's always going to be red bleeding on YV12 video anyways because of the lower resolution of chroma info, so unless you're trying to do an absolutely 100% pixel perfect version of your original video, I personally don't think it's a big deal.
However, if that's a concern for you, AVS+ doesn't have this behavior.
I'm not afraid of Python or pipes, but if AVS+ can get me more stability then I'll try it first.
Thanks!
about that youtube video,
-color space is changed two times, unnecessarily, because QTGMC can handle YUV422P10, so it could be resized from YUV411P8 input into that and using Zimg (internal Vapoursynth resize), fmtc does not have to be used
-so bleeding etc., might be introduced right there if going from 411 to 420 then to 422
-also matrix_in_s is input matrix, not output matrix,
matrix is used if changing YUV to RGB. If not, it is not necessary for the resize, YUV values are not transformed to other color space , just changed from YUV to YUV again, different bitdepth and mapped from 0-255 (8bit)to 0 to 1023 (10bit)
script:
import vapoursynth as vs
import havsfunc
clip = vs.core.ffms2.Source('/home/user_name/Python/DV.avi') #NTSC video loaded as YUV411P8
clip = vs.core.resize.Point(clip, format = vs.YUV422P10)
clip = havsfunc.QTGMC(clip, Preset = 'Medium', TFF = False)
clip = vs.core.resize.Spline36(clip, width = 720, height = 540)
clip.set_output()
To comment on this line:
clip = vs.core.resize.Point(clip, format = vs.YUV422P10)
Point resize was used to get 422 but upsampling does not makes new colors. Yes colors have new 10bit values, not 8bit, but they are the same color at the end. And we want to have same colors going to QTGMC as the source had. If we used:
clip = vs.core.resize.Spline36(clip, format = vs.YUV422P10)
colors might be shifted or changed even before going to QTGMC
Only after QTGMC, there is going to be resolution change so we might use Spline36.
Also different resize method could be used for Y and chroma if desired, check:
http://www.vapoursynth.com/doc/functions/resize.html
also Vapoursynth author mentioned audio support should be coming so stay tuned ...
Tony,
Unfortunately, with DV25 (MiniDV) video, it uses 411 chroma subsampling, which is incompatible with QTGMC. Therefore, I still have to convert to either 420, 422, or 444. If the subsampling is already any of those in the file you're using, then you would be absolutely correct about not needing to change subsampling before QTGMC. I will make a note of that in the blog post.
I've watched most of your videos about deinterlacing and learn a lot, so thank you. I have two general questions about deinterlacing and a specific question about AviSynth+.
-Which one of the deinterlacing tools has better quality? Field Kit or the QTGMC (Consider my sources are DVDs and I have the Plugin for AE)? And as far as QTGMC, is there any difference in quality or rendering speed between AviSynth+ or VapourSynth?
-In the case of using QTGMC and correcting the ratio, since we encoding to ProRes, isn't it better to fix the ration in AE, or are there issues to export as is (720x480 or 720x576) and fix it later?
-In AviSynth+ x64, the Edithreads and Prefetch different values causes the video to be rendered with wired, random pixelation. I finally managed to find the Interesting things right combination, but any idea why?
Harmik,
QTGMC is definitely better, but is harder to set up (as you can probably tell). There isn't a massive difference between VapourSynth and AVS+ as far as I've seen - as long as you're using the 64-bit version of AVS+.
Aspect ratio correction is a preference. I like taking care of it first if I'm working in a 1:1 PAR project so I can just drop it in, but you can do either.
What was your input video's settings (codec, framerate, source)? That might have an impact. Also, was this pixellation only in AVSPMod, or in the final render?
Got it, AVS+ is then.
The only reason I was thinking correct the ratio later in Ae was to get a better quality, but I have not tried this, so If you have, please let me know.
The input was .264 @ 60i if I am right about the FPS. Here is the ffmpeg info;
Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.92 fps, 29.97 tbr, 1200k tbn, 59.94 tbc
The pixelation is happening on both, AVSPMod, as well as the exported ProRes MOV file. I prefer to demux the source media to the RAW video code and then convert only the video to ProRes and, after any editing in Pr export to H265 using Media Encoder.
I finally managed to convert the video without pixelation, but I will just share some more info, and what I experienced, which I guess it is kind of confusing, so sorry about that.
My CPU is i7 6850K, 6 Core / 12Threads, and it is not overclocked.
I started with Edithreads=6 / Prefetch=12, no issues while encoding, CPU usage was around %100, but the final video has some pixelation. I began changing values, and with Edithreads=6 / Prefetch=8, finally, no pixelation, and even though the CPU usage was usually around %75, the encoding time was about %10 to %12 faster. More interesting is, the sweet spot was when Edithreads=6 / Prefetch=6, which was fastest.
SetFilterMTMode("QTGMC", 2)
FFmpegSource2("Input.264")
ConvertToYV12()
AssumeTFF()
QTGMC( Preset="Slower", FPSDivisor=2, Edithreads=6)
Prefetch(6)
I also have a MacBook Pro with Windows, 4 Core / 8 Threads, and tried the whole process. In the beginning, I used the same script, Edithreads=6 / Prefetch=12, and forgot the CPU is not the same, which was obviously wrong setting, and I have the same problem, but after changing to Edithreads=4 / Prefetch=8, the maximum values for the MacBook Pro, the video was converted clean.
Also, I even tried to avoid the Edithreads / Prefetch by using TR2 and make sure the source file itself is right, and the results were good, a little bit soft to me, and almost 3 times slower, but no pixelation.
SetFilterMTMode("QTGMC", 2)
FFmpegSource2("Input.264")
ConvertToYV12()
AssumeTFF()
QTGMC( Preset="Slower", FPSDivisor=2, TR2=3)
I believe interlaced h.264 has some issues with FFMPEG, which might be the cause of the artifacts. Unfortunately, I don't know of an input plugin that loads h.264 other than FFMPEGSource, so you might be stuck with using that and just tweaking stuff until it works. It's possible you might have some weird memory bug with the higher EdiThreads/Prefetch settings, but my bet is the issue is your source video.
I see your point, I guess you are right about the memory bug. Thanks gain for tips and help.
Post a Comment