AssRender: inappropriately named libass-based subtitles renderer

[TheFluff]

I know some of you insist on encoding on lunix. This is generally a pretty bad idea but if you insist you can actually do it using avs2yuv, wine and some other funky things. There's only one catch, you can't hardsub stuff because VSFilter doesn't work under wine.

Now, not that I encourage hardsubbing, but one fine day I got bored and decided to solve that problem. This avisynth filter is exactly like textsub(), but it uses libass instead of vsfilter. This thread is a cross-post from doom9 because I think this might be relevant to your interests.


Download
AssRender 0.11
NOTE: the included fontconfig directory MUST be in the same folder as the .dll, or it'll crash!
Note 2: when using it for the first time it'll probably take a while to open the script, especially if you have a lot of fonts installed, because fontconfig needs to cache all your system fonts.

Source code (under MIT license, binaries are under GPL for obvious reasons): assrender_0.11-src.7z


Syntax:

assrender(clip c, string file, int "hinting"=2, float "scale"=1.0, string "charset"="UTF-8", int "loglevel"=-1, string "logfile"="")

Parameter explanations (all parameters except the clip and the input file are optional):

• file: The .ass file to render; no other subtitle format is currently supported.
• hinting: What kind of font hinting to use. Valid values are 0-3; see below for further explanations of this.
• scale: How much to scale the rendered text. Default is 1.0, i.e. 100% scaling (no change).
• charset: The character set of the .ass file, in the standard GNU iconv format (i.e. ISO-8859-1, UTF-16, CP1252, etc are all recognized).
• loglevel: How much diagnostics libass should output to the logfile. Valid values are from -1 to 7, where -1 means nothing, 0 means fatal errors only and 7 means output several hundred kB of internal data structures. 5 or 6 is probably sane if you want warnings about things that might have gone wrong.
• logfile: Where to write the log. If you set loglevel >= 0, you must specify this too.

Regarding hinting:

• 0 means disable hinting completely. Might be useful with problematic fonts.
• 1 means light autohinting. This is what libass recommends for compatibility.
• 2 means normal hinting.
• 3 means use FreeType's native hinting, which may or may not be buggy.

Normally you'd use 2 or 1, unless things look odd.


Advantages and disadvantages
This filter should mainly have two advantages over VSFilter, namely:
a) it works under wine, and
b) it's probably a lot faster.

On the other hand it has a few limitations, most notably it only supports RGB32 input so far. If someone is sitting on a fast (preferably assembly optimized) routine that can overlay RGBA on YV12/YUY2, feel free to speak up.

Furthermore, libass isn't really bug-for-bug compatible with VSFilter, so it might render stuff a bit differently compared to what you're used to.


Todo

• Expose more libass parameters (aspect ratio compensation and default font comes to mind)
• Make overlaying faster (assembly optimize it?)
• Use Haali's matroska parser to support things like assrender("file.mks", track=2)
• Get rid of the fontconfig configuration directory
• Implement VFR compensation
• Implement an equivalent of VSFilter's MaskSub()
• Steal some subtitles parser code for various formats from Aegisub, convert things to ASS internally and support more subtitle formats than just .ass

Other stuff
It's compiled against a fairly recent libass, this one unless I misremember. I know that technically I'm probably not GPL-compliant because I'm not distributing the source code of the exact versions of freetype/fontconfig/expat/zlib it's linked against (Debian Squeeze's versions including their patches as of three days ago, apt-get source if you want them), but you know, I really don't give a fuck.

If you want to compile the source code you need a working mingw32 environment. If you don't want to compile libass and its deps yourself (afaik it's incredibly hard to get libass to build on msys-mingw32 because of the permanent state of autotools hell it is in; personally I didn't bother and just crosscompiled on linux), you can get my compiles. Oh, and you need stdint.h for Visual Studio too just to make it even more annoying to compile.

[Starks]

Not to be a nag, but assrender is already the name of the libass subtitle renderer for gstreamer on Linux and has been for at least a few months now.

<____<

[jfs]

Originally Posted by Starks

Not to be a nag, but assrender is already the name of the libass subtitle renderer for gstreamer on Linux and has been for at least a few months now.

And why is that a problem?

(Btw. that is already known and was considered in the naming of this plugin.)

[martino]

Originally Posted by TheFluff

I know some of you insist on encoding on lunix. This is generally a pretty bad idea but if you insist you can actually do it using avs2yuv, wine and some other funky things. There's only one catch, you can't hardsub stuff because VSFilter doesn't work under wine.

Last time I checked it did... just need to set the style spacing (or w/e it's called) to anything other than 0, so like 0.001 for example, in order to have the full line rendered.

But this looks like an interesting alternative nonetheless.

[TGEN]

TheFluff just needs to rename it "AssReamer", and all will be good.

[fireshark]

"I use AssReamer in Wine"

[palmje]

Originally Posted by TheFluff

I know some of you insist on encoding on lunix. This is generally a pretty bad idea but if you insist you can actually do it using avs2yuv, wine and some other funky things.

And what is so bad about encoding on Linux?

And why if you want to use Linux, would you want to use a tool designed for windows, when there are perfectly good native alternatives like Mencoder (and there is even a rewrite of avs that is cross-platform, though for linux it uses gstreamer, which, lets face it, sucks)?

[Daiz]

Mencoder equally capable as AviSynth for video processing?

Yeah, suuuuuure.

[twc]

I don't know about you, but my copies of MPlayer and MENcoder can play and process AviSynth just fine.

[TheFluff]

Originally Posted by palmje

And what is so bad about encoding on Linux?

And why if you want to use Linux, would you want to use a tool designed for windows, when there are perfectly good native alternatives like Mencoder (and there is even a rewrite of avs that is cross-platform, though for linux it uses gstreamer, which, lets face it, sucks)?

mencoder has its uses but it is completely worthless for everything that involves the slightest bit of filtering/non-linear editing/advanced fieldmatching; it is comparable with (but generally inferior to) virtualdub but cannot do a fraction of the things avisynth can do

avs3 is vaporware and will never do anything useful ever

BONUS CHALLENGE that you can take if you want to back your troll up: find me native linux tools that can:
- cut/edit isdb transport streams without reencoding
- filter video with a set of filters that is comparable to avisynth's, allowing selective filtering of different part of the video in an arbitrary way
- open and decode transport streams for display/reencode in a frame-accurate manner (anything that uses ffmpeg is not the answer, libavformat is guaranteed to be 1-2 frames off when seeking in mpeg ts/ps)
- do semi-automated manual improvement of ivtc matches

[palmje]

Originally Posted by Daiz

Mencoder equally capable as AviSynth for video processing?

Yeah, suuuuuure.

I never intended for it to mean that Mencoder was as capable as AviSynth, just that Mencoder can do just as good a job for the tasks that are typically needed - I have never found a situation where I have needed those features though.

However, if you do need to filter different parts of the video differently for whatever reason then it can be done, but it's certainly not a nice (or even easy) way of doing it. Mencoder also doesn't have the range of filters AviSynth has, though it does have a decent range of filters.

Originally Posted by TheFluff

mencoder has its uses but it is completely worthless for everything that involves the slightest bit of filtering/non-linear editing/advanced fieldmatching; it is comparable with (but generally inferior to) virtualdub but cannot do a fraction of the things avisynth can do

I've found the mencoder filters to be good - never had any problems with them, though admittedly you don't have the same range or number of alternatives for them. Yes, Mencoder does suck for non-linear editing, but how often do you really need to do that? What do you consider advanced fieldmatching to be?

Originally Posted by TheFluff

avs3 is vaporware and will never do anything useful ever

Don't really know much about it, but it's design idea seems pretty good. Any particular reason you think it'll never do anything useful?

Originally Posted by TheFluff

BONUS CHALLENGE that you can take if you want to back your troll up: find me native linux tools that can:
- cut/edit isdb transport streams without reencoding
- filter video with a set of filters that is comparable to avisynth's, allowing selective filtering of different part of the video in an arbitrary way
- open and decode transport streams for display/reencode in a frame-accurate manner (anything that uses ffmpeg is not the answer, libavformat is guaranteed to be 1-2 frames off when seeking in mpeg ts/ps)
- do semi-automated manual improvement of ivtc matches

Mencoder can cut TS files without reencoding the video, though the audio does have to be reencoded - Mencoder doesn't seem to be able to do the cuts on the AAC audio for some reason

From what I've seen the Mencoder filters are just as good as the avs ones, though the range of filters isn't as good and you can apply them arbitrarily, though it's far from pretty or easy.

"open and decode transport streams for display/reencode in a frame-accurate manner " - by this do you mean for the cuts in the file? If so them Mencoder has a high resolution mode which should be frame accurate, though it is slower since it decodes each frame rather than seeks. Other than that you could look at transcode or handbrake (never used them myself though...)

"semi-automated manual improvement of ivtc matches" - no idea. Never had any need to do this at all.

[TheFluff]

Originally Posted by palmje

I've found the mencoder filters to be good - never had any problems with them, though admittedly you don't have the same range or number of alternatives for them. Yes, Mencoder does suck for non-linear editing, but how often do you really need to do that? What do you consider advanced fieldmatching to be?

anything that involves cutting and splicing is a gigantic pain in the ass with mencoder

Originally Posted by palmje

Don't really know much about it, but it's design idea seems pretty good. Any particular reason you think it'll never do anything useful?

because it has never done anything at all besides produce blankclip()s (it doesn't even have a script parser) and it has been completely abandoned by all of its developers for over three years, and at the current rate we're going to see a linux port of 2.5 long before we see anyone do any work on avs3 again (this is because the latter will never happen ever)
parts of the design (ABI compatibility with 2.5 filters for example) are also patently insane

Originally Posted by palmje

Mencoder can cut TS files without reencoding the video, though the audio does have to be reencoded - Mencoder doesn't seem to be able to do the cuts on the AAC audio for some reason

afaik mencoder uses libavformat for mpeg ts parsing and is thus disqualified since it's guaranteed to be at least 1-2 frames off

Originally Posted by palmje

From what I've seen the Mencoder filters are just as good as the avs ones, though the range of filters isn't as good and you can apply them arbitrarily, though it's far from pretty or easy.

whoop de doo you have hqdn3d, smartblur and uspp for denoising, unsharp for sharpening and yadif for deinterlacing
sure is some quality filtering there, bro

I mean, hqdn3d was generally abandoned because it was only really useful for realtime denoising (there were much better alternatives if you could take a bit of slowness), nobody has actually used unsharp for anything serious in the last five years and while yadif is good at what it's designed for (realtime deinterlacing on playback) it really doesn't match up to something like nnedi, much less mcbob.

I have no idea how good/bad mencoder's ivtc filters are in general, but since they cannot handle vfr/hybrid material they are all completely useless for a lot of anime.

Originally Posted by palmje

"semi-automated manual improvement of ivtc matches" - no idea. Never had any need to do this at all.

this was a trick question, because yatta is the only program that can do this



TL;DR: the only people who would prefer encoding on lunix over encoding on windows are the "worse is better as long as it's GPL" kind of people.

(note: I actually use linux on a daily basis myself, I even used it as my desktop environment at work for a while)

[palmje]

Originally Posted by TheFluff

anything that involves cutting and splicing is a gigantic pain in the ass with mencoder

I've done cutting with mencoder before and I didn't find it that difficult - just need to set up a simple text with a list of the start/stop for each bit you want cut out. I did find that you do really need to use the hr mode on it though as it gives a better result and should be frame accurate as well since it decodes every frame rather than seeking.

Originally Posted by TheFluff

whoop de doo you have hqdn3d, smartblur and uspp for denoising, unsharp for sharpening and yadif for deinterlacing
sure is some quality filtering there, bro

I mean, hqdn3d was generally abandoned because it was only really useful for realtime denoising (there were much better alternatives if you could take a bit of slowness), nobody has actually used unsharp for anything serious in the last five years and while yadif is good at what it's designed for (realtime deinterlacing on playback) it really doesn't match up to something like nnedi, much less mcbob.

I have no idea how good/bad mencoder's ivtc filters are in general, but since they cannot handle vfr/hybrid material they are all completely useless for a lot of anime.

Well, the smartblur and uspp have worked well enough for the denoising for what I've dealt with, though I'm usually only dealing with fairly clean sources though. yadif is only one of the several deinterlacing filters in mencoder - there are several in the pp library and then there is the mcdeint one which is a motion compensating deinterlacer, which is, from what I could quickly see, what the mcbob does (though from what I saw it's incredibly slow 5-6FPS on a quad core...), though I don't know how it compares.

As for the ivtc filters, the one that I typically use is the filmdint one and from it's description: "It is designed to handle any pulldown pattern, including mixed soft and hard telecine", which should handle the hybrid material at least (not sure about the vfr stuff though - from what I know mencoder isn't great for vfr, though there are some patches for it).

Originally Posted by TheFluff

TL;DR: the only people who would prefer encoding on lunix over encoding on windows are the "worse is better as long as it's GPL" kind of people.

(note: I actually use linux on a daily basis myself, I even used it as my desktop environment at work for a while)

I prefer linux for my encoding simply because that's the OS I prefer and the tools available do what I need, I understand that it can't do alot of what AVS does, but it offers enough for the simple tasks.

[TheFluff]

Originally Posted by palmje

Well, the smartblur and uspp have worked well enough for the denoising for what I've dealt with, though I'm usually only dealing with fairly clean sources though. yadif is only one of the several deinterlacing filters in mencoder - there are several in the pp library and then there is the mcdeint one which is a motion compensating deinterlacer, which is, from what I could quickly see, what the mcbob does (though from what I saw it's incredibly slow 5-6FPS on a quad core...), though I don't know how it compares.

it really doesn't compare at all

the only reason you think smartblur and uspp "work well enough" is because you've never actually tried a proper filter solution

Originally Posted by palmje

As for the ivtc filters, the one that I typically use is the filmdint one and from it's description: "It is designed to handle any pulldown pattern, including mixed soft and hard telecine", which should handle the hybrid material at least (not sure about the vfr stuff though - from what I know mencoder isn't great for vfr, though there are some patches for it).

a mix of hard and soft telecine is not the type of hybrid material I was speaking of, try again

[palmje]

Originally Posted by TheFluff

it really doesn't compare at all

Even so, you have to make the trade off between the much longer encode time and the improvement that you will get - sometimes it might be worth it and others it won't.

Originally Posted by TheFluff

the only reason you think smartblur and uspp "work well enough" is because you've never actually tried a proper filter solution

The term "work well enough" is a subjective one - I'm well aware that there are better filters available.

Originally Posted by TheFluff

a mix of hard and soft telecine is not the type of hybrid material I was speaking of, try again

In that case it might not be able to, but there's no reason why you, or someone else, couldn't get your favourite ivtc algorithm and add it into mencoder =D

[D404]

Originally Posted by palmje

In that case it might not be able to, but there's no reason why you, or someone else, couldn't get your favourite ivtc algorithm and add it into mencoder =D

I wish linux users wouldn't always say "Why don't you add feature X yourself, the sources is available!". You could say that about practically anything.

P.S. My job is linux based, and I <3 it etc etc, but it's pretty crappy as a desktop OS still (and for encoding anything complex.. srsly).

[fireshark]

Originally Posted by palmje

In that case it might not be able to, but there's no reason why you, or someone else, couldn't get your favourite ivtc algorithm and add it into mencoder =D

The only question being, why would TheFluff do that when he already has the tools to do it?

[TheFluff]

Originally Posted by palmje

In that case it might not be able to, but there's no reason why you, or someone else, couldn't get your favourite ivtc algorithm and add it into mencoder =D

yes, there is such a reason, namely that mencoder is not useful as a Real Encoding tool and will never be so unless someone redesigns it completely and rewrites it from scratch, and seriously fuck that

[palmje]

Originally Posted by D404

I wish linux users wouldn't always say "Why don't you add feature X yourself, the sources is available!". You could say that about practically anything.

True, but at least the option is there for most linux apps if you want it. What it comes down to though is if people are happy with the current situation and, if not, have the motivation to do it.

Originally Posted by D404

P.S. My job is linux based, and I <3 it etc etc, but it's pretty crappy as a desktop OS still (and for encoding anything complex.. srsly).

I've been using Linux as my primary OS for many years now and personally I've found it to be much better for the job than Windows ever was (or is likely to be) and the only reason I ever use windows is for games, though there are a few areas where the Linux apps aren't as good (e.g. Open Office vs MS Office (though you can do this in wine), gimp vs PS (gimp is good enough for most things, main downside if lack of cymk & doesn't have the same range of filters, though there are many), encoding tools such as mencoder, transcode, etc. vs avs, adobe, etc.) and the tide goes the other way as well (openssh server vs ??? and then there's the horde of great linux apps that are ported to windows: Apache, Vim, Deluge, mysql, etc.)

In the end though, it's what each person prefers, which is why I'm a fan of cross platform programs.

Originally Posted by fireshark

The only question being, why would TheFluff do that when he already has the tools to do it?

If he's happy with the current situation then he has no motivation to do any such thing, but if he likes cross platform apps like I do, then that might be motivation enough to do something like that, or, perhaps, porting avs to linux and/or other OSes.

Originally Posted by TheFluff

yes, there is such a reason, namely that mencoder is not useful as a Real Encoding tool and will never be so unless someone redesigns it completely and rewrites it from scratch, and seriously fuck that

I can see why a redesign would be needed to make it good for non-linear encoding, but I don't see any reason why it couldn't be a good linear encoder. A simple plugin system for filters alone would go a long way since that would make it much easier for people to add in filters.

[TheFluff]

have you ever looked at mencoder.c