How to make fansubs playable on the PS3 [firmware 1.90]
 

Okay, so I recently bought a PS3 with the $100 price drop (and a hefty check from the insurance company I won't get into), and I've been blabbing about how my group's (Arienai)'s fansubs should be playable on the PS3 natively.

Turns out, they are, ALMOST. The only issue is that by default x264 sets the level of the AVC encode to 5.1, even if the settings don't need it to be that high, and the PS3 only plays things that are 4.1 or lower.

So what to do? There is a pretty dang easy solution. Note, I don't take credit for this at all, it was discovered by someone on doom9 (surprise, surprise), however this is a somewhat more lucid writeup.

Here are the steps:

I use x264, mp4box, besweet (for aac audio encoding) and virtualdub's built-in hex editor.

Encode the video to a raw .h264 file using x264 (or me-gui). Don't worry about settings at all, use whatever you see fit. Then, open up the resulting file in your hex editor and look at the very beginning of the file:

At bytes number 5-7, you should have 64 00 33 (assuming you are using high profile and level 5.1). All you need to change is the "33" to a "29", resulting in the number 5-7 bytes being 64 00 29

Then, encode the audio as AAC (LC, although I think that HE might be supported, someone try it out and tell me), mux as normal with mp4box (don't forget the -fps since you are importing raw video), and you have a completely compatible PS3 video file. Load it onto the PS3 and play away.

If you have an already muxed mp4 file, you can do the same trick, but there are _2_ places you have to change 64 00 33 to 64 00 29, both near the beginning of the file, and both very close to each other.


I'm sure someone could write a very simple utility for automating this, but frankly virtualdub's built in hex editor works fine for this purpose.


I've tried this on a number of my own group's fansubs and it works like a charm. The only problem is I don't have a HDTV so I can't play the 1280x720 ones, even though I know they are compatible (it plays them, it just doesn't output the video... it even tells me the currrent video bitrate).

So what about playing it on the 360? I believe it should also have MPEG-4 AVC support since some time ago.

And what's with the PS3 not displaying 720p videos at all on a non-HDTV? Can't it just downscale?!

Finally, there has to be some way to tell x264 what profile level to write in the bitstream... or push for one to be added.

I hate the idea of intentionally crippling encodes for dumb H/W limitations. PS3 is supposedly all powerful, so it should handle my encodes if I happened to not set the correct level or whatever (I almost always set the level anyway).

There is a blog or something which Sony seem to be taking note of, I say people should go bitch at them big time.

I also hate the idea of blindly guessing at levels or setting them incorrectly on purpose to trick hardware into playing it. I don't think we want to go down this route and complicate things more than they are compatability/spec wise.

Although I agree with you in principle, the fact is I can hardly see any reason to have a FANSUB break high profile 4.1 (in a significant way... w.r.t a previous question I had)... Of course it'd be great if this wasn't necessary and Sony just let the PS3 play whatever it wants, but until then I don't see any reason not to hex edit things when they ARE, indeed, 99% of the time, actually level 4.1 and not level 5.1

I think I'll probably go ahead and do this for all further releases of mine, just for kicks. It definitely doesn't do anything to playback on computers, in any case.

(and me-gui forces me to undo p4x4 if I choose 4.1, which I'd rather not do)

I don't own a 360, so you go experiment and tell me :).

The PS3 doesn't like to downscale, I think. It treated it like a game whose lowest resolution is 720p, so it refused to output the video signal over the 480i composite video hookup. It has nothing to do with the encoding, since the identical 640x360 version plays fine.

Sure, I think you can do it from the start on the command line when encoding (except with the one caveat of using p4x4 search, which is a very technical point that was brought up by me a while ago and frankly, shouldn't be something to worry about in reguards to the playstation 3). This was more a guide for pre-existing encodes that might work even if they weren't set right to begin with. I know that I'll make my encodes say 4.1 from now on personally.

Actually, I downloaded one of your 720p Pretty Cure encodes to test on the KiSS 1600. It played flawlessly, and by my reckoning, it was a level 5 encode (well I tried re-encoding it with the same settings and using trial and error, x264 kept giving warnings until I reached --level 5).

It's very easy for moderately complex 720p encodes to go beyond 4 or 4.1, a high complexity 720x480 encode can quite easily be level 4 anyway. The thing that tends to push the levels is the amount of reference frames; your encode was pretty much within most restrictions barring the DPB limit.

I would think that more important to the number of reference frames is the number of b-frames. Since b-frames require up to twice the number of motion vectors as p-frames, and having more of them in a row means higher calculational complexity. The number of reference frames would have more effect on total memory requirements than anything else, right? And I'm not sure how that effects the level specification.

Now I'm not fully into AVC encoding so some of this is guesswork.

If we say Nr means number of reference frames and Nb is (maximum) number of B-frames then...
Without B-pyramids, a B-frame can reference either the next I/P-frame or the Nr previous I/P-frames. (Or can it reference both Nr frames forwards and Nr backwards?) Ie. it requires memory for at least Nr+1 frames. One (Nr?) of those need to be decoded at least one frame earlier that they are displayed. (Stored order and decoded order of frames is the same. If displayed order is IBP stored order must be IPB because the P-frame is required to decode the B-frame, meaning to display the second frame, two frames must be decoded instead of decoding just one frame per frame displayed.)
With B-pyramid things get much worse. You can have I B1 B2 B3 P and say, B1 and B3 ref I and P but B2 ref I B1 B3 P. So stored order must be I P B1 B3 B2. Even more memory required and even more pre-decoding required.

I hope this post makes sense, I'm really tired atm ;)

This is a very informative topic! Quarkboy, were you able to play fansubs in your PS3 now? Hope it went well!:)

What about this tool?
http://www.bitburners.com/articles/c...-mkv2vob/4022/

Now this also sounds interesting, wonder if it works.

Odd though: HandBrake's PS3 profile doesn't use B-frames at all, but does pass level=41 to x264. For the X360 it does use B-frames, and passes level=40. It appears to me that they don't know either what exactly is supported. A slightly off-topic question: are all AVC features supported when you put an AVC stream in a (MPEG 2) transport stream to stream it to your PS3?

Useful and related links regarding how many b-frames/refs you can use if you want hardware compatibility:
http://forum.doom9.org/showthread.php?t=132924
http://www.avsforum.com/avs-vb/showthread.php?t=972503
http://forum.doom9.org/showthread.php?t=140223

Also useful if you (like me) want to figure out how to make your encodes unplayable in hardware players and undecodable with DXVA. :]