Decrypting Transport streams from Japanese DTV

[Tofusensei]

Originally Posted by DryFire

I know runpsi can cap CC. Might want to ask her.

It's easy to do with an analog capture, though. Do you know if she is doing it from MPEG-2 HD streams?

Anyway, to the original poster, please keep us updated with any progress you get. ^^

-Tofu

[Quarkboy]

Originally Posted by Tofusensei

It's easy to do with an analog capture, though. Do you know if she is doing it from MPEG-2 HD streams?

Anyway, to the original poster, please keep us updated with any progress you get. ^^

-Tofu

Not much... I'm pretty cure the encryption used is toshiba regza specific... It's not looking similar to the B-CAS system on a hard drive or anything.

The good news is after running share for 5 days, I got all the raws I needed anyway.

On the other hand, I have an idea to do easy transport stream capture: Buy a HD blu-ray recorder (they've come down to around $500)... They record the mpeg streams directly to internal hard drive, but you can also copy to BD-R's... Then it's just a matter of extracting mpeg transport streams from Blu-ray discs you've burnt yourself, which I believe should be relatively simple...

[SeijiSensei]

Originally Posted by Quarkboy

Not much... I'm pretty cure the encryption used is toshiba regza specific....

Hmm, is that a Freudian slip there, Quarkboy

[Quarkboy]

Originally Posted by SeijiSensei

Hmm, is that a Freudian slip there, Quarkboy

Nothing freudian... Just muscle memory...

[Eeknay]

Don't the recorders transcode to AVCHD on the fly though? Thus not preserving the original stream.

EDIT: Hm, guess they don't.

Son of EDIT: Or I guess they do. Must depend on the model.

[Quarkboy]

Originally Posted by Eeknay

Don't the recorders transcode to AVCHD on the fly though? Thus not preserving the original stream.

EDIT: Hm, guess they don't.

Son of EDIT: Or I guess they do. Must depend on the model.

Only the fancier ones can do AVC encoding on the fly, and even then it's pretty much always an option like, Extended Play.
At least judging by the printed times, it says a standard BD-R 1 layer can hold 2.5 hours of HD tv broadcasts. So doing the math, that 150 minutes being 25 GB, which works out to be around 22 Mb/s... The max over the air bitrate is 24 Mb/s so I can only assume that the player simply records the transport streams directly.

[NicestBoat]

Quarkboy,
There are quite a few ways of capturing and decrypting TS streams. Depending on your knowledge and time you have to mess with it, you may be able to do this at a fairly low cost ( ~15-30$ ) or more expensive if you do not have the needed knowledge or time ( some companies sell devices ~270$ for the whole TS capture/decryption package ). This is somewhat well documented, but It's a bit too much to discuss on a public forum, so I'll PM you where you can contact me on IRC to discuss about it. As for your current capture device which captures to encrypted streams, I'm not aware of its internal workings, but it would be possible to find out ( is there any software which you can use to play it on a pc? if not, possiblities of dumping the flash? software updates? what processor the code is running on? all of this can be reverse engineered and tools can be made, but i don't know the exact amount of work that this could take without actually knowing more about your system ) .

[Eeknay]

In answer to the closed captions question; capper pointed me towards this, which I just tested with a TS from NHK-E, and it works perfectly. The bare basic setup will just output to a TXT with no spacing or timestamps or anything, but after fiddling for five minutes you can get it to output to HTML with colours for differnet lines/sound effects, proper spacing, etc, probably a multitude of other options I haven't discovered yet.

So that's one problem solved

[Tofusensei]

Eeknay, you should really set up a website putting up captions for all the series you can! It'd be sooo helpful for fansubbers everywhere. *_*

-Tofu

[Quarkboy]

Originally Posted by Tofusensei

Eeknay, you should really set up a website putting up captions for all the series you can! It'd be sooo helpful for fansubbers everywhere. *_*

-Tofu

Unfortunately the number of series airing with captions is extremely few.
I know that one piece, pretty cure, sugo chara, and probably any other sunday or saturday morning anime gets captions.

Gundam 00 and other prime time anime also has them, if I recall.

But I've never seen captions on any late night anime, EXCEPT for Noitamina shows.

[Eeknay]

Originally Posted by Tofusensei

Eeknay, you should really set up a website putting up captions for all the series you can! It'd be sooo helpful for fansubbers everywhere. *_*

-Tofu

I don't have capping abilities, unfortunatly, I just asked a capper nicely for a sample to play around with Nice idea though.

Originally Posted by Quarkboy

Unfortunately the number of series airing with captions is extremely few.
I know that one piece, pretty cure, sugo chara, and probably any other sunday or saturday morning anime gets captions.

Gundam 00 and other prime time anime also has them, if I recall.

But I've never seen captions on any late night anime, EXCEPT for Noitamina shows.

I'm too lazy to check other channels schedules at the moment, but if you look at TBS' site Clannad and Shana II are marked as having CC, and they air around 2-3am?

[Quarkboy]

Originally Posted by Eeknay

I don't have capping abilities, unfortunatly, I just asked a capper nicely for a sample to play around with Nice idea though.



I'm too lazy to check other channels schedules at the moment, but if you look at TBS' site Clannad and Shana II are marked as having CC, and they air around 2-3am?

Let me check my electronic schedule...
Yeah, it seems like those two that air on TBS have closed captions, but their airings on BS-i doesn't have them. I couldn't find any other currently airing late night show with CC other than those two and Hakaba no Kitaro. I'm sure it depends on when and where the show airs, but CC are rare on late night animes in my experience so far. (I kind of care about them because watching the shows raw with CC on is a lot easier than just audio listening... Yay for living in Japan )

[Quarkboy]

Okay, I've found a way to get raw transport streams, at least. And it might even be legal.

If I buy one of these newish blu-ray recorders that are out there with built in tuners, they record the transport stream directly without reencoding, INCLUDING the captions and other digital television features. And since it's recorded to a blu-ray, I can then just open up the disc on my PC with anydvd or maybe even just normal software (since discs I record myself wouldn't be copy protected, probably?) to get the .m2ts files. Then us tsmuxer to convert the m2ts to ts, open up in the previous linked caption extraction program, and extract away.

I realize there are probably cheaper black market devices like the frillo that would enable direct ripping, but I think a blu-ray recorder might be nice to have anyway.

Am I missing something here that would foil my plans?

[Tofusensei]

I know the last 3 shows I translated had the "Closed Captioned" sign in the upper right corner, for what it's worth. I think it's fairly common.

-Tofu

[Eeknay]

If it's going to BD, then it's probably going to have to convert the subs to BDN/SUP format... so CCats will probably have no idea what to make of them. You'll have to use SUPread and OCR the subs, which will slow things down a lot.

BD supports support text and image subs though, but either way it's another step in the process.

[Quarkboy]

Originally Posted by Eeknay

If it's going to BD, then it's probably going to have to convert the subs to BDN/SUP format... so CCats will probably have no idea what to make of them. You'll have to use SUPread and OCR the subs, which will slow things down a lot.

BD supports support text and image subs though, but either way it's another step in the process.

Does it really have to? Blu-ray spec is m2ts files, which are slight modication to .ts files, right? So although the captions wouldn't play on other standard blu-ray players, they'd play back on the device itself (since it has all the caption rendering built in anyway).
I saw a blog entry about an HD-DVD recorder that played back the captions perfectly as they originally are, so I assumed the blu-rays just store remuxed ts files.

[Eeknay]

Honestly, I don't know. I just took a wild stab at why it "might" not work. I guess you'd have to bug someone who has one and steal a disc from them to find out exactly what's going on.

[NicestBoat]

Originally Posted by Quarkboy

Okay, I've found a way to get raw transport streams, at least. And it might even be legal.

If I buy one of these newish blu-ray recorders that are out there with built in tuners, they record the transport stream directly without reencoding, INCLUDING the captions and other digital television features. And since it's recorded to a blu-ray, I can then just open up the disc on my PC with anydvd or maybe even just normal software (since discs I record myself wouldn't be copy protected, probably?) to get the .m2ts files. Then us tsmuxer to convert the m2ts to ts, open up in the previous linked caption extraction program, and extract away.

I realize there are probably cheaper black market devices like the frillo that would enable direct ripping, but I think a blu-ray recorder might be nice to have anyway.

Am I missing something here that would foil my plans?

I don't know if the disks themself would be copy protected, however, if it just records raw transport streams, those are encrypted with MULTI2 ( almost all japanese broadcasters have mandatory encryption with this cipher), and you need your B-CAS card to decrypt to obtain the keys needed for decryption, if that is not the case, i doubt they actually store the streams not encrypted or not locked to your specific drive, there is no way to find out except to look at the specs of the bd recorder or buy one yourself. As for Friio, that device is nothing more than a tuner with usb2.0 output ( outputs raw m2ts streams, however they are encrypted) and a usb ISO 7816 smartcard reader, put in one box. Now their drivers and middleware decrypt the raw stream by using the smartcard reader and your valid B-CAS card. Of course one can do such a setup without buying a friio( which costs ~275$), if they have a way to get the raw streams and a smartcard reader(~15-30$), then just code or obtain the needed software for decryption(the standards for the B-CAS and MULTI2 are open, so coding such a tool does not involve any reverse engineering).

[Quarkboy]

Originally Posted by NicestBoat

I don't know if the disks themself would be copy protected, however, if it just records raw transport streams, those are encrypted with MULTI2 ( almost all japanese broadcasters have mandatory encryption with this cipher), and you need your B-CAS card to decrypt to obtain the keys needed for decryption, if that is not the case, i doubt they actually store the streams not encrypted or not locked to your specific drive, there is no way to find out except to look at the specs of the bd recorder or buy one yourself. As for Friio, that device is nothing more than a tuner with usb2.0 output ( outputs raw m2ts streams, however they are encrypted) and a usb ISO 7816 smartcard reader, put in one box. Now their drivers and middleware decrypt the raw stream by using the smartcard reader and your valid B-CAS card. Of course one can do such a setup without buying a friio( which costs ~275$), if they have a way to get the raw streams and a smartcard reader(~15-30$), then just code or obtain the needed software for decryption(the standards for the B-CAS and MULTI2 are open, so coding such a tool does not involve any reverse engineering).

According to about 3 different guys I asked in Yodobashi camera stores, the discs would be playable on Playstation 3s. That seems to imply that they would be stored un-encrypted.

[Eeknay]

Now that you mention it, I've heard the same thing. Someone recorded Star Wars from wowow onto BD, chucked it in his PS3, plays perfectly. Which implies that it "only" has AACS (and that's a sub based channel).