AnimeSuki Forums

Register Forum Rules FAQ Members List Social Groups Search Today's Posts Mark Forums Read

Go Back   AnimeSuki Forum > Anime Related Topics > General Anime > Fansub Groups

Notices

Reply
 
Thread Tools
Old 2007-11-29, 16:25   Link #21
IRJustman
Founder, Sprocket Hole
*Fansubber
 
 
Join Date: Apr 2004
Location: Fresno or Sacramento, CA
Age: 46
Quote:
Originally Posted by TheFluff View Post
perl > shell
Amen to that!

Quote:
Really dumb, probably very buggy and requires cksfv (apt-get install cksfv, what are you waiting for?) since AFAIK Digest::CRC32 is written in pure Perl and therefore slow. But it does what you asked for.
I use the crc32 program that comes with the package "libarchive-zip-perl" (also a Debian, though in my case, Ubuntu) pacakge, it it's not TOO bad. In fact, it runs really snappy on my Duron 1600 fileserver.

I'll have to muck with it. If there are any issues, I'll just grok the Camel, perhaps give you some diffs of what I've done.

Quote:
Rewriting the program to use fsum on Windows is left as an exercise to the reader.
Pass. ;)

--Ian.
IRJustman is offline   Reply With Quote
Old 2007-11-29, 18:55   Link #22
Sylf
翻訳家わなびぃ
*Fansubber
 
 
Join Date: Nov 2003
Age: 40
Send a message via MSN to Sylf Send a message via Yahoo to Sylf
I used to use perl script too. I became impatient about its speed.

I stole crc32() code off of somewhere, and made this. Compile it into crc32, and put it in your PATH.

Code:
/* crc32.c - the crc check algorithm for cksfv
   
   Copyright (C) 2000 Bryan Call <bc@fodder.org>

   This program is free software; you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
   the Free Software Foundation; either version 2 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program; if not, write to the Free Software
   Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. */
   
// http://www.fodder.org/blog/cksfv.php

#include <sys/types.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <limits.h>
#include <inttypes.h>

extern int errno;

#define BUFFERSIZE 16384   /* (16k) buffer size for reading from the file */

static const uint32_t crctable[256] = {
  0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
  0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
  0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
  0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
  0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
  0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
  0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
  0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
  0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
  0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
  0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
  0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
  0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
  0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
  0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
  0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
  0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a,
  0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
  0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818,
  0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
  0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
  0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
  0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c,
  0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
  0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
  0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
  0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
  0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
  0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086,
  0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
  0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4,
  0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
  0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
  0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
  0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
  0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
  0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe,
  0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
  0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
  0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
  0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252,
  0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
  0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60,
  0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
  0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
  0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
  0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04,
  0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
  0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a,
  0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
  0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
  0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
  0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e,
  0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
  0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
  0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
  0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
  0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
  0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0,
  0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
  0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6,
  0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
  0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
  0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d,
};


int crc32(int fd, uint32_t *main_val, int *siz)
{
  char buf[BUFFERSIZE];
  char *p;
  int nr;
  uint32_t crc = ~0;
  int totsize = 0;

  while ((nr = read(fd, buf, sizeof(buf))) != -1 && nr != 0) 
  {
    totsize += nr;
    for (p = buf; nr--; ++p)
      crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
  }
  if (nr < 0)
    return 1;

  *main_val = ~crc;
  *siz = totsize;
  return 0;
}

int main(int argc, char *argv[])
{
  FILE          *fd;
  char          buf[512], *end, filename[512], crc[9], path[256];
  int           file, rval = 0;
  unsigned long len, sfvcrc;
  uint32_t val;
    int siz;

    if (argc != 2)
    {
        printf("Please supply 1 file name.\n");
        exit(-1);
    }
    if ((file = open(argv[1], O_RDONLY, 0)) < 0)
    {
        printf("File not found - %s", argv[1]);
        exit(-1);
    }
    if (crc32(file, &val, &siz))
    {
        fprintf(stderr, "%s: %s\n", argv[1], strerror(errno));
    }
    close(file);
    printf ("CRC for %s - [%08X] (%i)\n", argv[1], val, siz);
    exit(0);
}
Sylf is offline   Reply With Quote
Old 2007-12-06, 02:01   Link #23
edogawaconan
LOL?
*Fansubber
 
 
Join Date: Aug 2006
Location: Indonesia
Send a message via MSN to edogawaconan Send a message via Yahoo to edogawaconan
Quote:
Originally Posted by TheFluff View Post
perl > shell
Code:
<freakingly long code that took ages to paste with ultra-slow-vpn>
Really dumb, probably very buggy and requires cksfv (apt-get install cksfv, what are you waiting for?) since AFAIK Digest::CRC32 is written in pure Perl and therefore slow. But it does what you asked for.

Rewriting the program to use fsum on Windows is left as an exercise to the reader.
thank you
...
and error o_O

Code:
/home/ftp/520-signum/edogawaconan/new/unsorted# /root/cek \[m.3.3.w]_Myself_Yourself_-_10_\(XviD\)_\[FC044F07].avi
sh: syntax error: `(' unexpected
Use of uninitialized value in string eq at /root/cek line 21.
Use of uninitialized value in concatenation (.) or string at /root/cek line 26.
[m.3.3.w]_Myself_Yourself_-_10_(XviD)_[FC044F07].avi:  - NOT OK! Should be FC044F07!
will debug it by myself later since I'll be having test in one hour ._.
__________________
edogawaconan is offline   Reply With Quote
Old 2007-12-06, 15:28   Link #24
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 28
Looks like you either don't have cksfv or it's not outputting stuff in the expected format.

Or it could be some interesting problem with shell escapes; try if it works with another input filename.
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Old 2007-12-07, 11:28   Link #25
edogawaconan
LOL?
*Fansubber
 
 
Join Date: Aug 2006
Location: Indonesia
Send a message via MSN to edogawaconan Send a message via Yahoo to edogawaconan
Quote:
Originally Posted by TheFluff View Post
Looks like you either don't have cksfv or it's not outputting stuff in the expected format.

Or it could be some interesting problem with shell escapes; try if it works with another input filename.
yea it worked parentheses destroyed the script
here's my fix:
replace (line 17 and 31)
Code:
$input = `cksfv $filename`;
with this
Code:
$input = `cksfv "$filename"`;
...I've never work with perl before btw - in fact, I rarely make scripts/programs

tried on OpenBSD 4.2, with tcsh as its shell

update:
replace this too: (line 32)
Code:
$checksum = parse_cksfv($input);
with
Code:
$real_checksum = parse_cksfv($input);
or else the script will fail when checking file without crc in the filename
__________________

Last edited by edogawaconan; 2007-12-07 at 18:22.
edogawaconan is offline   Reply With Quote
Old 2007-12-07, 23:06   Link #26
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 28
Quote:
Originally Posted by edogawaconan View Post
yea it worked parentheses destroyed the script
here's my fix:
replace (line 17 and 31)
Code:
$input = `cksfv $filename`;
with this
Code:
$input = `cksfv "$filename"`;
grr, it figures.
I know I should really just execute cksfv with system() instead of passing it through the shell (it'd be more secure too), but I'm too lazy to do proper piping right now. I'll fix it properly later.

Quote:
Originally Posted by edogawaconan View Post
replace this too: (line 32)
Code:
$checksum = parse_cksfv($input);
with
Code:
$real_checksum = parse_cksfv($input);
or else the script will fail when checking file without crc in the filename
Whoops, a bit too much copypasta there. Fixed but by renaming the variable in the print command instead.
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Old 2007-12-09, 11:59   Link #27
edogawaconan
LOL?
*Fansubber
 
 
Join Date: Aug 2006
Location: Indonesia
Send a message via MSN to edogawaconan Send a message via Yahoo to edogawaconan
it inspires me to learn perl!

like this, remove underscores in files (stealing your structures )
Code:
#!/usr/bin/perl

use warnings;
use strict;

my @files = @ARGV or print("Usage: ren file1 ... fileN\n") && exit(1);

foreach (@files) {
        my $newname = $_;
        print(qq(Could not find file "$_", skipping\n)) and next() unless -e $_;
        $newname =~ s/_/ /g;
        print(qq($_: $newname already exists, skipping\n)) and next() if -e $newname and $_ ne $newname;
        if ($_ eq $newname) {
                print(qq($_: no underscores, skipping\n))
        }
        else {
                rename($_, $newname);
                print(qq($_ -> $newname\n));
        }
        next();
}

exit(0);


edit: learned more skill!
__________________

Last edited by edogawaconan; 2007-12-09 at 12:38. Reason: better code
edogawaconan is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 15:08.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
We use Silk.