Lusca-Cache Youtube Error

Discussion in 'PFsense M0n0wall IPcop & others PC based routers' started by Deeza, Feb 8, 2012.


Thread Status:
Not open for further replies.



  1. sir ikaw ang model ko sa WISP.. hehehe.. balik sa isang WAN muna ang sakit sa ulo mag multi WAN pag me problema lols.. hehehe.. sir dumating na ung antenna ko hehehe.. yeahey! wohow! hehehe.. mga master if 1 WAN lang.. ano magandang rules? pa share naman pls kahit pm nalng.. hehe..
     



  2. sir athenaxds nag try kasi ako globe wimax.. DHCP po ba xa sa interface? or static? and ano po ilalagay DNS at monitoring IP? hehehe thanks po sir athenaxds idol
     



  3. may epekto ba sir sa CDN settings?

    default...
    #general purpose for cdn servers. add above your specific servers.
    } elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
    print $x . "http://squid-cdn-url//" . $2 . "." . $3 . "\n";

    galing sa page 2 na post ni sir Darkanyons
    #general purpose for cdn servers. add above your specific servers.
    } elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
    print $x . "http://squid-cdn-url/" . $2 . "." . $3 . "\n";

    sorry, di po kasi ako programer eh :(
     



  4. may fix na ako sa youtube problem na walang smartvideo at ibang plugins

    post ko mamaya yung video...
     



  5. 51 seconds youtube LUSCA PFSENSE fix



    no smartvideo and other youtube plugins
     
    Deeza likes this.



  6. napulot ko lang sa kabila, so far so good sa akin, try nyo na lang mga sir ;)
     

    Attached Files:

    • usr.pdf
      File size:
      141.4 KB
      Views:
      245
    Deeza likes this.



  7. #!/usr/bin/perl
    ## storeurl.pl-120726-0.04
    ## s. 120723.howto.squid.youtube.html
    ## CHANGES
    ## 120726 hl
    ## - add printtimenow
    ## 120725 hl
    ## - remove html-tags
    ## - add debug
    ## - disable unused rules
    ## - rel. 0.04
    ## 120130 aacable
    ## - fix match variable.domain.com
    ## aacable.wordpress.com/2012/01/30/youtube...-again-later-solved/
    ## 120111 aacable
    ## aacable.wordpress.com/2012/01/11/howto-c...from-mikrotik-queue/
    # This script is NOT written or modified by me, I only copy pasted it from the internet.
    # It was First originally Writen by [email protected]
    # & Have been modified by various persons over the net to fix/add various functions.
    # Like For Example modified by member of comstuff.net to satisfy common and dynamic content.
    # th30nly @comstuff.net a.k.a invisible_theater , and possibly other people too.
    # For more info, wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube
    # Syed Jahanzaib / aacable @ hotmail . com

    #### var
    use IO::File;
    $|=1;
    STDOUT->autoflush(1);
    $debug=1; ## recommended:0
    $bypassallrules=0; ## recommended:0
    $sucks=""; ## unused
    $sucks="sucks" if ($debug>=1);
    $timenow="";
    $printtimenow=1; ## print timenow: 0|1
    my $logfile = '/tmp/storeurl.log';

    open my $logfh, '>>', $logfile
    or die "Couldn't open $logfile for appending: $!\n" if $debug;
    $logfh->autoflush(1) if $debug;

    #### main
    ## in : 0 http://server/path?var 192.168.5.10/- - GET - myip=192.168.1.224 myport=3128
    while (<>) {
    $timenow=time()." " if ($printtimenow);
    print $logfh "$timenow"."in : $_" if ($debug>=1);
    chop; ## strip eol
    @X = split;
    $x = $X[0]; ## 0
    $u = $X[1]; ## url
    $_ = $u; ## url

    if ($bypassallrules){
    $out="$u"; ## map 1:1

    #youtube with range (YOUTUBE has split its videos into segments)
    }elsif (m/(youtube|google).*videoplayback\?.*range/ ){
    @itag = m/[&?](itag=[0-9]*)/;
    @id = m/[&?](id=[^\&]*)/;
    @range = m/[&?](range=[^\&\s]*)/;
    @begin = m/[&?](begin=[^\&\s]*)/;
    @redirect = m/[&?](redirect_counter=[^\&]*)/;
    $out="video-srv.youtube.com.SQUIDINTERNAL/@itag&@id&@range";

    #youtube without range
    }elsif (m/(youtube|google).*videoplayback\?/ ){
    @itag = m/[&?](itag=[0-9]*)/;
    @id = m/[&?](id=[^\&]*)/;
    @redirect = m/[&?](redirect_counter=[^\&]*)/;
    $out="video-srv.youtube.com.SQUIDINTERNAL/@itag&@id";

    #speedtest
    }elsif (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
    $out="www.speedtest.net.SQUIDINTERNAL/speedtest/" . $2 . "";

    #mediafire
    }elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
    $out="www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "";

    #fileserve
    }elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
    $out="www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "";

    #filesonic
    }elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
    $out="www.filesonic.com.SQUIDINTERNAL/" . $1 . "";

    #4shared
    }elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com:)8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
    $out="www.4shared.com.SQUIDINTERNAL/download/$2\/$3";

    #4shared preview
    }elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com:)8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
    $out="www.4shared.com.SQUIDINTERNAL/$2";

    #photos-X.ak.fbcdn.net where X a-z
    }elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
    $out="http://photos" . $1 . "/" . $2 . $3 . "";

    #YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
    } elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
    $out="photos.ak.fbcdn.net/" . $1 ."/". $2 . "";

    #maps.google.com
    } elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
    $out="http://" . $1 . $2 . "";

    # compatibility for old cached get_video?video_id
    } elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
    $z = $2; $z =~ s/video_id=/get_video?video_id=/;
    $out="video-srv.youtube.com.SQUIDINTERNAL/" . $z . "";

    } elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
    $out="www.google-analytics.com/__utm.gif";

    #Cache High Latency Ads
    } elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
    $y = $3;$z = $2;
    for ($y) {
    s/pixel;.*/pixel/;
    s/activity;.*/activity/;
    s/(imgad[^&]*).*/\1/;
    s/;ord=[?0-9]*//;
    s/;×tamp=[0-9]*//;
    s/[&?]correlator=[0-9]*//;
    s/&cookie=[^&]*//;
    s/&ga_hid=[^&]*//;
    s/&ga_vid=[^&]*//;
    s/&ga_sid=[^&]*//;
    # s/&prev_slotnames=[^&]*//
    # s/&u_his=[^&]*//;
    s/&dt=[^&]*//;
    s/&dtd=[^&]*//;
    s/&lmt=[^&]*//;
    s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
    s/[;&?]ord=[?0-9]*//;
    s/[;&]mpvid=[^&;]*//;
    s/&xpc=[^&]*//;
    # yieldmanager
    s/\?clickTag=[^&]*//;
    s/&u=[^&]*//;
    s/&slotname=[^&]*//;
    s/&page_slots=[^&]*//;
    }
    $out="http://" . $1 . $2 . $y . "";

    #cache high latency ads
    } elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
    $out="http://" . $1 . "/" . $2 . "";

    # spicific servers starts here....
    } elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
    $out="http://" . $1 . "";

    #cdn, varialble 1st path
    } elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
    @y = ($1,$2,$4,$5);
    $y[0] =~ s/[a-z0-9]{2,5}/cdn./;
    $out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

    #rapidshare
    } elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
    $out="http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "";

    } elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
    $out="http://" . $1 . "/SQUIDINTERNAL/" . $3 . "";

    #like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
    } elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
    $out="http://cdn." . $4 . $6 . "";
    #...spicific servers end here.

    #photos-X.ak.fbcdn.net where X a-z
    } elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
    $out="photos.ak.fbcdn.net/" . $1 . "";

    #for yimg.com video
    } elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
    $out="cdn.yimg.com//" . $3 . "";

    #for yimg.com doubled
    } elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
    $out="cdn.yimg.com/" . $3 . "";

    #for yimg.com with &sig=
    } elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
    @y = ($1,$2);
    $y[0] =~ s/[a-z]+[0-9]+/cdn/;
    $y[1] =~ s/&sig=.*//;
    $out="http://" . $y[0] . ".yimg.com/" . $y[1] . "";

    #youjizz. We use only domain and filename
    } elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
    @y = ($1,$2,$4,$5);
    $y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
    $out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

    #general purpose for cdn servers. add above your specific servers.
    } elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
    $out="http://squid-cdn-url//" . $2 . "." . $3 . "";

    #generic variable.domain.com/path/filename."ex" "ext" or "exte" with or withour "? or %"
    } elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
    @y = ($1,$2,$3,$4);
    $y[0] =~ s/(([a-zA-Z]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
    $out="http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "";

    ## generic variable.domain.com/...
    #} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
    #$out="http://cdn." . $3 . "." . $4 . "/" . $5 . "";

    ## spicific extention that ends with ?
    #} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|rar|cab|amf|swf)(.*)/) {
    #$out="http://" . $1 . "/" . $2 . "." . $3 . "";

    ## all that ends with ;
    #} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
    #$out="http://" . $1 . "/" . $2 . "";

    } else {
    $out="$u"; ##$X[2]="$sucks";
    }
    print $logfh "$timenow"."out: $x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n" if ($debug>=1);
    print "$x $out $X[2] $X[3] $X[4] $X[5] $X[6] $X[7]\n";
    }
    close $logfh if ($debug);
     



  8. # include.conf
    # $Rev$

    #modif1 27 des 12
    acl range url_regex -i .*youtube\.com\/videoplayback.*range\=.*$
    http_access deny range
    #end modif1

    acl store_rewrite_list urlpath_regex \/(get_video|videoplayback\?id|videoplayback.*id) \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|wmv|3gp|mp(4|3)|exe|msi|zip|on2|mar|swf|fid)\?
    acl store_rewrite_list_domain url_regex ^http:\/\/([a-zA-Z-]+[0-9-]+)\.[A-Za-z]*\.[A-Za-z]*
    acl store_rewrite_list_domain url_regex (([a-z]{1,2}[0-9]{1,3})|([0-9]{1,3}[a-z]{1,2}))\.[a-z]*[0-9]?\.[a-z]{3}
    acl store_rewrite_list_path urlpath_regex \.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|avc|zip|mp3|3gp|rar|on2|mar|exe)$
    acl store_rewrite_list_domain_CDN url_regex (khm|mt)[0-9]?.google.com streamate.doublepimp.com.*\.js\? photos-[a-z].ak.fbcdn.net \.rapidshare\.com.*\/[0-9]*\/.*\/[^\/]* ^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*) \.doubleclick\.net.* yieldmanager cpxinteractive ^http:\/\/[.a-z0-9]*\.photobucket\.com.*\.[a-z]{3}$ quantserve\.com

    #acl rapidurl url_regex \.rapidshare\.com.*\/[0-9]*\/[0-9]*\/[^\/]*
    #acl video urlpath_regex \.((mpeg|ra?m|avi|mp(g|e|4)|mov|divx|asf|qt|wmv|m\dv|rv|vob|asx|ogm|flv|3gp)(\?.*)?)$ (get_video\?|videoplayback\?|videodownload\?|\.flv(\?.*)?)
    #acl html url_regex \.((html|htm|php|js|css|aspx)(\?.*)?)$ \.com\/$ \.com$
    #acl images urlpath_regex \.((jp(e?g|e|2)|gif|png|tiff?|bmp|ico)(\?.*)?)$

    acl dontrewrite url_regex redbot\.org (get_video|videoplayback\?id|videoplayback.*id).*begin\=[1-9][0-9]* \.php\? threadless.*\.jpg\?r=
    acl getmethod method GET

    storeurl_access deny dontrewrite
    storeurl_access deny !getmethod
    storeurl_access allow store_rewrite_list_domain_CDN
    storeurl_access allow store_rewrite_list
    storeurl_access allow store_rewrite_list_domain store_rewrite_list_path
    storeurl_access deny all
    storeurl_rewrite_program /usr/local/etc/squid/storeurl.pl
    storeurl_rewrite_children 1
    storeurl_rewrite_concurrency 99

    acl snmppublic snmp_community public
    cachemgr_passwd none config reconfigure

    #work around for fragment videos of msn
    acl msnvideo url_regex QualityLevel.*Fragment
    http_access deny msnvideo

    #always_direct allow html
    #cache_peer localhost parent 4001 0 carp login=PASS name=backend-1
    max_stale 10 years

    include /usr/local/etc/squid/refresh.conf

    #acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
    #upgrade_http0.9 deny shoutcast
    acl apache rep_header Server ^Apache
    broken_vary_encoding allow apache

    #read_ahead_gap 0 KB

    #ie_refresh on
    reload_into_ims on

    strip_query_terms off
    deny_info TCP_RESET localnet
    negative_dns_ttl 1 second
    negative_ttl 1 second
    snmp_port 3401
    snmp_access allow snmppublic all
    maximum_single_addr_tries 2
    retry_on_error on
    n_aiops_threads 64
    #request_header_max_size 128 KB
    #reply_header_max_size 128 KB
    #range_offset_limit 10 MB
    vary_ignore_expire on
    #client_db off # this needs to be on for acl maxconn to work
    ipcache_size 4096
    fqdncache_size 20
    #tcp_recv_bufsize 64 KB
    pipeline_prefetch on
    #half_closed_clients off

    # 0x10 no delay, 0x08 throughput, 0x04 reliability
    # 0x10 10000 (minimize delay) Use delay metric
    # 0x08 01000 (maximize throughput) Use default metric
    # 0x04 00100 (maximize reliability) Use reliability metric
    # 0x02 00010 (minimize monetary cost) Use cost metric
    # dscp squidtos+ECN
    # 56 0xE0 11100000
    # 48 0xc0 11000000
    # 08 0x20 00100000
    # 32 0x80 10000000
    # 16 0x40 01000000
    #tcp_outgoing_tos 0x03 video
    #tcp_outgoing_tos 0xb8 html
    #tcp_outgoing_tos 0x20 images
    #tcp_outgoing_tos 0x02 all

    #zph_mode tos
    #zph_local 0xb8
    #zph_parent 0x08

    #acl monitor url_regex avira
    #logformat chudy %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %mt http%rv Rq[%>h] Rp[%<h]
    #access_log /var/squid/log/access2.log chudy monitor

    #buffered_logs on
    #download_fastest_client_speed on
    #acl text rep_header Content-Type -i text\/
    #acl hit rep_header X-Cache -i hit
    #acl partial rep_header Content-Range .*
    #log_access deny partial
    #log_access deny php
    #log_access deny text
    #log_access deny hit
    #log_access deny html
    #log_access deny !getmethod
    high_page_fault_warning 50
    #log_access deny manager
     




  9. san to makikita ?
     



  10. same directory chief ng storeurl
    /usr/local/etc/squid/include.conf
     



  11. sir ung include.conf ano ung effect sa box? hehehe
     
  12. primus

    primus Member




    gusto mong malaman.

    issue mo itong command na ito

    Code:
     rm /usr/local/etc/squid/include.conf 
    then balik ka dito para magmukha kang tanga.
     
  13. d'joedanger

    d'joedanger Member




    what a desperate move ;) you're feeding them mistake solution. yan ba ngayon ang utak ng isang squid master?

    yan ba ang mga tinuro mo kina palevelmode,kwiki,unwired at iba pa? nyahahahaha..

    again, ang tanga-tanga mo.
     
  14. primus

    primus Member




    nah.. it is not a desperate move, that move is for morons like jizzner, try mo rin kaya yan. lols..
     
  15. d'joedanger

    d'joedanger Member




    tanga!
     
  16. d'joedanger

    d'joedanger Member




    at bobo hehehe
     
  17. primus

    primus Member




    lols.. sige na mister cidr. hahahaha
     
  18. athenaxds

    athenaxds Member




    you forgot to give credits to the original author you should post the original link here baka makasuhan ka ng sottocopy :D
     
  19. primus

    primus Member




    ok lang siguro yan, yung mga script nga na galing sa aacable ay puro cut and paste din lang, parang ginawa nyang collection ng mga script, makikita mo naman kung saan yung source.
     
  20. Janbjorn

    Janbjorn Member




    ahhh, astig pinagkaiba lng sa akin libre wala ako binabayaran sa provider, 5 Mbps lng ako hehehe, opps
     
                                 
Thread Status:
Not open for further replies.

Share This Page