PFSense Adding Second Harddirve for /Cache directory

Discussion in 'PFsense M0n0wall IPcop & others PC based routers' started by Deeza, May 24, 2011.


  1. primus

    primus Member




    again, I am not a PFsense user, my squid is a dedicated box. if you want it, just hope that PFsense has an MRTG package otherwise, you are out-of-luck again.
     
  2. primus

    primus Member




    this could be your lucky day! I found mrtg for pfsense

    http://samovojski.com/blog/2011/01/15/install-mrtg-on-pfsense-2-0/
     
    Darkanyons likes this.
  3. athenaxds

    athenaxds Member




    Code:
    Start Time:    Tue, 16 Oct 2012 03:11:58 GMT
    Current Time:    Tue, 16 Oct 2012 13:26:50 GMT
     
    Connection information for squid:
        Number of clients accessing cache:    16
        Number of HTTP requests received:    102792
        Number of ICP messages received:    0
        Number of ICP messages sent:    0
        Number of queued ICP replies:    0
        Request failure ratio:    0.00
        Average HTTP requests per minute since start:    167.2
        Average ICP messages per minute since start:    0.0
        Select loop called: 6121099 times, 6.027 ms avg
    Cache information for squid:
        Request Hit Ratios:    5min: 0.2%, 60min: 1.1%
        Byte Hit Ratios:    5min: 6.2%, 60min: 29.1%
        Request Memory Hit Ratios:    5min: 0.0%, 60min: 0.0%
        Request Disk Hit Ratios:    5min: 0.0%, 60min: 50.0%
        Storage Swap size:    14591864 KB
        Storage Mem size:    2344 KB
        Mean Object Size:    1875.56 KB
        Requests given to unlinkd:    0
    Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):  0.76407  1.31166
        Cache Misses:          0.76407  1.31166
        Cache Hits:            0.00000  2.25116
        Near Hits:            0.00000  0.58309
        Not-Modified Replies:  0.00000  0.00463
        DNS Lookups:          0.00904  0.03223
        ICP Queries:          0.00000  0.00000
     
  4. Darkanyons

    Darkanyons Member




    There's something wrong with your build bro. your Request Hit Ratios & Request Memory Hit Ratios has very small or zero values, perhaps increasing memory allocation for cache can remedy this. The cache hits on Median Times is quite high. You may also check those out on how to improve your stats.
     
  5. Darkanyons

    Darkanyons Member




    there's my stats at this hour:
    Code:
    Connection information for squid:
        Number of clients accessing cache:    161
        Number of HTTP requests received:    3961079
        Number of ICP messages received:    0
        Number of ICP messages sent:    0
        Number of queued ICP replies:    0
        Request failure ratio:    0.00
        Average HTTP requests per minute since start:    2031.9
        Average ICP messages per minute since start:    0.0
        Select loop called: 134057144 times, 1.718 ms avg
    Cache information for squid:
        Request Hit Ratios:    5min: 21.4%, 60min: 26.0%
        Byte Hit Ratios:    5min: 9.9%, 60min: 23.4%
        Request Memory Hit Ratios:    5min: 25.8%, 60min: 26.9%
        Request Disk Hit Ratios:    5min: 62.9%, 60min: 50.8%
        Storage Swap size:    50568834 KB
        Storage Mem size:    524272 KB
        Mean Object Size:    44.38 KB
        Requests given to unlinkd:    0
    Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):  0.35832  0.28853
        Cache Misses:          0.55240  0.49576
        Cache Hits:            0.00091  0.00179
        Near Hits:            0.20843  0.15888
        Not-Modified Replies:  0.00091  0.00000
        DNS Lookups:          0.00000  0.00000
        ICP Queries:          0.00000  0.00000
    Resource usage for squid:
        UP Time:    230321.402 seconds
        CPU Time:    14147.719 seconds
        CPU Usage:    6.14%
        CPU Usage, 5 minute avg:    2.16%
        CPU Usage, 60 minute avg:    3.49%
        Process Data Segment Size via sbrk(): 0 KB
        Maximum Resident Size: 1381424 KB
        Page faults with physical i/o: 5
    Memory accounted for:
        Total accounted:      713449 KB
        memPoolAlloc calls: 1368587421
        memPoolFree calls: 1363426428
    File descriptor usage for squid:
        Maximum number of file descriptors:  11095
        Largest file desc currently in use:    505
        Number of file desc currently in use:  194
        Files queued for open:                  0
        Available number of file descriptors: 10901
        Reserved number of file descriptors:  100
        Store Disk files open:                  1
        IO loop method:                    kqueue
    Internal Data Structures:
        1140912 StoreEntries
        116453 StoreEntries with MemObjects
        116444 Hot Object Cache Items
        1139359 on-disk objects
     
  6. incubus14

    incubus14 Member




    ito pong sa akin sir darkanyons, ano po masasabi nyo?

    thanks!

    Code:
    Connection information for squid:
        Number of clients accessing cache:    9
        Number of HTTP requests received:    171334
        Number of ICP messages received:    0
        Number of ICP messages sent:    0
        Number of queued ICP replies:    0
        Request failure ratio:    0.00
        Average HTTP requests per minute since start:    298.6
        Average ICP messages per minute since start:    0.0
        Select loop called: 4799205 times, 7.175 ms avg
    Cache information for squid:
        Request Hit Ratios:    5min: 47.5%, 60min: 50.3%
        Byte Hit Ratios:    5min: 38.1%, 60min: 46.5%
        Request Memory Hit Ratios:    5min: 1.2%, 60min: 14.3%
        Request Disk Hit Ratios:    5min: 34.9%, 60min: 75.0%
        Storage Swap size:    24150242 KB
        Storage Mem size:    65528 KB
        Mean Object Size:    31.85 KB
        Requests given to unlinkd:    0
    Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):  0.09219  0.10857
        Cache Misses:          0.76407  0.55240
        Cache Hits:            0.00815  0.00678
        Near Hits:            0.12106  0.15048
        Not-Modified Replies:  0.00179  0.00091
        DNS Lookups:          0.00000  0.00000
        ICP Queries:          0.00000  0.00000
    Resource usage for squid:
        UP Time:    34432.581 seconds
        CPU Time:    1582.266 seconds
        CPU Usage:    4.60%
        CPU Usage, 5 minute avg:    0.41%
        CPU Usage, 60 minute avg:    0.89%
        Process Data Segment Size via sbrk(): 0 KB
        Maximum Resident Size: 238888 KB
        Page faults with physical i/o: 4
    Memory accounted for:
        Total accounted:      131189 KB
        memPoolAlloc calls: 28957633
        memPoolFree calls: 27041387
    File descriptor usage for squid:
        Maximum number of file descriptors:  58982
        Largest file desc currently in use:    66
        Number of file desc currently in use:  27
        Files queued for open:                  0
        Available number of file descriptors: 58955
        Reserved number of file descriptors:  100
        Store Disk files open:                  0
        IO loop method:                    kqueue
    Internal Data Structures:
        758863 StoreEntries
        14738 StoreEntries with MemObjects
        14735 Hot Object Cache Items
        758195 on-disk objects
     
  7. Darkanyons

    Darkanyons Member




    You have a healthy build.
    Average HTTP requests per minute since start: 298.6
    Request Hit Ratios: 5min: 47.5%, 60min: 50.3%
    Byte Hit Ratios: 5min: 38.1%, 60min: 46.5%
    Request Memory Hit Ratios: 5min: 1.2%, 60min: 14.3%
    Request Disk Hit Ratios: 5min: 34.9%, 60min: 75.0%
    your squid is just working fine.
    and its also serving fast:
    Median Service Times (seconds) 5 min 60 min:
    HTTP Requests (All): 0.09219 0.10857
    Cache Misses: 0.76407 0.55240
    Cache Hits: 0.00815 0.00678
    Near Hits: 0.12106 0.15048

    DNS lookup is execellent. at the same time very relax cpu load.
    Nothing to worry about.
     
    incubus14 likes this.
  8. incubus14

    incubus14 Member




    thanks sir darkanyons :)
     
  9. patchacker

    patchacker Member




    Sir yung sa akin po ba ayos lang?

    Code:
    Start Time:    Tue, 16 Oct 2012 23:34:43 GMT
    Current Time:    Wed, 17 Oct 2012 09:25:54 GMT
    Connection information for squid:
        Number of clients accessing cache:    19
        Number of HTTP requests received:    113407
        Number of ICP messages received:    0
        Number of ICP messages sent:    0
        Number of queued ICP replies:    0
        Request failure ratio:    0.00
        Average HTTP requests per minute since start:    191.8
        Average ICP messages per minute since start:    0.0
        Select loop called: 11935177 times, 2.972 ms avg
    Cache information for squid:
        Request Hit Ratios:    5min: 5.2%, 60min: 4.7%
        Byte Hit Ratios:    5min: 92.1%, 60min: 74.6%
        Request Memory Hit Ratios:    5min: 0.0%, 60min: 0.0%
        Request Disk Hit Ratios:    5min: 94.0%, 60min: 97.1%
        Storage Swap size:    370980194 KB
        Storage Mem size:    168 KB
        Mean Object Size:    2149.30 KB
        Requests given to unlinkd:    0
    Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):  0.30459  0.27332
        Cache Misses:          0.28853  0.28853
        Cache Hits:            0.58309  0.16775
        Near Hits:            0.00000  2.50793
        Not-Modified Replies:  0.00307  0.00463
        DNS Lookups:          0.00000  0.00000
        ICP Queries:          0.00000  0.00000
    Resource usage for squid:
        UP Time:    35471.444 seconds
        CPU Time:    839.733 seconds
        CPU Usage:    2.37%
        CPU Usage, 5 minute avg:    2.17%
        CPU Usage, 60 minute avg:    3.63%
        Process Data Segment Size via sbrk(): 0 KB
        Maximum Resident Size: 76940 KB
        Page faults with physical i/o: 2
    Memory accounted for:
        Total accounted:        18437 KB
        memPoolAlloc calls: 26654548
        memPoolFree calls: 26306101
    File descriptor usage for squid:
        Maximum number of file descriptors:  11095
        Largest file desc currently in use:    405
        Number of file desc currently in use:  159
        Files queued for open:                  0
        Available number of file descriptors: 10936
        Reserved number of file descriptors:  100
        Store Disk files open:                  0
        IO loop method:                    kqueue
    Internal Data Structures:
        172650 StoreEntries
            45 StoreEntries with MemObjects
            41 Hot Object Cache Items
        172605 on-disk objects
     
  10. Darkanyons

    Darkanyons Member




    Your stats looks good. But this system hast just booted up, or the squid server has just run.you have very little objects loaded on memory and you have zero on mem_hit.
    Let your cache system run for a day to a week and get some reading during that time to have better understanding of your build.
     
  11. patchacker

    patchacker Member




    Sir 3 months na po ang box ko, marami na po akong na cache

    Code:
    [COLOR=#000000][SIZE=13px][FONT=courier new]$ df -h
    Filesystem     Size    Used   Avail Capacity  Mounted on
    /dev/ad4s1a    1.3T    363G    875G    29%    /
    devfs          1.0K    1.0K      0B   100%    /dev
    /dev/md0       3.6M     34K    3.3M     1%    /var/run
    devfs          1.0K    1.0K      0B   100%    /var/dhcpd/dev[/FONT][/SIZE][/COLOR]
    
     
  12. Darkanyons

    Darkanyons Member




    Sorry to confuse you. you might have this system for 3 months already thats why you have great disk_hit ratio. what im trying to say is that you stopped and restarted your proxy just recently:
    CPU Time: 839.733 seconds = ~14mins

    and it might be the reason why your mem hit ratio is very small............
    Or perhaps you have very small memory allocated for squid:
    Memory accounted for:
    Total accounted: 18437 KB

    Continue running your system for a day or a week nonstop, and have several readings during that time to have
    a better evaluation of your build.
    :)
     
  13. patchacker

    patchacker Member




    ah ok po... Salamat sir.
     
  14. athenaxds

    athenaxds Member




    Magkano ba pinaka murang SSD balak ko ilipat pfsense ko sa SSD mabagal itong hitachi deskstar
     



  15. Code:
    Connection information for squid:
        Number of clients accessing cache:    10
        Number of HTTP requests received:    91744
        Number of ICP messages received:    0
        Number of ICP messages sent:    0
        Number of queued ICP replies:    0
        Request failure ratio:    0.00
        Average HTTP requests per minute since start:    330.2
        Average ICP messages per minute since start:    0.0
        Select loop called: 4730682 times, 3.524 ms avg
    Cache information for squid:
        Request Hit Ratios:    5min: 0.2%, 60min: 0.4%
        Byte Hit Ratios:    5min: 0.1%, 60min: 10.0%
        Request Memory Hit Ratios:    5min: 0.0%, 60min: 0.0%
        Request Disk Hit Ratios:    5min: 0.0%, 60min: 63.9%
        Storage Swap size:    997132 KB
        Storage Mem size:    636 KB
        Mean Object Size:    1962.86 KB
        Requests given to unlinkd:    0
    Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):  2.13280  1.24267
        Cache Misses:          2.13280  1.31166
        Cache Hits:            0.00000  0.20843
        Near Hits:            0.00000  1.31166
        Not-Modified Replies:  0.00000  0.00000
        DNS Lookups:          0.00103  0.00094
        ICP Queries:          0.00000  0.00000
    Resource usage for squid:
        UP Time:    16671.867 seconds
        CPU Time:    934.675 seconds
        CPU Usage:    5.61%
        CPU Usage, 5 minute avg:    6.06%
        CPU Usage, 60 minute avg:    6.14%
        Process Data Segment Size via sbrk(): 0 KB
        Maximum Resident Size: 11748 KB
        Page faults with physical i/o: 671
    Memory accounted for:
        Total accounted:        3942 KB
        memPoolAlloc calls: 15655700
        memPoolFree calls: 15647625
    File descriptor usage for squid:
        Maximum number of file descriptors:  3405
        Largest file desc currently in use:    392
        Number of file desc currently in use:  337
        Files queued for open:                  0
        Available number of file descriptors: 3068
        Reserved number of file descriptors:  100
        Store Disk files open:                  1
        IO loop method:                    kqueue
    Internal Data Structures:
          689 StoreEntries
          181 StoreEntries with MemObjects
          143 Hot Object Cache Items
          508 on-disk objects
    mga idol pacheck naman kung ok itong akin.. ^_^ thank you.
     
  16. incubus14

    incubus14 Member




    @grinminded

    page faults pa lang alam na :D

    Code:
     Page faults with physical i/o: 671
     



  17. :( idol paano ba ayusin yan?
     
  18. athenaxds

    athenaxds Member




    Add more memory
     
    grinminded and incubus14 like this.



  19. 2gb lang kasi gamit ko sa virtualbox. :( thank you sir :)
     
  20. Darkanyons

    Darkanyons Member




    Dahan2x lang dito bro. SSD might have driver problems on PF or on squid/lusca itself as Primus pointed out. so better check that out before purchasing one.
     
                                 

Share This Page