CastleCops® Optimizing Performance

Need help? Click here to register for free! Absolutely zero advertisements on this site!

**$9736.22 of $21422.68**

Help CastleCops serve the community on new servers, Donate Here to reach our goal.

	Donation/Premium

	Donations. Become Premium Today!

	Security Central

	· Home · PIRT/Fried Phish · MIRT · SIRT · Deutsch · Wiki · Newsletter · O16/ActiveX · CLSID List · Contest2007 · Downloads · Feedback (send) · Forums · HijackThis · Hijacktrend · LSPs · My Downloads · O18 · O20 · O21 · O22 · O23 · O9 · Premium · Private Messages · Proxomitron · Reviews · Search · StartupList · Stories Archive · Submit News · WsIRT · Your Account · Acceptable Use Policy

Survey

Forum FAQ

Usergroups

Profile

Select FavForums

Optimizing Performance
Goto page Previous 1, 2, 3, 4, 5 ... 9, 10, 11, 12 Next

All -> FavForums -> perf

[del.icio.us!] [digg it!] [reddit!]

View previous topic :: View next topic

Author

Message

Blair

Microsoft MVP

Joined: Feb 17, 2004
Posts: 26
Location: USA

Posted: Thu Feb 21, 2008 3:54 am Post subject:

Pretty confident in your httpd.conf? KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 StartServers 5 MinSpareServers 5 MaxSpareServers 10 MaxClients 150 MaxRequestsPerChild 0 ...

tetak

MIRT Team Lead
Premium Member

Joined: Jan 19, 2007
Posts: 5801

Posted: Thu Feb 21, 2008 5:00 am Post subject:

I think you've already done this but if not try changing how often the "views" counter updates when someone opens a forum e.g. /f281-MIRT_Reports.html The top 3 posts currently have a view count of 0, I've read each topic once so it should be at least 1. The 3rd topic down was posted over an hour ago so if the current value is below an hour that might be a new bug. _________________ Got Windows XP? Help protect your PC from malware with Microsofts anti-spyware program Windows Defender. Download it for free from http://www.microsoft.com/athome/security/spyware/software/default.mspx

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 1:42 pm Post subject:

@Blair, yes I think they are having just checked. But, I didn't have that portion enabled til just now. @tetak, as of yesterday, forum views will not get updated in real time. I created a middle table yesterday and put the updates of views there. Now I just have to create a cron where those values increment those found in forum view. So the counts are being kept, just not transferred atm. _________________ Paul Laudanski - http://www.laudanski.com http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 1:45 pm Post subject:

Code:

last pid: 67171; load averages: 10.86, 16.29, 24.21
214 processes: 11 running, 202 sleeping, 1 zombie
CPU states: 82.0% user, 0.0% nice, 17.6% system, 0.4% interrupt, 0.0% idle
Mem: 1961M Active, 2671M Inact, 367M Wired, 324M Cache, 214M Buf, 2360M Free
Swap: 16G Total, 48K Used, 16G Free

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
40376 mysql 77 20 0 1247M 1243M kserel 310:10 12.79% mysqld
67102 www 1 109 0 237M 111M RUN 0:07 3.57% httpd
66997 www 1 109 0 138M 47252K RUN 0:07 3.42% httpd
67108 www 1 108 0 131M 40048K RUN 0:04 3.27% httpd
66983 www 1 108 0 238M 112M RUN 0:18 3.27% httpd

mysqld seems to have fully geared up now. Before the restart of apache moments ago the load was in the 30s. Now keepalive, et al, is enabled. Unsure what the default is if its not mentioned.

_________________
Paul Laudanski - http://www.laudanski.com
http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 2:17 pm Post subject:

Code:

# iostat -x
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 1.0 6.7 9.4 238.5 0 50.9 3
da1 6.0 17.8 303.9 687.3 0 345.6 13

SYSTEM MEMORY SUMMARY:
mem_used: 4775297024 ( 4554MB) [ 55%] Logically used memory
mem_avail: + 3814637568 ( 3637MB) [ 44%] Logically available memory

MySQL on localhost (5.1.23-rc-log)
Queries: 87.8M qps: 1364 Slow: 0.0 Se/In/Up/De(%): 92/00/00/00
qps now: 2118 Slow qps: 0.0 Threads: 23 ( 15/ 50) 99/00/00/00
Cache Hits: 80.6M Hits/s: 1251.4 Hits now: 2105.8 Ratio: 100.0% Ratio now: 100.0%
Key Efficiency: 99.4% Bps in/out: 1.7/314.5 Now in/out: 79.5/ 8.5k

53 httpd's right now.

iostat seems to show da1 is not handling the mysqld workhorse well.

Code:

# iostat -x 5
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 1.0 6.7 9.4 238.3 0 50.9 3
da1 6.0 17.8 303.6 687.4 1 345.7 13
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 1.4 0.0 18.0 0 0.1 0
da1 53.2 0.2 3020.7 6.4 0 4.4 21
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 3.0 0.0 266.3 0 0.4 0
da1 2.6 0.6 166.3 19.2 0 4.0 1
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 1.6 0.0 14.4 0 0.2 0
da1 0.4 2.2 19.2 70.4 0 2.3 1
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 11.8 0.0 395.5 0 1.0 0
da1 1.0 0.0 51.2 0.0 0 8.3 1
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 2.0 0.0 256.0 0 0.7 0
da1 1.6 0.4 76.8 25.6 0 4.0 1
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 14.2 0.0 415.1 0 1.8 0
da1 0.0 104.0 0.0 3789.9 0 35.3 2
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 1.6 0.0 14.4 0 0.2 0
da1 0.2 3.6 6.4 121.6 0 384.8 20
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 2.8 0.0 26.8 0 0.7 0
da1 5.0 0.4 217.6 12.8 0 6.3 3
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0

I don't know, perhaps here is the bottleneck.

_________________
Paul Laudanski - http://www.laudanski.com
http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 2:46 pm Post subject:

A random iostat -x 5:

Code:

extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.0 0.0 0.0 0.0 0 0.0 0
da1 0.8 138.0 25.6 5044.1 1 58.6 56
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s wait svc_t %b
da0 0.2 13.2 3.2 200.0 0 2.5 0
da1 0.6 0.0 38.4 0.0 0 1035.8 29
pass0 0.0 0.0 0.0 0.0 0 0.0 0
pass1 0.0 0.0 0.0 0.0 0 0.0 0

Not sure how to "resolve" this.

_________________
Paul Laudanski - http://www.laudanski.com
http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 2:55 pm Post subject:

Quote:

Not sure how to "resolve" this.

http://www.dbapool.com/articles/013106.html

Quote:

Analysis the results:

The following are main areas to focus in the report.

Reads/writes per second (r/s , w/s)
Percentage busy (%b)
Service time (svc_t)

If a disk shows consistently high reads/writes along with , the percentage busy (%b) of the disks is greater than 5 percent, and the average service time (svc_t) is greater than 30 milliseconds, then one of the following action needs to be taken

Modifying the disk queries and using available cache facilities of application servers to use disk i/o more efficiently.

Implement the RAID or similar solution.

Increase the system parameter values for inode cache, ufs_ninode, which is Number of inodes to be held in memory. Inodes are cached globally (for UFS), not on a per-file system basis.

Replace the existing disk/controller with a faster one or move files to another file system.

vmstat 5

Code:

procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id
20 24 1 4232468 1415100 10300 0 0 0 10391 79 0 0 3511 27183 8240 73 19 8
23 1 1 4202836 1410764 70561 0 0 0 66783 0 7 35 29824 383375 60595 78 22 0
26 5 0 4178220 1418776 6849 0 0 0 8632 0 0 118 4129 38716 9562 86 14 0
26 2 0 4167004 1454468 16355 0 0 0 16638 0 2 28 7763 196144 16082 75 25 0
21 22 1 4171544 1473176 12334 0 0 0 9076 0 43 3 3620 98709 8662 78 22 0
21 24 1 4029076 1507644 61675 0 0 0 64739 0 5 21 26668 336397 53146 77 23 0
29 2 0 4184440 1518012 118818 0 0 0 124417 0 5 7 42807 439285 87951 79 21 0

To decipher the above, see here:

http://www.adminschoice.com/docs/iostat_vmstat_netstat.htm

Again, don't know what to make of that. Sounds like we do have an issue with page sr, and procs r.

More research...

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 3:20 pm Post subject:

OK I stand corrected on the above vmstat. I juxtaposed the fr column data under sr. So the page sr data is fine. _________________ Paul Laudanski - http://www.laudanski.com http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351

Posted: Thu Feb 21, 2008 3:37 pm Post subject:

Code:

last pid: 69510; load averages: 45.92, 46.35, 42.79
276 processes: 59 running, 216 sleeping, 1 zombie
CPU states: 90.6% user, 0.7% nice, 7.5% system, 1.1% interrupt, 0.0% idle
Mem: 5201M Active, 1761M Inact, 396M Wired, 275M Cache, 214M Buf, 51M Free
Swap: 16G Total, 4K Used, 16G Free

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
40376 mysql 77 20 0 1247M 1243M kserel 333:11 2.20% mysqld
69029 www 1 104 0 111M 22604K RUN 0:12 1.86% httpd
69121 www 1 104 0 110M 21820K RUN 0:12 1.76% httpd
69380 www 1 107 0 113M 24316K RUN 0:04 1.61% httpd
69374 www 1 103 0 109M 21680K select 0:03 1.17% httpd
69171 www 1 107 0 152M 56656K RUN 0:12 0.93% httpd
69247 www 1 107 0 237M 110M RUN 0:12 0.93% httpd
69272 www 1 104 0 109M 21272K RUN 0:11 0.93% httpd
69224 www 1 107 0 109M 20716K RUN 0:09 0.88% httpd
69443 www 1 107 0 124M 36476K RUN 0:02 0.88% httpd

MySQL on localhost (5.1.23-rc-log)
Queries: 91.5M qps: 1324 Slow: 0.0 Se/In/Up/De(%): 92/00/00/00
qps now: 258 Slow qps: 0.0 Threads: 18 ( 4/ 55) 117/00/00/00
Key Efficiency: 99.4% Bps in/out: 0.2/ 25.1 Now in/out: 28.5/ 2.6k

_________________
Paul Laudanski - http://www.laudanski.com
http://www.linkedin.com/pub/1/49a/17b

Paul

CastleCops Founder

Joined: Feb 22, 2002
Posts: 27351