Posts
监控进程的IO统计信息
1. I/O accounting
Linux内核2.6.20增加了IO审计功能, 要使用这个功能需要首先在编译内核时打开TASK_DELAY_ACCT和TASK_IO_ACCOUNTING参数
RedHat Linux在内核2.6.18-144之后也加入了该功能, 并在RHEL5.4版本(kernel 2.6.18-164)中发布, 可以直接使用
一些基于IO审计的监控工具:
1)IOtop
Iotop is a Python program with a top like UI used to show of behalf of which process is the I / O going on. It requires Python ≥ 2.5 and a Linux kernel ≥ 2.6.20 with the TASK_DELAY_ACCT and TASK_IO_ACCOUNTING options enabled. It requires Python ≥ 2.5 and ≥ Linux kernel 2.6.20 with the TASK_DELAY_ACCT and TASK_IO_ACCOUNTING options enabled.
2)pidstat(SYSSTAT工具之一)
Report I/O statistics (kernels 2.6.20 and later only)
3)Dstat: Versatile resource statistics tool
0.6.7 版本后支持显示topio,topbio
4)htop - an interactive process viewer for Linux
显示RBYTES,WBYTES,IO_RATE等列
5)collectl - Process I/O Stats
2.4.0版本之后支持显示io统计信息, 见Process I/O Stats
外部链接
Red Hat backported I/O accounting to RHEL5
Red Hat Enterprise Linux 5.4 - Release Notes
2. block_dump
设置内核参数block_dump为非零值, 内核将报告所有磁盘读写操作和脏块刷新的信息
下载iodump脚本, 用于分析处理内核报告
wget http://maatkit.googlecode.com/svn/trunk/util/iodump
打开block_dump
echo 1 > /proc/sys/vm/block_dump
通过iodump分析dmesg输出的内核报告
while true; do sleep 1; dmesg -c; done | perl iodump
运行一会儿, 按Ctrl-c停止输出, 得到类似下面的结果
# while true; do sleep 1; dmesg -c; done | perl iodump # Caught SIGINT. TASK PID TOTAL READ WRITE DIRTY DEVICES kjournald 478 1294 0 1294 0 sda2 pdflush 21422 857 0 857 0 sda5, sdb6 kjournald 1752 365 0 365 0 sda5 kjournald 1758 174 0 174 0 sdb6 kjournald 1756 52 0 52 0 sdb5 syslogd 26752 28 0 28 0 sdb6 firefox 12811 27 27 0 0 sdb5 perl 3003 25 25 0 0 sda2 process_perfdat 3026 12 12 0 0 sda5 bash 14749 3 3 0 0 sda2 firefox 12654 3 3 0 0 sdb5 bash 3004 2 2 0 0 sda2 bash 3017 1 1 0 0 sda2 squid 25555 1 1 0 0 sda2 check_nrpe 3001 1 1 0 0 sda2
最后关闭block_dump
echo 0 > /proc/sys/vm/block_dump
外部链接:
block_dump
How to find per-process I/O statistics on Linux
Monitoring filesystem activity under Linux with block_dump
How can I record what process or kernel activity is using the disk in GNU/Linux?
How to identify what processes are generating IO Wait load.
3. blktrace
支持block trace的内核版本:
– Patch for Linux 2.6.14rc3
(or later, up to 2.6.17)
– Linux 2.6.17 (or later) – built in
安装
yum -y install blktrace
挂载debugfs文件系统
mount -t debugfs debugfs /sys/kernel/debug
运行blktrace
blktrace -d /dev/sda2 -o - | blkparse -i - -s
运行一段时间后, CTRL-C退出, 显示统计信息:
... ... ... 8,2 0 1576 0.618648628 478 U N [kjournald] 1 8,2 0 1577 0.618669180 0 C W 375869 + 128 [0] 8,2 0 1578 0.618763920 478 Q W 376045 + 8 [kjournald] 8,2 0 1579 0.618765468 478 G W 376045 + 8 [kjournald] 8,2 0 1580 0.618766125 478 P N [kjournald] 8,2 0 1581 0.618766440 478 I W 376045 + 8 [kjournald] 8,2 0 1582 0.618767278 478 U N [kjournald] 1 8,2 0 1583 0.618768643 478 D W 376045 + 8 [kjournald] 8,2 0 1584 0.618824993 0 C W 376045 + 8 [0] blktrace (5609) Reads Queued: 0, 0KiB Writes Queued: 0, 0KiB Read Dispatches: 0, 0KiB Write Dispatches: 2, 8KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 2, 12KiB Read Merges: 0, 0KiB Write Merges: 0, 0KiB IO unplugs: 0 Timer unplugs: 0 Allocation wait: 0 Allocation wait: 0 Dispatch wait: 0 Dispatch wait: 0 Completion wait: 0 Completion wait: 0 kjournald (478) Reads Queued: 0, 0KiB Writes Queued: 330, 1,320KiB Read Dispatches: 0, 0KiB Write Dispatches: 42, 1,320KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 4, 76KiB Read Merges: 0, 0KiB Write Merges: 288, 1,152KiB IO unplugs: 12 Timer unplugs: 0 Allocation wait: 0 Allocation wait: 0 Dispatch wait: 0 Dispatch wait: 0 Completion wait: 0 Completion wait: 0 nagios (5574) Reads Queued: 0, 0KiB Writes Queued: 0, 0KiB Read Dispatches: 0, 0KiB Write Dispatches: 0, 0KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 1, 16KiB Read Merges: 0, 0KiB Write Merges: 0, 0KiB IO unplugs: 0 Timer unplugs: 0 Allocation wait: 0 Allocation wait: 0 Dispatch wait: 0 Dispatch wait: 0 Completion wait: 0 Completion wait: 0 pdflush (776) Reads Queued: 0, 0KiB Writes Queued: 126, 504KiB Read Dispatches: 0, 0KiB Write Dispatches: 33, 132KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 33, 136KiB Read Merges: 0, 0KiB Write Merges: 1, 4KiB IO unplugs: 0 Timer unplugs: 0 Allocation wait: 0 Allocation wait: 0 Dispatch wait: 0 Dispatch wait: 0 Completion wait: 0 Completion wait: 0 swapper (0) Reads Queued: 0, 0KiB Writes Queued: 0, 0KiB Read Dispatches: 0, 0KiB Write Dispatches: 131, 536KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 233, 2,404KiB Read Merges: 0, 0KiB Write Merges: 0, 0KiB IO unplugs: 0 Timer unplugs: 0 Allocation wait: 0 Allocation wait: 0 Dispatch wait: 0 Dispatch wait: 0 Completion wait: 0 Completion wait: 0 CPU0 (8,2): Reads Queued: 0, 0KiB Writes Queued: 456, 1,824KiB Read Dispatches: 0, 0KiB Write Dispatches: 208, 1,996KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 273, 2,644KiB Read Merges: 0, 0KiB Write Merges: 289, 1,156KiB Read depth: 0 Write depth: 20 IO unplugs: 11 Timer unplugs: 0 CPU2 (8,2): Reads Queued: 0, 0KiB Writes Queued: 0, 0KiB Read Dispatches: 0, 0KiB Write Dispatches: 0, 0KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 0, 0KiB Read Merges: 0, 0KiB Write Merges: 0, 0KiB Read depth: 0 Write depth: 20 IO unplugs: 1 Timer unplugs: 0 Total (8,2): Reads Queued: 0, 0KiB Writes Queued: 456, 1,824KiB Read Dispatches: 0, 0KiB Write Dispatches: 208, 1,996KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 273, 2,644KiB Read Merges: 0, 0KiB Write Merges: 289, 1,156KiB IO unplugs: 12 Timer unplugs: 0 Throughput (R/W): 0KiB/s / 4,278KiB/s Events (8,2): 1,585 entries Skips: 0 forward (0 - 0.0%)
最后卸载debug文件系统
umount /sys/kernel/debug
外部链接:
Block I/O Layer Tracing: blktrace
blktrace User Guide
Tracing I/O usage on Linux
4. atop
对内核源码打补丁, 能够支持显示每个进程的IO
5. systemtap
例子见http://bpineau.livejournal.com/
6. DTrace
Solaris专用
iotop脚本见Top Ten DTrace (D) Scripts
iotop - display top disk I/O events by process.
7. 其它
pio and topio: for solaris,hp-ux,windows
Psio + other disk I/O by process tools: for solaris only
-fin-
No comments:
Post a Comment