Gopi Desaboyina Solaris Blogs

Just another WordPress.com weblog

tcpListenDrop and DTrace to get PID

Yesterday I had strange issue with one of the application box which is causing lot of 
tcpListenDrops. Even after Increasing the tcp_conn_req_max_q & tcp_conn_req_max_q0 has lot
lot of dropped packets. In the process of troubleshooting this issue I learnt
various DTrace scripts and commands.
Below DTrace script will give the PID which is cause for network tcp listen drops.
Using below DTrace you can find which are processes trying to make 
#!/usr/sbin/dtrace -s
#pragma D option quiet
fbt:ip:tcp_conn_request:entry
{
self->connp = (conn_t *)arg0;
self->tcp = (tcp_t *)self->connp->conn_sqp;
@backlog[self->tcp->tcp_cpid,self->tcp->tcp_conn_req_cnt_q,self->tcp->tcp_conn_req_max]=count();
printa("PID:%d \ttcp_conn_req_cnt_q/tcp_conn_req_max: %d/%d Number of times:%@d\n",@backlog);
}
listen system call and arguments.
dtrace -n syscall::listen:entry'{trace(execname);trace(pid);trace(arg0);trace(arg1);} 
You can use below command for finding portwise TCP Q size. 
Note: even though your system has large queue size. 
But applications while starting the listeners they specify how much 
max connections their app listener can handle.
/usr/sbin/ndd /dev/tcp tcp_listen_hash

BTW issue with my application was, It setting backlog Queue size as 2 there are more clients which are trying to connect to listener and which is causing listendrop. anyway I had to kill all client connections & reboot my app..some jackaxx client is making too many cons.

January 8, 2010 Posted by | Solaris | Leave a comment