Gopi Desaboyina Solaris Blogs

Just another WordPress.com weblog

Enabling Jumbo Frames in Solaris

Yesterday while setting up oracle RAC cluster we’ve to enable Jumbo Frames which has many advantages while transferring the data through interconnect. RAC advantages are out of scope in this blog. Below are the steps I followed for enabling Jumbo Frames ( MTU size more than 1500 bytes) in Sun E2900 which has CE interfaces.

 

1. Get the h/w addresses for CE interfaces.

# grep -i ce /etc/path_to_inst
"/ssm@0,0/pci@18,700000/network@1" 0 "ce"
"/ssm@0,0/pci@18,700000/network@3" 1 "ce"
"/ssm@0,0/pci@19,700000/pci@1/network@0" 2 "ce"
"/ssm@0,0/pci@19,700000/pci@1/network@1" 3 "ce"
"/ssm@0,0/pci@19,700000/pci@1/network@2" 4 "ce"
"/ssm@0,0/pci@19,700000/pci@1/network@3" 5 "ce"

 

2. Make a ce.conf file in /kernel/drv/ce.conf with below format

name="ce" parent=" " unit-address=" " accept-jumbo=1;

parent : It’s the device path with out network@

ex: /ssm@0,0/pci@18,700000 & /ssm@0,0/pci@19,700000/pci@1

unit-address : It’s the number immediately after network@ .

For example "/ssm@0,0/pci@18,700000/network@1" 0 "ce" device entry unit-address is 1   which immediately after network@. Note: it’s not 0 which is prior to ce. this has been mentioned in sun docs clearly.

accept-jumbo=1 ; For enabling jumbo frames on particular interface.

In my case I want to enable Jumbo frames for CE 1 & CE3 which are used for cluster interconnect. so my ce.conf looks like this.

#cat /kernel/drv/ce.conf
name="ce" parent="/ssm@0,0/pci@18,700000" unit-address="3" accept-jumbo=1; # For CE 0
name="ce" parent="/ssm@0,0/pci@19,700000/pci@1" unit-address="1" accept-jumbo=1; # For CE 1

 

3. Change the mtu in /etc/hostname.ce1 /etc/hostname.ce3 like below.

# cat /etc/hostname.ce1
systemname mtu 9194 group ipmp-private up
# cat /etc/hostname.ce3
mtu 9194 group ipmp-private up

4. Re-configure reboot the box and check the mtu for ce1 & ce3

  ifconfig –a |grep- i mtu

5. If MTU is configured correctly you can check the how many jumbo frames are passed through interface using

kstat –p ce | grep –i jumbo

* Above all steps makes sure that jumbo frames persistent across reboots ce1 & ce3 .

If you want to enable just temporarily you can do that using ndd commands.

#ifconfig ce1 unplumb
#ifconfig ce3 unplumb
# #Above 2 steps are imporant otherwise Jumbo Frames won’t be enable while interface is active.
#ndd -set /dev/ce instance 1
#ndd -set /dev/ce accept-jumbo 1
#ndd -set /dev/ce instance 3
#ndd -set /dev/ce accept-jumbo 1
#ifconfig ce1 plumb mtu 9194
#ifconfig ce3 plumb mtu 9194

About these ads

September 28, 2009 - Posted by | Solaris |

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: