Archive for the ‘Network’ category

How to find the port of particular application

November 29th, 2010

In my case Tomcat is using java as their process so i grep with java. Your application might be using different name.
#netstat -ntpl | grep java

tcp 0 0 ::ffff:127.0.0.1:8005 :::* LISTEN 6375/java
tcp 0 0 :::8009 :::* LISTEN 6375/java
tcp 0 0 :::8080 :::* LISTEN 6375/java

You should see at least one java process and you can use ps to identify if this is Tomcat.

# ps -ef | grep 6375
root 6375 1 0 May18 pts/2 00:01:06 /usr/java/jdk1.6.0_20/bin/java -Djava.util.logging.config.file=/install/apache-tomcat-5.5.29/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/install/apache-tomcat-5.5.29/common/endorsed -classpath /install/apache-tomcat-5.5.29/bin/bootstrap.jar -Dcatalina.base=/install/apache-tomcat-5.5.29 -Dcatalina.home=/install/apache-tomcat-5.5.29 -Djava.io.tmpdir=/install/apache-tomcat-5.5.29/temp org.apache.catalina.startup.Bootstrap start
root 9222 5091 0 18:29 pts/2 00:00:00 grep 6375

for killing the process just use.
kill -9 <process_id>

Tripwire,Intrusion Detection System

October 25th, 2010

Tripwire software can help to ensure the integrity of critical system files and directories by identifying all changes made to them. Tripwire configuration options include the ability to receive alerts via email if particular files are altered and automated integrity checking via a cron job. Using Tripwire for intrusion detection and damage assessment helps you keep track of system changes and can speed the recovery from a break-in by reducing the number of files you must restore to repair the system.

Tripwire compares files and directories against a baseline database of file locations, dates modified, and other data. It generates the baseline by taking a snapshot of specified files and directories in a known secure state. (For maximum security, Tripwire should be installed and the baseline created before the system is at risk from intrusion.) After creating the baseline database, Tripwire compares the current system to the baseline and reports any modifications, additions, or deletions.

1. Install Tripwire

Install Tripwire and customize the policy file — If not already done, install the tripwire RPM or download source from www.tripwire.org. Then, customize the sample configuration (/etc/tripwire/twcfg.txt) and policy (/etc/tripwire/twpol.txt) files and run the configuration script (/etc/tripwire/twinstall.sh).

*

Configuration file: /etc/tripwire/twcfg.txt

Usually it is not necessary to edit this file. The setting you may need to modify is MAILNOVIOLATIONS. By default this is set to true, which will send out an email everytime an integrity check is run. Setting MAILNOVIOLATIONS=false will cause Tripwire to only send an email if there is a violation.

*

Policy file: /etc/tripwire/twpol.txt

Most is already done, however there may be some files to check which doesn’t exist in the filesystem, uncomment them. Add your own files which should be checked.

Tripwire can email someone if a specific type of rule in the policy file is violated.

# Tripwire Binaries
(
rulename = “Tripwire Binaries”,
severity = $(SIG_HI),
emailto = martin dot zahn at akadia dot ch
)

*

Run the configuration script (/etc/tripwire/twinstall.sh).

# twinstall.sh

2. Initializing the Database

When initializing its database, Tripwire builds a collection of filesystem objects based on the rules in the policy file. This database serves as the baseline for integrity checks.

To initialize the Tripwire database, use the following command:

# tripwire –init

The The /var/lib/tripwire directory contains the Tripwire database of your system’s files (*.twd) and a report directory where Tripwire reports are stored. The Tripwire reports, named host_name-date_of_report-time_of_report.twr, detail the differences between the Tripwire database and your actual system files.

3. Run a Tripwire integrity check

When running an integrity check, Tripwire compares the current, actual filesystem objects with their properties as recorded in its database. Violations are printed to standard output and saved in a report file that can be accessed later by twprint.

An email configuration option in the policy file even allows particular email addresses to be sent notices when certain integrity violations occur.

To run an integrity check, use the following command:

# tripwire –check

4. Printing Reports

The twprint -m r command will display the contents of a Tripwire report in clear text. You must tell twprint which report file to display.

A twprint command for printing Tripwire reports looks similar to the following (all on one line):

# twprint -m r –twrfile /var/lib/tripwire/report/.twr

The -m r option in the command tells twprint to decode a Tripwire report. The –twrfile option tells twprint to use a specific Tripwire report file.

The name of the Tripwire report that you want to see includes the name of the host that Tripwire checked to generate the report, plus the creation date and time. You can review previously saved reports at any time. Simply type ls /var/lib/tripwire/report to see a list of Tripwire reports.

5. Take appropriate security measures

If monitored files have been altered inappropriately, you can either replace the originals from backups or reinstall the program.

6. Updating the Database after an Integrity Check

If you run an integrity check and Tripwire finds violations, you will first need to determine whether the violations discovered are actual security breaches or the product of authorized modifications. If you recently installed an application or edited critical system files, Tripwire will (correctly) report integrity check violations. In this case, you should update your Tripwire database so those changes are no longer reported as violations. However, if unauthorized changes are made to system files that generate integrity check violations, then you should restore the original file from a backup or reinstall the program.

To update your Tripwire database to accept the violations found in a report, you must specify the report you wish to use to update the database. When issuing the command to integrate those valid violations into your database, be sure to use the most recent report.

Type the following command (all on one line), where name is the name of the report to be used:

# tripwire –update –twrfile /var/lib/tripwire/report/.twr

Tripwire will show you the particular report using the default text editor (specified in the Tripwire configuration file on the EDITOR line). This is your chance to deselect files that you do not wish to be updated in the Tripwire database. It is important that you only allow authorized integrity violations to be changed in the database.

All proposed updates to the Tripwire database start with a [x] before the file name. If you want to specifically exclude a valid violation from being added to the Tripwire database, remove the x from the box. To accept any files with an x beside them as changes, write the file in the editor and quit the text editor. This signals to Tripwire to alter its database and not report these files as violations.

For example, the default text editor for Tripwire is vi. To write the file with vi and make the changes to the Tripwire database when updating with a specific report, type :wq in vi’s command mode and press [Enter]. You will be asked to enter your local passphrase. Then, a new database file will be written to include the valid violations.

After a new Tripwire database is written, the newly authorized integrity violations will no longer show up as warnings when the next integrity check is run.

7. Updating the Policy File

If you want to actually change the files Tripwire records in its database or modify the severity in which violations are reported, you need to edit your Tripwire policy file.

First, make whatever changes are necessary to the sample policy file (/etc/tripwire/twpol.txt). A common change to this policy file is to comment out any files that do not exist on your system so that they will not generate a file not found error in your Tripwire reports. For example, if your system does not have a /etc/smb.conf file, you can tell Tripwire not to try to look for it by commenting out its line in twpol.txt:

# /etc/smb.conf -> $(SEC_CONFIG) ;

Next, you must tell Tripwire to generate a new /etc/tripwire/tw.pol signed file and then generate an updated database file based on this policy information. Assuming /etc/tripwire/twpol.txt is the edited policy file, use this command:

# twadmin –create-polfile -S site.key /etc/tripwire/twpol.txt

You will be asked for the site passphrase. Then, the twpol.txt file will be parsed and signed.

It is important that you update the Tripwire database after creating a new /etc/tripwire/tw.pol file. The most reliable way to accomplish this is to delete your current Tripwire database and create a new database using the new policy file.

If your Tripwire database file is named /var/lib/tripwire/paragon.twd, type this command:

# rm /var/lib/tripwire/paragon.twd

Then type the command to create a new database:

# tripwire –init

A new database will be created according to the instructions in the new policy file. To make sure the database was correctly changed, run the first integrity check manually and view the contents of the resulting report. See the section called Running an Integrity Check and the section called Printing Reports for specific instructions on these points.

8. Signing the Configuration File

The text file with the configuration file changes (commonly /etc/tripwire/twcfg.txt) must be signed to replace the /etc/tripwire/tw.cfg and be used by Tripwire when it runs its integrity check. Tripwire will not recognize any configuration changes until the configuration text file is correctly signed and used to replace the /etc/tripwire/tw.pol file.

If your altered configuration text file is /etc/tripwire/twcfg.txt, type this command to sign it, replacing the current /etc/tripwire/tw.pol file:

# twadmin –create-cfgfile -S site.key /etc/tripwire/twcfg.txt

Since the configuration file does not not alter any Tripwire policies or files tracked by the application, it is not necessary to regenerate the database of monitored system files.

9. Run tripwire from cron

You can run a Tripwire integrity check daily by inserting the following commands in the script /etc/cron.daily/tripwire-check:

#!/bin/sh
HOST_NAME=`uname -n`
if [ ! -e /var/lib/tripwire/${HOST_NAME}.twd ]; then
echo “Error: Tripwire database for ${HOST_NAME} not found”
echo “Run “/etc/tripwire/twinstall.sh” and/or “tripwire –init”"
else
test -f /etc/tripwire/tw.cfg && /usr/sbin/tripwire –check
fi

10. Check if tripwire can send E-Mails

Tripwire can email someone if a specific type of rule in the policy file is violated. To configure Tripwire to do this, you first have to know the email address of the person to be contacted if a particular integrity violation occurs, plus the name of the rule you would like to monitor. Note that on large systems with multiple administrators, you can have different sets of people notified for certain violations and no one notified for minor violations.

Once you know who to notify and what to notify them about, add an emailto= line to the rule directive section of each rule. Do this by adding a comma after the severity= line and putting emailto= on the next line, followed by the email addresses to send the violation reports for that rule. Multiple emails will be sent if more than one email address is specified and they are separated by a semi-colon.

(
rulename = “Networking Programs”,
severity = $(SIG_HI),
emailto = pawan.kda@gmail.com
)

Sending Test Email Messages

To make sure that Tripwire’s email notification configuration can actually send email correctly, use the following command:

/usr/sbin/tripwire –test –email pawan.kda@gmail.com

A test email will immediately be sent to the email address by the tripwire program.

How to Install Munin on CentOS

June 18th, 2010

Introduction:-

Munin is a highly flexible and powerfull solution used to create graphs of virtually everything imaginable throughout your network, while still maintining a rattling easy of installation and configuration. Munin is a networked resource monitoring tool that can help analyze resource trends and “what just happened to kill our performance?” problems. It is designed to be very plug and play. A default installation provides a lot of graphs with almost no work.

About Munin

Munin the monitoring tool surveys all your computers and remembers what it saw. It presents all the information in graphs through a web interface. Its emphasis is on plug and play capabilities. After completing a installation a high number of monitoring plugins will be playing with no more effort.

Using Munin you can easily monitor the performance of your computers, networks, SANs, applications, weather measurements and whatever comes to mind. It makes it easy to determine “what’s different today” when a performance problem crops up. It makes it easy to see how you’re doing capacity-wise on any resources.

Munin uses the excellent RRDTool (written by Tobi Oetiker) and the framework is written in Perl, while plugins may be written in any language. Munin has a master/node architecture in which the master connects to all the nodes at regular intervals and asks them for data. It then stores the data in RRD files, and (if needed) updates the graphs. One of the main goals has been ease of creating new plugins (graphs).

This site is a wiki as well as a project management tool. We appreciate any contributions to the documentation. While this is the homepage of the Munin project, we will still make all releases through Sourceforge.

I will use the following scenario:

munin-server.example – 192.168.1.254        # Munin Server
station1.example.com – 192.168.1.1            # Munin client
station2.example.com – 192.168.1.2            # Munin client

You can add many clients as much you want. Here i added two clients. Here on more thing to notice, using hostname is not necessary. We are not required any DNS resolution, it is just for clarity to recognize your client easily.

Server Side
Let start with Installation and configuration:

wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
Enable The RPMforge Repository
yum install
yum install perl-Net-SNMP perl-Net-Server perl-HTML-Template perl-Log-Log4perl-RRDs perl-RRD-Simple

Install And Configure munin
yum install munin munin-node
chkconfig –levels 235 munin-node on
/etc/init.d/munin-node start

By default it will create the following directories,files.

Main Server main Configuration file:
/etc/munin/munin.conf

Munin cron file:
/usr/bin/munin-cron

Munin libraries:
/var/lib/munin

Munin logs:
/var/log/munin

Munin Document Root
/var/www/munin

Step 2 : Changing default Munin Document Root Path:
mv /var/www/munin /var/www/html/munin

Step 3 : Adding clients/nodes to monitor them in /etc/munin/munin.conf:

vi /etc/munin/munin.conf

[station1.example.com]
address 192.168.1.1
use_node_name yes
[station2.example.com]
address 192.168.1.2
use_node_name yes

Step 4 : Setting up permissions:

groupadd munhttp
usermod -G munhttp munin
usermod -G munhttp apache
chown -v apache.munhttp /var/www/html/munin

Step 4 : Setting up cron to schedule the munin to update every 5 minutes:
crontab -e
*/5 * * * * /usr/bin/munin-cron –force-root

Step 5 : Restart Apache:
service httpd restart

Step 6: Password protection for munin
AuthType Basic
AuthName “Members Only”
AuthUserFile /var/www/munin/.htpasswd
<limit GET PUT POST>
require valid-user
</limit>

htpasswd -c /var/www/munin/.htpasswd admin

Client Side
Here we go with installing the clients/nodes.

Step 1 : Installing the required packages :
yum install perl-Net-Server perl-Net-SNMP perl-Digest-HMAC perl-Digest-SHA1 munin-node

Step 2 : Editing /etc/munin/munin-node.conf for server ip address, you have to add it bottom of the file, below the line ” allow ^127.0.0.1$”
vi /etc/munin/munin-node.conf
allow ^192.168.1.254$

Step 3 : The following command will scan the system for available services and will auto configure it. e.g Sendmail, mysql.

munin-node-configure

Step 4 : Start Munin-node client to available for the server.
munin-node

Note : If you modify the configure file “/etc/munin/munin-node.conf” you will need to run the following command to re-read the configure file.
service munin-node restart

You can follow the “Client Side” configuration for the rest of clients/nodes.

Common Issues:

1. No values/stats are displayed or blank.
Please verify that munin-node service is running at machine. Verify it by telnet from machine itself:
# telnet localhost 4949
You should see munin prompt here, write fetch cpu and press ENTER. You should see various values related to your cpu. If you are not able to telnet to port 4949 (munin port) then possibly munin-node is not running or your iptable (firewall) or SELinux policy is preventing its usage.

After checking from localhost, go to Server machine and try to telnet from there:
# telnet 192.168.0.5 4949
You should get munin prompt and see values after issuing fetch cpu command there. If not, then firewall/selinux is preventing access this port or you didnt allowed this server from your munin-node.conf to get updates.

2. Why not compile from source?
Thats the best idea but when I tried the same, I got lots of perl modules dependencies/compile errors. Though, I managed to get its working but for a quick setup, yum is your friend.

3. I do not need all these values coming in graphs.
You can remove any plugin for which you do not want graph. Plugins are stored in /etc/munin/plugins directory and configured in plugins.conf. Configure these as per your requirements.

Thanks
Manoj

Network Load Balancing, Bonding /Teaming of Two NIC cards

April 16th, 2010

Step #1: Create a bond0 configuration file
First,creating bond0 config file:

# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.1.20
NETWORK=192.168.1.0
NETMASK=255.255.255.0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes

Replace above IP address with your actual IP address.
Step #2: Modify eth0 and eth1 config files:

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Step # 3: Load bond driver/module

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configuration file:
# vi /etc/modprobe.conf

Append following two lines:
alias bond0 bonding
options bond0 mode=balance-alb miimon=100

Step # 4: Test configuration

First, load the bonding module:

# modprobe bonding Restart networking service in order to bring up bond0 interface:

# service network restart

Verify everything is working:

# less /proc/net/bonding/bond0Output:

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:59

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:63

List all interfaces:
# ifconfigOutput:
Now you have bond multiple network interfaces into a single channel (NIC)

Thanks
Pawan Kumar

Enabling SNMP Traps and Informs

April 11th, 2010

Problem

You want the router to generate SNMP traps or informs in response to various network events.

Solution

The following configuration commands will enable your router to send unsolicited SNMP traps to a network management server:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#snmp-server enable traps

Router(config)#snmp-server host 172.25.1.1 ORATRAP config entity envmon hsrp

Router(config)#snmp-server host router.manoj.com ORATRAP bgp snmp envmon

Router(config)#end

Router# 

Notice that the snmp-server host command will accept either an IP address or a hostname.Beginning with SNMP Version 2c, Cisco routers also support SNMP informs. To enable SNMP informs, use the following commands:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#snmp-server enable informs

Router(config)#snmp-server host 172.25.1.1 informs version 2c ORATRAP snmp envmon

Router(config)#end

Router#

Discussion

SNMP Traps originate from the router’s agent and are sent via UDP (port 162) to the network management station (NMS). Unlike the information that the router sends to the NMS in response to an SNMP poll, a trap is unsolicited. The router’s agent decides that something important has happened, and that it needs to tell the NMS about it. Before the router agent can send traps, you must enable global trap support (see Table 17-4) and configure the trap host.

SNMP traps are one of the basic elements of fault management. In fact, Requirements for IP Version 4 Routers (RFC 1812) states that all routers must be capable of sending SNMP traps.

Cisco routers can send a large variety of different SNMP traps, including both standard traps, described in RFCs, and Cisco specific traps. The first step in configuring trap support is to enable the particular trap types you wish to use. In our examples, we choose to enable all SNMP trap types by using the configuration command snmp-server enable traps. The fact that we didn’t specify individual trap types implicitly enables all trap types. However, you can restrict the router to send only certain types of traps that you are interested in receiving. The various trap-type keywords are shown in Table 17-4. Note that this is a global command that affects all SNMP trap receivers.

Table 17-4. Cisco SNMP trap types

Keyword

Description

bgp Allow BGP state change traps
calltracker Send Call Tracker cal start/end notifications
config Allow SNMP configuration traps
cpu Send cpu related notifications
director Allow Distributed Director notifications
dspu Allow dspu event traps
eigrp Enable EGIRP SIA and neighbor auth failure traps
entity Allow SNMP entity traps
envmon Allow environmental monitor traps
flash Send flash insertion and removal traps
frame-relay Allow SNMP frame-relay traps
hsrp Allow SNMP HSRP traps
iplocalpool Allow IP local pool traps
ipmobile Allow mobile IP notifications
Ipsec Send IPsec notifications
isdn Allow SNMP ISDN traps
l2tun-pseudowire-status Send pseudowire state change notifications
l2tun-session Send Layer 2 tunnel session traps
llc2 Sends logical link control type-2 notifications
memory Allow memory pool and buffer pool notifications
mpls-ldp Send MPLS ldp status change traps
mpls-traffic-eng Send MPLS TE tunnel status change notifications
mpls-vpn Send MPLS VPN notifications
ospf Send OSPF sham link notifications
pim Allow PIM notificaitons
repeater Send standard repeater notifications
rsrb Allow rsrb event traps
rsvp Allow RSVP flow change traps
rtr Allow SNMP Response Time Reporter traps
sdlc Allow SDLC event traps
sdllc Allow SDLLC event traps
snmp Allow SNMP-type notifications
srp Send SRP notifications
stun Allow stun event traps
syslog Allow SNMP syslog traps
tty Allow TCP connection traps
udp-port The server host’s UDP port number
voice Allow SNMP voice traps
vrrp Send VRRP notifications
vsimaster Send VSI master notifications
X25 Allow x25 event traps

 

For example, you would use the following commands to tell the router to send only BGP and environmental-type traps:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#snmp-server enable traps bgp
Router(config)#snmp-server enable traps envmon
Router(config)#end
Router# 

You can also disable a particular type of SNMP trap by using the following command:

Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#no snmp-server enable traps envmon
Router(config)#end
Router#

The following command displays which SNMP trap-types are enabled on a router:
Router#show running-config | include snmp-server enable

snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps hsrp
snmp-server enable traps config

snmp-server enable traps entity

snmp-server enable traps envmon

snmp-server enable traps bgp

snmp-server enable traps ipmulticast

snmp-server enable traps msdp

snmp-server enable traps rsvp

snmp-server enable traps frame-relay

snmp-server enable traps syslog

snmp-server enable traps rtr

snmp-server enable traps dlsw

snmp-server enable traps dial

snmp-server enable traps dsp card-status

snmp-server enable traps voice poor-qov

Router#

 

The second step in configuring SNMP traps is to define the trap recipient by using the snmp-server host command. This command has the following attributes:

snmp-server host host-addr [traps | informs] [version {1 | 2c} ] community-string  [udp-port port] [trap-type]

 

The host-addr argument is the name or IP address of the NMS server that will receive the traps. You can define whether the router will send SNMP traps or informs to this host by specifying either the traps or informs keyword. If neither is specified, the default is to send traps. Also, you can specify which version of SNMP traps the router will send by including either version 1 or version 2c. If neither version is specified, the router will default to Version 1. Note that informs don’t exist in SNMP Version 1, so you must specify Version 2c (or version 3) if you want to enable this feature.

The community string argument specifies the community string that the router will send within the SNMP trap or inform. This doesn’t need to match either the read-only or read-write community strings on the router.

You can change the default SNMP trap port from 162 (the default) to another value with the optional udp-port keyword. This keyword must be followed by the alternative UDP port number that you want to use.

Finally, if the trap-type keyword is present, it allows you to configure the types of types that the router will send to this server. There is a list of valid trap types in Table 17-4. The command can accept one or more types. However, if no trap types are included, the router will default to sending every enabled trap type.

There are two important things to note about this command. First, you must enable trap-types via the global command before you can specify them for a particular host. Second, this command will allow you to send different sets of traps to different servers. This can sometimes be useful if you have multiple NMS servers that handle different management functions.

The configuration for SNMP informs is almost the same as SNMP traps. The main difference is that you can’t enable individual inform types by using the global snmp-server enable informs command. The global inform command lacks the granularity of the same trap-based command. However, you can still enable specific inform types on the host-level command. This can mean more typing if there are several inform recipients. But there is no loss of functionality.

Thanks
Manoj Chauhan

Proposed IT subnetting Infrastructure

January 24th, 2010

Proposed IT subnetting Infrastructure 

  1. For External communication we will use Cisco Router(Cisco ASA5520 firewall)
  2. For internal communication we will use managed switch Dell Power Connect 6248

Why use a Managed Switch?

  • Limit broadcast traffic and increase security using VLANs
  • Remove traffic bottlenecks using port trunking
  • Guarantee bandwidth for time-sensitive voice and video traffic using Traffic Prioritisation
  • We will manage switch for internal communication. I mean to say when one server (Test5) wants to talk to another server (Test) at that time it will use manager switch for the internal communication. It will not forward the traffic to the router.

Switching Services

Unlike bridges that use software to create and manage a filter table, switches use application specific integrated circuits (ASICs) to build and maintain their filter tables. But it’s still okay to think of a layer 2 switch as a multiport bridge because their basic reason for being is the same: to break up collision domains.

Layer 2 switches are faster than routers because they don’t take up time looking at the Network layer header information. Instead, they look at the frame’s hardware addresses before deciding to either forward the frame or drop it. 

Switches create private dedicated collision domains and provide independent bandwidth on each port, unlike hubs. Figure 1.1 shows five hosts connected to a switch—all running 10Mbps half duplex to the server. Unlike a hub, each host has 10Mbps dedicated communication to the server.

nnetwork
FIGURE 1.1 Switches create private domains.

Three Switch Functions at Layer 2 

  1.  
    1. Address learning
    2. Forward/filter decisions
    3. Loop avoidance 

Here’s a list of the basic tasks we’ll be done in the switch Dell Power Connect 6248 

  1. Setting the passwords
  2. Setting the hostname
  3. To configure the switch with different VLANs and other network functions

            Add VLAN and Description

            Configuring the IP address and subnet mask & gateway address

            Assigning switch ports to VLANs.

            Routing between VLANs

  1. Hosts in a VLAN live in their own broadcast domain and can communicate freely. VLANs create network partitioning and traffic separation at layer 2 of the OSI, and as I said when I told you why we still need routers, if you want hosts or any other IP-addressable device to communicate between VLANs, a layer 3 device is absolutely necessary.

What we see in Figure 1.2 is that each router interface is plugged into an access link. This means that each of the routers’ interface IP addresses would then become the default gateway address for each host in each VLAN.

FIGURE 1.2 Router with individual VLAN associations

nnetwork

  1. Router connecting three VLANs together for inter-VLAN communication, one interface for each VLAN.
  2. Remember that a created VLAN is unused until it is assigned to a switch port or ports, and that all ports are always assigned in VLAN 1 unless set otherwise. 

Configuring Inter-VLAN
nnetwork
The first thing we need to do here is figure out which subnets are being used. By looking at the router configuration in the figure, you can see that we’re using 192.168.1.0 with DMZ, 192.168.1.32 with Corporate and 192.168.1.64 with Production. And by looking at the switch configuration, you can see that ports 1,2 and 3 etc are in DMZ, and port 4,5 and 6 etc is in Corporate and port 7 and 8 etc is in Production. This means that Hosts on DMZ are in VLAN 1, hosts in Corporate are in VLAN 2 and hosts on Production are in VLAN 3. Here’s what the hosts’ IP addresses should be: 

DMZ: 192.168.1.0, 255.255.255.224, default gateway 192.168.1.30
Corporate: 192.168.1.32, 255.255.255.224, default gateway 192.168.1.62
Production: 192.168.1.64, 255.255.255.224, default gateway 192.168.1.94 

Inter-VLAN example
nnetwork
Complete Working Flow Diagram

 

 

 

nnetwork

Thanks
Manoj Chauhan

Boost Network Performance / Throughput

January 24th, 2010

Network Performance Document
Linux Configure Jumbo Frames to Boost Network Performance / Throughput
Most modern Linux distros (read as Linux Kernel 2.6.17+) does support frames larger than 1500 bytes. This can improve the performance. First, make sure your network driver supports custom MTU. Second you need to have a compatible gigabit NIC

Jumbo frames can reduce server overhead such as a big decrease in CPU usage when transferring larger file. Also you should see some increase in network throughput.

# ifconfig eth0 mtu 9000

Make changes permanent

Edit the network configuration file for eth0 interface – for example, /etc/sysconfig/network-script/ifcfg-eth0 (CentOS / RHEL / Fedora Linux):

# vi /etc/sysconfig/network-script/ifcfg-eth0

Append the following configuration directive, which specifies the size of the frame in bytes:

MTU=9000

# /etc/init.d/networking restart

To confirm the MTU used between two specific devices. use ip command as follows:

# ip route get {IP-address}
# ip route get 192.168.1.1

Output:
192.168.1.1 dev eth0 src 192.168.1.100 cache mtu 9000 advmss 1460 hoplimit 64

Tuning sysctl.conf

The sysctl.conf of a server is something that is seldom optimized for performance. You can get a tremendous boost in throughput by adjusting these settings. This configuration has been written by Steve from Rack911. I have applied this configuration to servers ranging from Celeron 1.7Ghz to Dual Xeon 2.8Ghz servers, and on the whole, the load on each lowered after making the changes.

First make a backup of your old /etc/sysctl.conf file by running the following command, logged in as root:
cp /etc/sysctl.conf /etc/sysctl.conf.bak
Now enter:
vi /etc/sysctl.conf
and replace the contents of the file with the following:

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Disables packet forwarding
net.ipv4.ip_forward=0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.eth0.log_martians = 0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Disables the magic-sysrq key
kernel.sysrq = 0
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
# Enable TCP SYN Cookie Protection
net.ipv4.tcp_syncookies = 1
# Enable ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts = 1
# Enable bad error message Protection
net.ipv4.icmp_ignore_bogus_error_responses = 1
# Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 1
# Increases the size of the socket queue (effectively, q0).
net.ipv4.tcp_max_syn_backlog = 1024
# Increase the tcp-time-wait buckets pool size
net.ipv4.tcp_max_tw_buckets = 1440000
# Allowed local port range
net.ipv4.ip_local_port_range = 16384 65536
CTRL + X to exit and save the file
To make your changes take effect immediately, type this command:
/sbin/sysctl -p
You can watch your server load by entering “uptime” command via SSH.There you have it, a quick few things you can do to your server to boost performance and lower CPU load.
TCP Tuning
For servers that are serving up huge numbers of concurrent sessions, there are some TCP options that should probably be enabled. In order to optimize TCP performance for this situation, I would suggest tuning the following parameters:
#echo 1024 65000 > /proc/sys/net/ipv4/ip_local_port_range
Tuning Syn Backlog
Set the Syn backlog to a high number so Squid will never be starved due to a kernel limiting network connection. The default value is 1024.
#echo 16384 > /proc/sys/net/ipv4/tcp_max_syn_backlog
Set up Network Bonding
Linux allows binding multiple network interfaces into a single channel/NIC using special kernel module called bonding. The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical “bonded” interface.

TCP Tuning Guide
Linux TCP Tuning
There are a lot of differences between Linux version 2.4 and 2.6, so first we’ll cover the tuning issues that are the same in both 2.4 and 2.6. To change TCP settings in, you add the entries below to the file /etc/sysctl.conf, and then run “sysctl -p”.
Like all operating systems, the default maximum Linux TCP buffer sizes are way too small. I suggest changing them to the following settings:
# increase TCP max buffer size setable using setsockopt()
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
# set max to at least 4MB, or higher if you use very high BDP paths
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
Note: you should leave tcp_mem alone. The defaults are fine.
Another thing you can try that may help increase TCP throughput is to increase the size of the interface queue. To do this, do the following:
ifconfig eth0 txqueuelen 1000
I’ve seen increases in bandwidth of up to 8x by doing this on some long, fast paths. This is only a good idea for Gigabit Ethernet connected hosts, and may have other side effects such as uneven sharing between multiple streams.
Also, I’ve been told that for some network paths, using the Linux ‘tc’ (traffic control) system to pace traffic out of the host can help improve total throughput.
________________________________________
Linux 2.4
Starting with Linux 2.4, Linux has implemented a sender-side autotuning mechanism, so that setting the opitimal buffer size on the sender is not needed. This assumes you have set large buffers on the recieve side, as the sending buffer will not grow beyond the size of the recieve buffer.
However, Linux 2.4 has some other strange behavior that one needs to be aware of. For example: The value for ssthresh for a given path is cached in the routing table. This means that if a connection has has a retransmition and reduces its window, then all connections to that host for the next 10 minutes will use a reduced window size, and not even try to increase its window. The only way to disable this behavior is to do the following before all new connections (you must be root):
sysctl -w net.ipv4.route.flush=1
More information on various tuning parameters for Linux 2.4 are available in the Ipsysctl tutorial .
________________________________________
Linux 2.6
Starting in Linux 2.6.7 (and back-ported to 2.4.27), linux includes alternative congestion control algoritms beside the traditional ‘reno’ algorithm. These are designed to recover quickly from packet loss on high-speed WANs.
Linux 2.6 also includes and both send and receiver-side automatic buffer tuning (up to the maximum sizes specified above). There is also a setting to fix the ssthresh caching weirdness described above.
There are a couple additional sysctl settings for 2.6:
# don’t cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
# recommended to increase this for 1000 BT or higher
net.core.netdev_max_backlog = 2500
# for 10 GigE, use this
# net.core.netdev_max_backlog = 30000
Starting with version 2.6.13, Linux supports pluggable congestion control algorithms . The congestion control algorithm used is set using the sysctl variable net.ipv4.tcp_congestion_control, which is set to cubic or reno by default, depending on which version of the 2.6 kernel you are using.
To get a list of congestion control algorithms that are available in your kernel, run:
sysctl net.ipv4.tcp_available_congestion_control
The choice of congestion control options is selected when you build the kernel. The following are some of the options are available in the 2.6.23 kernel:
• reno: Traditional TCP used by almost all other OSes. (default)
• cubic: CUBIC-TCP
• bic: BIC-TCP
• htcp: Hamilton TCP
• vegas: TCP Vegas
• westwood: optimized for lossy networks
For very long fast paths, I suggest trying cubic or htcp if reno is not is not performing as desired. To set this, do the following:

sysctl -w net.ipv4.tcp_congestion_control=htcp
More information on each of these algorithms and some results can be found here .
More information on tuning parameters and defaults for Linux 2.6 are available in the file ip-sysctl.txt, which is part of the 2.6 source distribution.
And finally a warning for both 2.4 and 2.6: for very large BDP paths where the TCP window is > 20 MB, you are likely to hit the Linux SACK implementation problem. If Linux has too many packets in flight when it gets a SACK event, it takes too long to located the SACKed packet, and you get a TCP timeout and CWND goes back to 1 packet. Restricting the TCP buffer size to about 12 MB seems to avoid this problem, but clearly limits your total throughput. Another solution is to disable SACK.
________________________________________
Linux 2.2
If you are still running Linux 2.2, upgrade! If this is not possible, add the following to /etc/rc.d/rc.local
echo 8388608 > /proc/sys/net/core/wmem_max
echo 8388608 > /proc/sys/net/core/rmem_max
echo 65536 > /proc/sys/net/core/rmem_default
echo 65536 > /proc/sys/net/core/wmem_default

Network Monitoring Tools

http://acs.lbl.gov/NCS/download/

Small to Medium-Sized Network Using Catalyst 3560 Switches

January 17th, 2010

Figure 1-1 shows a configuration for a network of up to 500 employees. This network uses Catalyst 3560 Layer 3 switches with high-speed connections to two routers. For network reliability and load balancing, this network has HSRP enabled on the routers and on the switches. This ensures connectivity to the Internet, WAN, and mission-critical network resources in case one of the routers or switches fails. The switches are using routed uplinks for faster failover. They are also configured with equal-cost routing for load sharing and redundancy.

The switches are connected to workstations, local servers, and IEEE 802.3af compliant and noncompliant powered devices (such as Cisco IP Phones). The server farm includes a call-processing server running Cisco CallManager software. Cisco CallManager controls call processing, routing, and IP phone features and nfiguration. The switches are interconnected through Gigabit interfaces.

This network uses VLANs to logically segment the network into well-defined broadcast groups and for security management. Data and multimedia traffic are configured on the same VLAN. Voice traffic from the Cisco IP Phones are configured on separate VVIDs. If data, multimedia, and voice traffic are assigned to the same VLAN, only one VLAN can be configured per wiring closet.

When an end station in one VLAN needs to communicate with an end station in another VLAN, a router or Layer 3 switch routes the traffic to the appropriate destination VLAN. In this network, the switches are roviding inter-VLAN routing. VLAN access control lists (VLAN maps) on the switches provide
intra-VLAN security and prevent unauthorized users from accessing critical pieces of the network. In addition to inter-VLAN routing, the multilayer switches provide QoS mechanisms such as DSCP
priorities to prioritize the different types of network traffic and to deliver high-priority traffic in a predictable manner. If congestion occurs, QoS drops low-priority traffic to allow delivery of
high-priority traffic.

For pre-standard and IEEE 802.3af-compliant powered devices connected to Catalyst Power over Ethernet (PoE) switches, 802.1p/Q QoS gives voice traffic forwarding-priority over data traffic.
Catalyst PoE switch ports automatically detect any Cisco pre-standard and IEEE 802.3af-compliant powered devices that are connected. Each PoE switch port provides 15.4 W of power per port. The
powered device, such as an IP phone, can receive redundant power when it is also connected to an AC power source. Powered devices not connected to Catalyst PoE switches must be connected to AC power
sources to receive power.

Cisco CallManager controls call processing, routing, and IP phone features and configuration. Users with workstations running Cisco SoftPhone software can place, receive, and control calls from their PCs.
Using Cisco IP Phones, Cisco CallManager software, and Cisco SoftPhone software integrates telephony and IP networks, and the IP network supports both voice and data.
With the multilayer switches providing inter-VLAN routing and other network services, the routers focus on firewall services, Network Address Translation (NAT) services, voice-over-IP (VoIP) gateway
services, and WAN and Internet access.

Figure 1-1 Catalyst 3560 Switches in a Collapsed Backbone Configuration

Switch-failover
Failover Overview

The failover feature lets you use a standby FWSM to take over the functionality of a failed FWSM. Failover is compatible with both routed and transparent firewall modes, and with single and
multiple context modes. When the active module fails, it changes to the standby state, while the standby module changes to the
active state.

The module that becomes active takes over the active module IP addresses (or, for transparent firewall, the management IP address) and MAC address, and it begins passing traffic. The FWSM has one MAC
address for all interfaces. The module that was active and is now in standby state takes over the standby IP addresses and MAC address.

Because network devices see no change in the MAC to IP address pairing, failover is unnoticed by the rest of the network. However, the host switch needs to reassociate the new active and standby chassis
slots with their corresponding MAC addresses. The FWSM helps this process by sending out gratuitous ARPs on all its VLAN interfaces.

The standby module can effectively take over as the active module because it has the same configuration, and it is assigned the same VLANs from the switch.

Regular and Stateful Failover

The FWSM supports two types of failover:

• Regular failover—When a failover occurs, all active connections are dropped and clients need to reestablish connections when the new active module takes over.
• Stateful failover—During normal operation, the active module continually passes per-connection stateful information (for each context) to the standby module. The interval between stateful
information updates is 10 seconds, but if you set the module polltime to be greater than 10 seconds, then that interval is used.

After a failover occurs, the same connection information is available at the new active module. Supported end-user applications are not required to reconnect to keep the same communication
session.

The state information passed to the standby module includes the following data:

– NAT translation table
– TCP connection states
– UDP connection states (for connections lasting at least 15 seconds)
– HTTP connection states (Optional)
– H.323, SIP, and MGCP UDP media connections
– ARP table
– (Transparent firewall mode only) MAC address table

Failover and State Links

This section describes the failover link and, for stateful failover, the state link, and it includes the following topics:

• Failover Link
• State Link

Failover Link

The two modules constantly communicate over a failover link to determine the operating status of each module. Communications over the failover link include the following data:

• The module state (active or standby).
• Hello messages (also sent on all other interfaces).
• Configuration synchronization between the two modules.

The failover link uses a special VLAN interface that you do not configure as a normal networking interface; rather, it exists only for failover communications. This VLAN should only be used for the
failover link (and optionally for the state link). For multiple context mode, the failover link resides in the system configuration. This interface (and the
state link, if used) is the only configurable interface in the system configuration.

State Link

To use stateful failover, configure a state link to pass all state information. This link can be the same as the failover link, but we recommend that you assign a separate VLAN and IP address for the state link.
The state traffic can be large, and performance is improved with separate links. In multiple context mode, the state link resides in the system configuration. This interface and the
failover interface are the only interfaces in the system configuration.

Network Performance Document

October 31st, 2009

Network Performance

Linux Configure Jumbo Frames to Boost Network Performance / Throughput
Most modern Linux distros (read as Linux Kernel 2.6.17+) does support frames larger than 1500 bytes. This can improve the performance. First, make sure your network driver supports custom MTU. Second you need to have a compatible gigabit NIC

Jumbo frames can reduce server overhead such as a big decrease in CPU usage when transferring larger file. Also you should see some increase in network throughput.

# ifconfig eth0 mtu 9000

Make changes permanent

Edit the network configuration file for eth0 interface – for example, /etc/sysconfig/network-script/ifcfg-eth0 (CentOS / RHEL / Fedora Linux):

# vi /etc/sysconfig/network-script/ifcfg-eth0

Append the following configuration directive, which specifies the size of the frame in bytes:

MTU=9000

# /etc/init.d/networking restart

To confirm the MTU used between two specific devices. use ip command as follows:

# ip route get {IP-address}
# ip route get 192.168.1.1

Output:
192.168.1.1 dev eth0 src 192.168.1.100 cache mtu 9000 advmss 1460 hoplimit 64

Tuning sysctl.conf

The sysctl.conf of a server is something that is seldom optimized for performance. You can get a tremendous boost in throughput by adjusting these settings. This configuration has been written by Steve from Rack911. I have applied this configuration to servers ranging from Celeron 1.7Ghz to Dual Xeon 2.8Ghz servers, and on the whole, the load on each lowered after making the changes.

First make a backup of your old /etc/sysctl.conf file by running the following command, logged in as root:
cp /etc/sysctl.conf /etc/sysctl.conf.bak
Now enter:
vi /etc/sysctl.conf
and replace the contents of the file with the following:

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Disables packet forwarding
net.ipv4.ip_forward=0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.eth0.log_martians = 0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Disables the magic-sysrq key
kernel.sysrq = 0
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
# Enable TCP SYN Cookie Protection
net.ipv4.tcp_syncookies = 1
# Enable ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts = 1
# Enable bad error message Protection
net.ipv4.icmp_ignore_bogus_error_responses = 1
# Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 1
# Increases the size of the socket queue (effectively, q0).
net.ipv4.tcp_max_syn_backlog = 1024
# Increase the tcp-time-wait buckets pool size
net.ipv4.tcp_max_tw_buckets = 1440000
# Allowed local port range
net.ipv4.ip_local_port_range = 16384 65536
CTRL + X to exit and save the file
To make your changes take effect immediately, type this command:
/sbin/sysctl -p
You can watch your server load by entering “uptime” command via SSH.There you have it, a quick few things you can do to your server to boost performance and lower CPU load.
TCP Tuning
For servers that are serving up huge numbers of concurrent sessions, there are some TCP options that should probably be enabled. In order to optimize TCP performance for this situation, I would suggest tuning the following parameters:
#echo 1024 65000 > /proc/sys/net/ipv4/ip_local_port_range
Tuning Syn Backlog
Set the Syn backlog to a high number so Squid will never be starved due to a kernel limiting network connection. The default value is 1024.
#echo 16384 > /proc/sys/net/ipv4/tcp_max_syn_backlog
Set up Network Bonding
Linux allows binding multiple network interfaces into a single channel/NIC using special kernel module called bonding. The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical “bonded” interface.

TCP Tuning Guide
Linux TCP Tuning
There are a lot of differences between Linux version 2.4 and 2.6, so first we’ll cover the tuning issues that are the same in both 2.4 and 2.6. To change TCP settings in, you add the entries below to the file /etc/sysctl.conf, and then run “sysctl -p”.
Like all operating systems, the default maximum Linux TCP buffer sizes are way too small. I suggest changing them to the following settings:
# increase TCP max buffer size setable using setsockopt()
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
# set max to at least 4MB, or higher if you use very high BDP paths
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
Note: you should leave tcp_mem alone. The defaults are fine.
Another thing you can try that may help increase TCP throughput is to increase the size of the interface queue. To do this, do the following:
ifconfig eth0 txqueuelen 1000
I’ve seen increases in bandwidth of up to 8x by doing this on some long, fast paths. This is only a good idea for Gigabit Ethernet connected hosts, and may have other side effects such as uneven sharing between multiple streams.
Also, I’ve been told that for some network paths, using the Linux ‘tc’ (traffic control) system to pace traffic out of the host can help improve total throughput.
________________________________________
Linux 2.4
Starting with Linux 2.4, Linux has implemented a sender-side autotuning mechanism, so that setting the opitimal buffer size on the sender is not needed. This assumes you have set large buffers on the recieve side, as the sending buffer will not grow beyond the size of the recieve buffer.
However, Linux 2.4 has some other strange behavior that one needs to be aware of. For example: The value for ssthresh for a given path is cached in the routing table. This means that if a connection has has a retransmition and reduces its window, then all connections to that host for the next 10 minutes will use a reduced window size, and not even try to increase its window. The only way to disable this behavior is to do the following before all new connections (you must be root):
sysctl -w net.ipv4.route.flush=1
More information on various tuning parameters for Linux 2.4 are available in the Ipsysctl tutorial .
________________________________________
Linux 2.6
Starting in Linux 2.6.7 (and back-ported to 2.4.27), linux includes alternative congestion control algoritms beside the traditional ‘reno’ algorithm. These are designed to recover quickly from packet loss on high-speed WANs.
Linux 2.6 also includes and both send and receiver-side automatic buffer tuning (up to the maximum sizes specified above). There is also a setting to fix the ssthresh caching weirdness described above.
There are a couple additional sysctl settings for 2.6:
# don’t cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
# recommended to increase this for 1000 BT or higher
net.core.netdev_max_backlog = 2500
# for 10 GigE, use this
# net.core.netdev_max_backlog = 30000
Starting with version 2.6.13, Linux supports pluggable congestion control algorithms . The congestion control algorithm used is set using the sysctl variable net.ipv4.tcp_congestion_control, which is set to cubic or reno by default, depending on which version of the 2.6 kernel you are using.
To get a list of congestion control algorithms that are available in your kernel, run:
sysctl net.ipv4.tcp_available_congestion_control
The choice of congestion control options is selected when you build the kernel. The following are some of the options are available in the 2.6.23 kernel:
• reno: Traditional TCP used by almost all other OSes. (default)
• cubic: CUBIC-TCP
• bic: BIC-TCP
• htcp: Hamilton TCP
• vegas: TCP Vegas
• westwood: optimized for lossy networks
For very long fast paths, I suggest trying cubic or htcp if reno is not is not performing as desired. To set this, do the following:

sysctl -w net.ipv4.tcp_congestion_control=htcp
More information on each of these algorithms and some results can be found here .
More information on tuning parameters and defaults for Linux 2.6 are available in the file ip-sysctl.txt, which is part of the 2.6 source distribution.
And finally a warning for both 2.4 and 2.6: for very large BDP paths where the TCP window is > 20 MB, you are likely to hit the Linux SACK implementation problem. If Linux has too many packets in flight when it gets a SACK event, it takes too long to located the SACKed packet, and you get a TCP timeout and CWND goes back to 1 packet. Restricting the TCP buffer size to about 12 MB seems to avoid this problem, but clearly limits your total throughput. Another solution is to disable SACK.
________________________________________
Linux 2.2
If you are still running Linux 2.2, upgrade! If this is not possible, add the following to /etc/rc.d/rc.local
echo 8388608 > /proc/sys/net/core/wmem_max
echo 8388608 > /proc/sys/net/core/rmem_max
echo 65536 > /proc/sys/net/core/rmem_default
echo 65536 > /proc/sys/net/core/wmem_default

Network Monitoring Tools

http://acs.lbl.gov/NCS/download/