Archive for January, 2010

« Older Entries

Proposed IT subnetting Infrastructure

January 24th, 2010

Proposed IT subnetting Infrastructure 

  1. For External communication we will use Cisco Router(Cisco ASA5520 firewall)
  2. For internal communication we will use managed switch Dell Power Connect 6248

Why use a Managed Switch?

  • Limit broadcast traffic and increase security using VLANs
  • Remove traffic bottlenecks using port trunking
  • Guarantee bandwidth for time-sensitive voice and video traffic using Traffic Prioritisation
  • We will manage switch for internal communication. I mean to say when one server (Test5) wants to talk to another server (Test) at that time it will use manager switch for the internal communication. It will not forward the traffic to the router.

Switching Services

Unlike bridges that use software to create and manage a filter table, switches use application specific integrated circuits (ASICs) to build and maintain their filter tables. But it’s still okay to think of a layer 2 switch as a multiport bridge because their basic reason for being is the same: to break up collision domains.

Layer 2 switches are faster than routers because they don’t take up time looking at the Network layer header information. Instead, they look at the frame’s hardware addresses before deciding to either forward the frame or drop it. 

Switches create private dedicated collision domains and provide independent bandwidth on each port, unlike hubs. Figure 1.1 shows five hosts connected to a switch—all running 10Mbps half duplex to the server. Unlike a hub, each host has 10Mbps dedicated communication to the server.

nnetwork
FIGURE 1.1 Switches create private domains.

Three Switch Functions at Layer 2 

  1.  
    1. Address learning
    2. Forward/filter decisions
    3. Loop avoidance 

Here’s a list of the basic tasks we’ll be done in the switch Dell Power Connect 6248 

  1. Setting the passwords
  2. Setting the hostname
  3. To configure the switch with different VLANs and other network functions

            Add VLAN and Description

            Configuring the IP address and subnet mask & gateway address

            Assigning switch ports to VLANs.

            Routing between VLANs

  1. Hosts in a VLAN live in their own broadcast domain and can communicate freely. VLANs create network partitioning and traffic separation at layer 2 of the OSI, and as I said when I told you why we still need routers, if you want hosts or any other IP-addressable device to communicate between VLANs, a layer 3 device is absolutely necessary.

What we see in Figure 1.2 is that each router interface is plugged into an access link. This means that each of the routers’ interface IP addresses would then become the default gateway address for each host in each VLAN.

FIGURE 1.2 Router with individual VLAN associations

nnetwork

  1. Router connecting three VLANs together for inter-VLAN communication, one interface for each VLAN.
  2. Remember that a created VLAN is unused until it is assigned to a switch port or ports, and that all ports are always assigned in VLAN 1 unless set otherwise. 

Configuring Inter-VLAN
nnetwork
The first thing we need to do here is figure out which subnets are being used. By looking at the router configuration in the figure, you can see that we’re using 192.168.1.0 with DMZ, 192.168.1.32 with Corporate and 192.168.1.64 with Production. And by looking at the switch configuration, you can see that ports 1,2 and 3 etc are in DMZ, and port 4,5 and 6 etc is in Corporate and port 7 and 8 etc is in Production. This means that Hosts on DMZ are in VLAN 1, hosts in Corporate are in VLAN 2 and hosts on Production are in VLAN 3. Here’s what the hosts’ IP addresses should be: 

DMZ: 192.168.1.0, 255.255.255.224, default gateway 192.168.1.30
Corporate: 192.168.1.32, 255.255.255.224, default gateway 192.168.1.62
Production: 192.168.1.64, 255.255.255.224, default gateway 192.168.1.94 

Inter-VLAN example
nnetwork
Complete Working Flow Diagram

 

 

 

nnetwork

Thanks
Manoj Chauhan

View Comments »

Posted in Cisco, Network, Network Performance

Tags:

SSL Configuration in Apache

January 24th, 2010

Secure Sockets Layer (SSL) enables the HTTP protocol to be secured. This page will show you how to configure SSL in Apache and SquirrelMail.

Generate a Private Key

Make sure you are logged in as the root user when doing steps below.

1. Generate a pass phrase protected private key using the command below. Provide a pass phrase when asked.
2. #openssl genrsa -des3 -out localhost.key 1024
3. Remove the pass phrase protection using the command below. Provide the pass phrase when asked.
4. #openssl rsa -in localhost.key -out localhost.key
5. Type in the command below to ensure that the private key will be readable by the root user only.
6.#chmod 400 localhost.key

Generate a Certificate

Generate a certificate signing request by typing in the command below and filling in your host information.

#openssl req -new -key localhost.key -out localhost.csr

To self sign your certificate request, type in the command below.
#openssl x509 -req -days 365 -in localhost.csr -signkey localhost.key -out localhost.crt

Configuring Apache for SSL

Move the file localhost.key into /etc/pki/tls/private/

Place the certificate file into /etc/pki/tls/certs/ and name the file as localhost.crt. The command below applies to self-signed certificate only.

mv localhost.crt /etc/pki/tls/certs/

Edit the file /etc/httpd/conf.d/ssl.conf and edit the lines below.

DocumentRoot = /usr/share/squirrelmail
ServerName = mail.acme.local:443

Restart web server

Thanks
Manoj Chauhan

View Comments »

Posted in Apache, Security

Tags:

SQL Injection Attacks

January 24th, 2010

In this article I’ll attempt to shed some light on this under-documented attack, explaining what an SQL injection attack is and how you can prevent one from occurring within your company. By the end of this article you’ll be able to identify situations where an SQL injection attack may allow unauthorized persons to penetrate your system, and you’ll learn ways to fix existing code to prevent an SQL injection attack.

 

 

What is an SQL Injection Attack?

As you may know, SQL stands for Structured Query Language. It comes in many different dialects, most of which are based on the SQL-92 ANSI standard. An SQL query comprises one or more SQL commands, such as SELECT, UPDATE or INSERT. For SELECT queries, each query typically has a clause by which it returns data, for example:

 

SELECT * FROM Users WHERE userName = ‘justin’;

 

The clause in the SQL query above is WHERE username = ‘justin’, meaning that we only want the rows from the Users table returned where the userName field is equal to the string value of Justin.

 

It’s these types of queries that make the SQL language so popular and flexible… it’s also what makes it open to SQL injection attacks. As the name suggests, an SQL injection attack “injects” or manipulates SQL code. By adding unexpected SQL to a query, it is possible to manipulate a database in many unanticipated ways.

 

 

One of the most popular ways to validate a user on a Website is to provide them with an HTML form through which they can enter their username and password. Let’s assume that we have the following simple HTML form:

 

<form action=”login.asp” method=”post”>

Username: <input>

Password: <input>

<input>

</form>

 

When the form is submitted, the contents of the username and password fields are passed to the login.asp script, and are available to that script through the Request.Form collection. The easiest way to validate this user would be to build an SQL query, and then check that query against the database to see whether that user exists. We could create a login.asp script like this:

 

 

<%

 

dim userName, password, query

dim conn, rS

 

userName = Request.Form(“userName”)

password = Request.Form(“password”)

 

set conn = server.createObject(“ADODB.Connection”)

set rs = server.createObject(“ADODB.Recordset”)

 

query = “select count(*) from users where userName=’” & 

userName & “‘ and userPass=’” & password & “‘”

 

conn.Open “Provider=SQLOLEDB; Data Source=(local); 

Initial Catalog=myDB; User; Password=”

rs.activeConnection = conn

rs.open query

 

if not rs.eof then

response.write “Logged In”

else

response.write “Bad Credentials”

end if

 

%>

 

In the example above, the user either sees “Logged In” if their credentials matched a record in the database, or “Bad Credentials” if they didn’t. Before we continue, let’s create the database that we have queried in the sample code.

 

 

Let’s also create a users table with some dummy records:

 

create database myDB

go

 

use myDB

go

 

create table users

(

userId int identity(1,1) not null,

userName varchar(50) not null,

userPass varchar(20) not null

)

 

insert into users(userName, userPass) values(‘john’, ‘doe’)

insert into users(userName, userPass) values(‘admin’, ‘wwz04ff’)

insert into users(userName, userPass) values(‘fsmith’, ‘mypassword’)

 

So if I entered a username of john and password of doe, then I would be presented with the text “Logged In”. The query would look something like this:

 

 

select count(*) from users where userName=’john’ and userPass=’doe’

 

There’s nothing insecure or dangerous about this query… is there? Maybe not at first glance, but what about if I entered a username of john and a password of ‘ or 1=1 –

 

The resultant query would now look like this:

 

select count(*) from users where userName=’john’ and userPass=” 

or 1=1 –’

 

In the example above I’ve italicised the username and password so they are a bit easier to read, but basically what happens is that the query now only checks for any user with a username field of john. Instead of checking for a matching password, it now checks for an empty password, or the conditional equation of 1=1. This means that if the password field is empty OR 1 equals 1 (which it does), then a valid row has been found in the users table. Notice how the last quote is commented out with a single-line comment delimiter (–). This stops ASP from returning an error about any unclosed quotations.

 

So with the login.asp script we created above, one row would be returned, and the text “Logged In” would be displayed. We could take this a bit further by doing the same thing to the username field, like this:

 

Username: ‘ or 1=1 —

Password: [Empty]

 

This would execute the following query against the users table:

 

select count(*) from users where userName=” or 1=1 –’ and userPass=”

 

The query above now returns a count of all rows in the user table. This is the perfect example of an SQL injection attack: adding code that manipulates the contents of a query to perform an undesired result.

 

Another popular way to validate a user against a table of logins is to compare their details against the table, and retrieve the valid username from the database, like this:

 

query = “select userName from users where userName=’” & 

userName & “‘ and userPass=’” & password & “‘”

 

conn.Open “Provider=SQLOLEDB; Data Source=(local); 

Initial Catalog=myDB; User; Password=”

rs.activeConnection = conn

rs.open query

 

if not rs.eof then

response.write “Logged In As ” & rs.fields(0).value

else

response.write “Bad Credentials”

end if

 

So, if we entered a username of john and a password of doe, then we would be presented with:

 

Logged In As john

 

However, if we used the following login credentials:

 

Username: ‘ or 1=1 —

Password: [Anything]

 

Then we would also be logged in as John, because the row whose username field is John comes first in the list, based on the insert queries we saw earlier:

 

insert into users(userName, userPass) values(‘john’, ‘doe’)

insert into users(userName, userPass) values(‘admin’, ‘wwz04ff’)

insert into users(userName, userPass) values(‘fsmith’, ‘mypassword’)

 

 

Injection Attack Examples

Forcing a login through a HTML form like the one we just saw on is a typical example of an SQL injection attack, and we’ll look at ways to fix these types of attacks a little later.

 

But first, I want to take a look at some examples of SQL injection attack executions. First of, let’s stick with our example login form, which contains a username and password field.

 

Example #1

 

Microsoft SQL Server has its own dialect of SQL, which is called Transact SQL, or TSQL for short. We can exploit the power of TSQL in a number of ways to show how SQL injection attacks work. Consider the following query, which is based on the users table we created on the last page:

 

select userName from users where userName=” having 1=1

 

If you’re an SQL buff, then you’ll no doubt be aware that this query raises an error. We can easily make our login.asp page query our database with this query by using these login credentials:

 

 

Username: ‘ having 1=1 —

 

Password: [Anything]

 

When I click on the submit button to start the login process, the SQL query causes ASP to spit the following error to the browser:

 

Microsoft OLE DB Provider for SQL Server (0x80040E14)

 

Column ‘users.userName’ is invalid in the select list because it is not contained in an aggregate function and there is no GROUP BY clause.

 

/login.asp, line 16

 

Well well. It appears that this error message now tells the unauthorized user the name of one field from the database that we were trying to validate the login credentials against: users.userName. Using the name of this field, we can now use SQL Server’s LIKE keyword to login with the following credentials:

 

Username: ‘ or users.userName like ‘a%’ —

Password: [Anything]

 

Once again, this performs an injected SQL query against our users table:

 

select userName from users where userName=” or 

users.userName like ‘a%’ –’ and userPass=”

 

When we created the users table, we also created a user whose userName field was admin and userPass field was wwz04ff. Logging in with the username and password shown above uses SQL’s like keyword to get the username. The query grabs the userName field of the first row whose userName field starts with a, which in this case is admin:

 

Logged In As admin

 

Example #2

 

SQL Server, among other databases, delimits queries with a semi-colon. The use of a semi-colon allows multiple queries to be submitted as one batch and executed sequentially, for example:

 

select 1; select 1+2; select 1+3;

 

…would return three recordsets. The first would contain the value 1, the second the value 3, and the third the value 4, etc. So, if we logged in with the following credentials:

 

Username: ‘ or 1=1; drop table users; –

Password: [Anything]

 

Then the query would execute in two parts. Firstly, it would select the userName field for all rows in the users table. Secondly, it would delete the users table, so that when we went to login next time, we would see the following error:

 

Microsoft OLE DB Provider for SQL Server (0x80040E37)

Invalid object name ‘users’.

/login.asp, line 16

 

Example #3

 

The last example relating to our login form that we’ll consider is the execution of TSQL specific commands and extended stored procedures. Many Websites use the default system account (sa) user when logging into SQL Server from their ASP scripts or applications. By default, this user has access to all commands and can delete, rename, and add databases, tables, triggers, and more.

 

One of SQL Server’s most powerful commands is SHUTDOWN WITH NOWAIT, which causes SQL Server to shutdown, immediately stopping the Windows service. To restart SQL server after this command is issued, you need to use the SQL service manager or some other method of restarting SQL server.

 

Once again, this command can be exploited through our login example:

 

Username: ‘; shutdown with nowait; –

Password: [Anything]

 

This would make our login.asp script run the following query:

 

select userName from users where userName=”; 

shutdown with nowait; –’ and userPass=”

 

If the user is set up as the default sa account, or the user has the required privileges, then SQL server will shut down, and will require a restart before it will function again.

 

SQL Server also includes several extended stored procedures, which are basically special C++ DLL’s that can contain powerful C/C++ code to manipulate the server, read directories and the registries, delete files, run the command prompt, etc. All extended stored procedures exist under the master database and are prefixed with “xp_”.

 

There are several extended stored procedures that can cause permanent damage to a system. We can execute an extended stored procedure using our login form with an injected command as the username, like this:

 

Username: ‘; exec master..xp_xxx; –

Password: [Anything]

 

All we have to do is pick the appropriate extended stored procedure and replace xp_xxx with its name in the sample above. For example, if IIS was installed on the same machine as SQL Server (which is typical for small one/two man setups), then we could restart it by using the xp_cmdshell extended stored procedure (which executes a command string as an operating-system command) and IIS reset. All we need to do is enter the following user credentials into our getlogin.asp page:

 

Username: ‘; exec master..xp_cmdshell ‘iisreset’; –

Password: [Anything]

 

This would send the following query to SQL Server:

 

select userName from users where userName=”; 

exec master..xp_cmdshell ‘iisreset’; –’ and userPass=”

 

As I’m sure you’ll agree, this can cause serious problems, and with the right commands, can cause an entire Website to malfunction.

 

Example #4

 

OK, time to move away from looking at the login.asp script and onto another common method to perform an SQL injection attack.

 

How many times have you been to a Website that sells you favourite gear and seen a URL like this:

 

 

www.mysite.com/products.asp?productId=2

 

Obviously the 2 is the ID of the product, and a lot of sites would simply build a query around the productId querystring variable, like this:

 

Select prodName from products where id = 2

 

Before we continue, let’s assume that we have the following table and rows setup on our SQL server:

 

create table products

(

id int identity(1,1) not null,

prodName varchar(50) not null,

)

 

insert into products(prodName) values(‘Pink Hoola Hoop’)

insert into products(prodName) values(‘Green Soccer Ball’)

insert into products(prodName) values(‘Orange Rocking Chair’)

 

Let’s also assume that we have created the following ASP script, and called it products.asp:

 

<%

 

dim prodId

prodId = Request.QueryString(“productId”)

 

set conn = server.createObject(“ADODB.Connection”)

set rs = server.createObject(“ADODB.Recordset”)

 

query = “select prodName from products where id = ” & prodId

 

conn.Open “Provider=SQLOLEDB; Data Source=(local); 

Initial Catalog=myDB; User; Password=”

rs.activeConnection = conn

rs.open query

 

if not rs.eof then

response.write “Got product ” & rs.fields(“prodName”).value

else

response.write “No product found”

end if
%>

So if we visited products.asp in the browser with the following URL: 
http://localhost/products.asp?productId=1

…we’d see the following line of text in our browser:

Got product Pink Hoola Hoop
Notice that this time around, product.asp returns a field from the recordset based on the field’s name: 

response.write “Got product ” & rs.fields(“prodName”).value  

Although this may seem more secure, it really isn’t, and we can still manipulate the database just as we have in our last three examples. Notice also that this time the WHERE clause of the query is based on a numerical value: 

query = “select prodName from products where id = ” & prodId 

In order for the products.asp page to function correctly, all that’s required is a numerical product Id passed as the productId querystring variable. Getting around this isn’t too much of a problem, however. Consider the following URL to products.asp:http://localhost/products.asp?productId=0%20or%201=1  

Each %20 in the URL represents a URL-encoded space character, so the URL really looks like this: 

http://localhost/products.asp?productId=0 or 1=1
When used in conjunction with products.asp, the query looks like this: 

select prodName from products where id = 0 or 1=1 

Using a bit of know-how and some URL-encoding, we can just as easily pull the name of the products field from the products table: http://localhost/products.asp?productId=0%20having%201=1 

This would produce the following error in the browser:

Microsoft OLE DB Provider for SQL Server (0x80040E14)

Column ‘products.prodName’ is invalid in the select list because it is not contained in an aggregate 
function and there is no GROUP BY clause.
/products.asp, line 13

Now, we can take the name of the products field (products.prodName) and call up the following URL in the browser:

http://localhost/products.asp?productId=0;insert%20into%20products
(prodName)%20values(left(@@version,50))
Here’s the query without the URL-encoded spaces:

http://localhost/products.asp?productId=0;insert into 

products(prodName) values(left(@@version,50)) 

Basically it returns “No product found”, however it also runs an INSERT query on the products table, adding the first 50 characters of SQL server’s @@version variable (which contains the details of SQL Server’s version, build, etc.) as a new record in the products table. 

In a real-life situation, you would obviously have to exploit the products table more than this as it would contain dozens of other fields, however the methods would remain the same. 

To get to the version, it’s now a simple matter of calling up the products.asp page with the value of the latest entry in the products table, like this:

 http://localhost/products.asp?productId=(select%20max(id)

%20from%20products) 

What this query does is grab the ID of the latest row added to the products table using SQL server’s MAX function. The result outputs the new row that contains the SQL server version details: 

Got product Microsoft SQL Server 2000 – 8.00.534 (Intel X86)

This method of injection can be used to perform numerous tasks. However the point of this article was to give tips on how to prevent SQL injection attacks, which is what we will look at next. 

Preventing SQL Injection Attacks

If you design your scripts and applications with care, SQL injection attacks can be avoided most of the time. There are a number of things that we as developers can do to reduce our site’s susceptibility to attack. Here’s a list (in no particular order) of our options: 

Limit User Access 

 The default system account (sa) for SQL server 2000 should never be used because of its unrestricted nature. You should always setup specific accounts for specific purposes. 

For example, if you run a database that lets users of your site view and order products, then you should set up a user called webUser_public that has SELECT rights on the products table, and INSERT rights only on the orders table.

 If you don’t make use of extended stored procedures, or have unused triggers, stored procedures, user-defined functions, etc, then remove them, or move them to an isolated server. Most extremely damaging SQL injection attacks attempt to make use of several extended stored procedures such as xp_cmdshell and xp_grantlogin, so by removing them, you’re theoretically blocking the attack before it can occur. 

Escape Quotes

 As we’ve seen from the examples discussed above, the majority of injection attacks require the user of single quotes to terminate an expression. By using a simple replace function and converting all single quotes to two single quotes, you’re greatly reducing the chance of an injection attack succeeding.

Using ASP, it’s a simple matter of creating a generic replace function that will handle the single quotes automatically, like this:
<%
function stripQuotes(strWords)
stripQuotes = replace(strWords, “‘”, “””)
end function
%>

Now if we use the stripQuotes function in conjunction with our first query for example, then it would go from this:

select count(*) from users where userName=’john’ and userPass=” or 1=1 –’ 

…to this
select count(*) from users where userName=’john” and 

userPass=”’ or 1=1 –’ 

This, in effect, stops the injection attack from taking place, because the clause for the WHERE query now requires both the userName and userPass fields to be valid.

Remove Culprit Characters/Character Sequences

 As we’ve seen in this article, certain characters and character sequences such as ;, –, select, insert and xp_ can be used to perform an SQL injection attack. By removing these characters and character sequences from user input before we build a query, we can help reduce the chance of an injection attack even further. As with the single quote solution, we just need a basic function to handle all of this for us:

<%
function killChars(strWords) 

dim badChars
dim newChars
badChars = array(“select”, “drop”, “;”, “–”, “insert”, 
“delete”, “xp_”)
newChars = strWords
for i = 0 to uBound(badChars)
newChars = replace(newChars, badChars(i), “”)
next
killChars = newChars
end function
%>

Using stripQuotes in combination with killChars greatly removes the chance of any SQL injection attack from succeeding. So if we had the query:

select prodName from products where; xp_cmdshell ‘format 

c: /q /yes ‘; drop database myDB; –

and ran it through stripQuotes and then killChars, it would end up looking like this:

prodName from products where cmdshell ”format c: 

/q /yes ” database myDB

…which is basically useless, and will return no records from the query.

Limit the Length of User Input

It’s no good having a text box on a form that can accept 50 characters if the field you’ll compare it against can only accept 10. By keeping all text boxes and form fields as short as possible, you’re taking away the number of characters that can be used to formulate an SQL injection attack.

If you’re accepting a querystring value for a product ID or the like, always use a function to check if the value is actually numeric, such as the IsNumeric() function for ASP. If the value isn’t numeric, then either raise an error or redirect the user to another page where they can choose a product.

Also, always try to post your forms with the method attribute set to POST, so clued-up users don’t get any ideas — they might if they saw your form variables tacked onto the end of the URL.

Conclusion
In this article we’ve seen what an SQL injection attack is and also how to tamper with forms and URLs to product the results of an attack. 

It’s not always possible to guard against every type of SQL injection attack, however, hopefully you now know about the various types of SQL injection attacks that exist and have also planned ways to combat them on your servers.  Although we’ve only looked at SQL injection attacks with Microsoft SQL server in this article, keep in mind that no database is safe: SQL injection attacks can also occur on MySQL and Oracle database servers — among others. 

SQL Injection Attacks Linger

To experience Web developers, it’s a cardinal sin: passing user input into a database query without first processing it to escaped special characters is bad! But as a beginner-friendly language, PHP is host to this mistake more often than most. If you don’t understand SQL injection attacks, read this!

The danger is easy to explain. Your site’s security relies on a database query that includes one or more values taken from user input (e.g. login credentials). But, because you neglected to encode special characters in those values, an attacker is able to bypass your site’s security by entering carefully-chosen values that alter the meaning of the query.

PHP’s magic quotes feature is designed to save inexperienced coders from themselves. Values in the $_GET, $_POST, $_COOKIE, and $_REQUEST arrays have backslashes added to them so that they are safe for use in database queries. But magic quotes isn’t enabled on all servers, and not all user input comes through the $_GET, $_POST, $_COOKIE, and $_REQUEST arrays, which magic quotes protects.

Essential CSS Hacks

When performing cross-browser page layout with CSS, you have two choices: go with the lowest common denominator, or use CSS hacks to include work-arounds for ornery browsers. The hacks definitely give the best result, but who can remember them all?

When it all boils down, two browsers are responsible for the vast majority of CSS layout incompatibilities: Internet Explorer 5.x for Windows, and Internet Explorer 5.x for Macintosh. With the most recent CSS hacks out there, you can write remedial style sheets for both these browsers and get away with just two CSS hacks!

First of all, you need the Mid Pass Filter for Internet Explorer 5.x for Windows:

<style>
@media tty {
 i{content:”\”;/*” “*/}} @import ‘ie5.css’;
/*”;}
}/* */
</style>

Then, you need the IE5/Mac Band Pass Filter for Internet Explorer 5.x for Macintosh:

<style>
/*\*//*/  @import “ie5mac.css”;/**/
</style>

By taking advantage of various bugs in the CSS processing of these browsers, the above blocks of code ensure that the rules in ie5.css will only affect Internet Explorer 5.x for Windows, and the rules in ie5mac.css will only affect Internet Explorer 5.x for Macintosh. 

By using these two style sheets to work around the browser-specific bugs, you can keep the nice, standards-compliant CSS code destined for other browsers in your main style sheet.

View Comments »

Posted in Security

Tags:

PERL-DATETIME-FORMAT-ICAL RPM PACKAGES

January 24th, 2010

http://dag.wieers.com/rpm/packages/perl-DateTime-Format-ICal/perl-DateTime-Format-ICal-0.08-1.el5.rf.noarch.rpm

rpm -ivh perl-DateTime-Format-Epoch-0.10-1.n0i.2.noarch.rpm –nodeps

yum install perl-DateTime*

View Comments »

Posted in CentOS, Linux

Tags:

Setup NIS Server

January 24th, 2010

NIS Server

Build NIS Server in order to share users’ accounts among virtual networks. Following examples show that NIS Server is built on HostOS in virtual networks like NFS Server. It’s necessary to install ypserv for NIS.
First we need to install the ypserv package, we can install it by using the following command

#yum -y install ypserv
// set domain name
#ypdomainname example.com
// add at the bottom of file
#vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=ns.server-linux.info
GATEWAY=192.168.0.1
NISDOMAIN=example.com

#vi /var/yp/Makefile
# MERGE_PASSWD=true|false
MERGE_PASSWD=false// line 42: change
#
# MERGE_GROUP=true|false
MERGE_GROUP=false// line 46: change
#
all: passwd shadow group hosts rpc services netid protocols   // line 109: add shadow
// create a directory for email automatically when a user is added in the system

[root@ns ~]# mkdir -p /etc/skel/Maildir/cur
[root@ns ~]# mkdir -p /etc/skel/Maildir/new
[root@ns ~]# mkdir -p /etc/skel/Maildir/tmp
[root@ns ~]# chmod -R 700 /etc/skel/Maildir/
[root@ns ~]# useradd cent
[root@ns ~]# passwd cent
Changing password for user cent.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
 [root@ns ~]# /usr/lib/yp/ypinit -m
At this point, we have to construct a list of the hosts which will run NIS servers. ns.server-linux.info is in the list of NIS server hosts. Please continue to add the names for the other hosts, one per line. When you are done with the list, type a <control D>.
next host to add: ns.server-linux.info
next host to add: // push Ctrl + D keys
The current list of NIS servers looks like this:
ns.server-linux.info
Is this correct? [y/n: y] y// input ‘y’ and push Enter key
We need a few minutes to build the databases…
Building /var/yp/server-linux.info/ypservers…
Running /var/yp/Makefile…
gmake[1]: Entering directory `/var/yp/server-linux.info’

Updating passwd.byname…
Updating passwd.byuid…
Updating group.byname…
Updating group.bygid…
Updating hosts.byname…
Updating hosts.byaddr…
Updating rpc.byname…
Updating rpc.bynumber…
Updating services.byname…
Updating services.byservicename…
Updating netid.byname…
Updating protocols.bynumber…
Updating protocols.byname…
Updating mail.aliases…
gmake[1]: Leaving directory `/var/yp/server-linux.info’
ns.server-linux.info has been set up as a NIS master server.
Now you can run ypinit -s ns.server-linux.info on all slave server.
[root@ns ~]# /etc/rc.d/init.d/portmap start
Starting portmap: [  OK  ]
[root@ns ~]# /etc/rc.d/init.d/ypserv start
Starting YP server services: [  OK  ]
[root@ns ~]# /etc/rc.d/init.d/yppasswdd start
Starting YP passwd service: [  OK  ]

[root@ns ~]# chkconfig portmap on
[root@ns ~]# chkconfig ypserv on
[root@ns ~]# chkconfig yppasswdd on
// It’s neccessary to update NIS database with following way if new user is added again
[root@ns ~]# cd /var/yp
[root@ns yp]# make
After building NIS Server, Configure on clients in order to share users’ accounts. Following examples show config on GuestOS ‘www’.

[root@www ~]# vi /etc/sysconfig/network
 
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=www.server-linux.info
GATEWAY=192.168.0.1
NISDOMAIN=server-linux.info// add the line
 
[root@www ~]# vi /etc/sysconfig/authconfig
 
USEWINBINDAUTH=no
USEKERBEROS=no
USESYSNETAUTH=no
FORCESMARTCARD=no
USESMBAUTH=no
USESMARTCARD=no
USELDAPAUTH=no
USELOCAUTHORIZE=no
USEWINBIND=no
USESHADOW=yes
USEDB=no
USEMD5=yes
USEPASSWDQC=no
USELDAP=no
USEHESIOD=no
USECRACKLIB=yes
USENIS=yes// change
 
[root@www ~]# vi /etc/yp.conf
 
# Valid entries are
#
# domain NISDOMAIN server HOSTNAME
#Use server HOSTNAME for the domain NISDOMAIN.
#
# domain NISDOMAIN broadcast
#Use broadcast on the local net for domain NISDOMAIN
#
# domain NISDOMAIN slp
#Query local SLP server for ypserver supporting NISDOMAIN
#
# ypserver HOSTNAME
#Use server HOSTNAME for the local domain. The
#IP-address of server must be listed in /etc/hosts.
#
# broadcast
#If no server for the default domain is specified or
#none of them is rechable, try a broadcast call to
#find a server.
#
domain server-linux.info server nfs.server-linux.info  // add the line
 
[root@www ~]# vi /etc/nsswitch.conf
 
passwd:files nis// line 33: add
shadow:files nis// add
group:files nis// add
 
#hosts:db files nisplus nis dns
hosts:files dns nis// add
 
[root@www ~]# chkconfig ypbind on
[root@www ~]# chkconfig portmap on
[root@www ~]# reboot
 
www login: cent// user name on NIS
Password:// input password
Last login: Sun Mar 11 22:02:12 on tty1
[cent@www ~]$// could login
[cent@www ~]$ ypwhich
nfs.server-linux.info
[cent@www ~]$ ypcat passwd
cent:x:500:500::/home/cent:/bin/bash
[cent@www ~]$ yppasswd// change password
Changing NIS account information for cent on nfs.server-linux.info.
Please enter old password:// input current password
Changing NIS password for cent on nfs.server-linux.info.
Please enter new password:// input new password
Please retype new password:// verify
 
The NIS password has been changed on nfs.server-linux.info.

View Comments »

Posted in CentOS, Linux

Nagios Architecture

January 24th, 2010

Overview 

Nagios is a host and service monitor designed to inform you of network problems before your clients, end-users or managers do. It has been designed to run under the Linux operating system, but works fine under most *NIX variants as well. The monitoring daemon runs intermittent checks on hosts and services you specify using external “plugins” which return status information to Nagios. When problems are encountered, the daemon can send notifications out to administrative contacts in a variety of different ways (email, instant message, SMS, etc.). Current status information, historical logs, and reports can all be accessed via a web browser.

  Architecture

Nagios is built on a server/agents architecture. Usually, on a network, a Nagios server is running on a host, and plugins are running on all the remote hosts that need to be monitored. These plugins send information to the server, which displays them in a GUI.

nagios

Nagios is composed of three parts: 

1) A scheduler: this is the server part of Nagios. At regular interval, the scheduler checks the plugins, and according to their results do some actions.

2) A GUI: the interface of Nagios (with the configuration, the alerts, …). It is displayed in web pages generated by CGI.It can be state buttons (green,OK/red,Error), sounds, MRTG graphs, …

3) The plugins. They are configurable by the user. They check a service and return a result to the Nagios server.

 A soft alert is raised when a plugin returns a warning or an error. Then on the GUI, a green button turns to red, and a sound is emitted. When this soft alert is raised many times (the number is configurable), a hard alert is raised, and the Nagios server sends notifications: email, SMS… 

Nagios Architecture (internal)
nagios

nagios
 
 

 

Nagios functionalities

 Nagios® is an open source tool specially developed to monitor host and service and designed to inform you of network incidents before your clients, end-users or managers do. It has been designed to run under the Linux operating system, but works fine under most *NIX variants as well initially developed for servers and application monitoring, it is now widely used to monitor networks availability. It is possible with the development of specific plugins around Nagios process. Nagios works with a set of “plugins” to provide local and remote service status. The monitoring daemon runs intermittent checks on hosts and services you specify using external “plugins” which return status information to Nagios.  When incidents are detected, the daemon send notifications out to administrative contacts in a variety of different ways (email, instant message, SMS, etc.). Current status information, historical logs, and reports can all be accessed via a Web browser.Custom “plugins” are relatively easy to develop Different methods are provided for remote resource discovery Nagios is freely available from http://www.nagios.org

Requirements

Other things you will need to get Nagios working are:

1) Nagios Plugins (from Nagios download URL)

2) GD – Graphics Libraries

3) JPEG Lib Sources

4) PNG Lib Sources

5) FPing (Fast Ping), this is optional but useful.

6) For SNMP monitoring you will need:

7) net-snmp-tools, and

8 )  net-snmp-utils

9) MySQL database for storing: Elements status logs

Plugins and Extensions

Developments on Nagios can be found at http://www.nagiosexchange.org/

Add-On projects are freely available. They cover subjects on:

1) Charts,

2) Communications,

3) Configuration,

4) Development,

5) Downtimes,

6) FrontEnds,

7) Notifications,

8 )  Misc

Plugins have been developed on:

1) Networking,

2) SNMP,

3) Hardware,

4) Linux,

5) Solaris,

6) Windows, … 

PLUGINS
 
 
 

 

1) A plugin is a small program (in Perl, C, java, python …) that checks a service (a daemon, some free space on a disk …). It must return a value and a small line of text (Nagios will only grab the first line of text). Output should be in the format: METRIC STATUS: information text performance data The allowed METRIC STATUS are 0 (OK), 1 (WARNING), 2 (CRITICAL) or 3 (UNKNOWN) 

2) The warning and critical thresholds are parameters, set by the user, passed as arguments to the plugin.

 3) A plugin can also return performance data in the format: “label1=value1 label2=value2 …”
These data are stored by Nagios and may be later displayed with MRTG (http://people.ee.ethz.ch/~oetiker/webtools/mrtg/)

The plugins can be run:
1) Locally, on the Nagios server. But such a plugin can check remote hosts, for example check_ping which pings remote hosts to check if they are running.
 
 

 

2) Remotely, through a remote Nagios server, with ssh, with snmp, with NRPE (Nagios Remote Plugin Executor), or with NSCA (Nagios Service Check Acceptor). It means that the plugin either waits for a verification request from the Nagios server before sending its result, or executes itself and sends the result to the Nagios server. 

Other useful developments

 Alarm resiliency

1) Nagios gives an immediate status of the monitored elements, it has no memory (except in log). It is useful to keep trace of an  incident until it has been checked and acknowledged by an operator.

 Network Interfaces discovery

1) Within big networks, it is useful to « compare » real configuration with database configuration. An external program can check every day (auto-discovery) the real network configuration versus Nagios database.
2) If differences appear, notify network administrator of the change.

 

Semi-automatic configuration
1) For each new element, multiple identified checking have to be configured and started
2) Semi-automatic configuration tool will write Nagios configuration files based on higher level network description files
 
 

 

References

1) Nagios source program
 http://www.nagios.org/download/

2) Nagios Extra developments
 http://www.nagiosexchange.org/

3) Official plugins
 http://nagiosplug.sourceforge.net/

4) Conferences
 http://www.nagios.org/propaganda/conferences/

View Comments »

Posted in Nagios

Jopr Monitoring Tool – Installation

January 24th, 2010

1. Please installed the postgresql by using the following command, the general installation. As root install postgres core:

yum install postgresql*

2. Now create postgres user:
adduser postgres

3. Create the datafile for the database:
mkdir -p /usr/local/pgsql/data

4. Change ownership of the data files to the postgres user:
chown postgres /usr/local/pgsql/data

5. Now assume the role of a postgres user:
su – postgres

6. Important note: Installation of the postgres executables on Centos 5 / RHEL 5 is /usr/bin not /usr/local as Postgres official documentation suggests. Initialize the datafiles for the database:
/usr/bin/initdb -D /usr/local/pgsql/data

7. Start the database with initialized datafiles as the background process (&) and log all messages and errors (2&1) in the logfile:
/usr/bin/postgres -D /usr/local/pgsql/data > logfile 2>&1 &

8. Create the test database:
/usr/bin/createdb rhq

Install a database like PostgreSQL (version 8.2.4 +, please choose C locale for initdb), create a database instance called ‘rhq’ in it and a user ‘rhqadmin’ that owns this ‘rhq’ database.
#create user rhqadmin password ‘rhqadmin’;
#create database rhq owner rhqadmin;

9. Log in to the rhq database:
/usr/bin/psql rhq

10. You should see “Welcome to Postgres 8…” intro message and prompt:
rhq=#

11. Now make the following changes in this file
vi /var/lib/pgsql/data/pg_hba.conf
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

# “local” is for Unix domain socket connections only
local   all         all                               trust #sameuser
# IPv4 local connections:
host    all         all         127.0.0.1/32          trust #sameuser
# IPv6 local connections:
host    all         all         ::1/128               trust #sameuser

12. Restart the postgresql service
/etc/init.d/postgresql restart

13. Test the newly created database i.e. rhq
#psql -d rhq -U rhqadmin -W (-d = databasename, -U = username, -W=password)

14. Unzip the Jopr server
#unzip jopr-server-2.1.0.GA.zip

15. Set the java path or Eventually set RHQ_SERVER_JAVA_HOME or RHQ_SERVER_JAVA_EXE_FILE_PATH env variables appropriately.
#set RHQ_SERVER_JAVA_HOME=/usr/java/jdk1.6.0_11/
OR
#export RHQ_SERVER_JAVA_HOME=/usr/java/jdk1.6.0_11/

16. cd into the jopr-server-*/ directory and start the server:
bin/rhq-server.sh console

17. After a few seconds, the messages on the console will stop. When this happens, point your browser to http://localhost:7080/ and run the installer.

18. When this is done and you can log in (default user / pass are: rhqadmin/rhqadmin), it is time to start the agent. Start a second shell for this.

19. cd into the agent directory.

20. start the agent by issuing
bin/rhq-agent.sh
(Answer the questions asked by the agent.)

21. Go back to the Server GUI and to its Dashboard. Watch resources show up in the upper right in the Autodiscovery portlet and import them.

22. Write script for agent and server
Start-agent
export RHQ_AGENT_JAVA_HOME=/usr/java/jdk1.6.0_11/
nice /usr/jopr-agent-2.1.0.GA/bin/rhq-agent.sh &

Start-jopr
export RHQ_SERVER_JAVA_HOME=/usr/java/jdk1.6.0_11/
nice /usr/jopr-server-2.1.0.GA/bin/rhq-server.sh console &

22. Done :-)  
 
http://172.16.0.16:7080/installer/start.jsf
psql -U rhqadmin -d rhq -W

View Comments »

Posted in Monitoring

Installing SSL Certificate – Apache 2.x

January 24th, 2010

Once your SSL certificate has been signed and issued,Go Daddy® will send you an e-mail message that allows you to download the signed certificate and our intermediate certificate bundle, both of which must be installed on your Web site.

Note: You must use the provided certificate-download link within three days of receiving the certificate-issuance e-mail message. If the download link is allowed to expire, you must request a certificate re-key in order to retrieve your signed SSL certificate.

Follow the instructions below to download and install an SSL certificate on your Web server.

Note: Before you install your issued SSL certificate you must install our intermediate certificate bundle (null) on your Web server. You may also download the intermediate certificate bundle from the repository. 

Once your SSL certificate has been signed and issued,Go Daddy® will send you an e-mail message that allows you to download the signed certificate and our intermediate certificate bundle, both of which must be installed on your Web site.

Note: You must use the provided certificate-download link within three days of receiving the certificate-issuance e-mail message. If the download link is allowed to expire, you must request a certificate re-key in order to retrieve your signed SSL certificate.

Follow the instructions below to download and install an SSL certificate on your Web server.

Note: Before you install your issued SSL certificate you must install our intermediate certificate bundle (null) on your Web server. You may also download the intermediate certificate bundle from the repository.

Installing SSL Certificate and the Intermediate Certificate
i. Copy your issued certificate, intermediate certificate and key file (generated when you created the Certificate Signing Request (CSR)) into the directory that you will be using to hold your certificates.
ii. Open the Apache ssl.conf file and add the following directives:
  o SSLCertificateFile /path to certificate file/your issued certificate
  o SSLCertificateKeyFile /path to key file/your key file
  o SSLCertificateChainFile /path to intermediate certificate/null
iii. Save your ssl.conf file and restart Apache.

Restarting Your Web Server
You may use SSH to access your server. If you are using Windows on your computer, you can use the free, downloadable putty SSH/Telnet client to access your Linux server.

To restart your server:
i. Start Putty
ii. Select the “SSH” protocol option; then type in your IP address in the “Host Name” field.
iii. Click “Open.” This will connect you to your server.
iv. Log in as “root”; or enter “su:root”
v. Once logged in, you can restart the Web server by entering the following command: /usr/local/sbin/apachectl graceful

Note: Reissued certificates are not delivered via e-mail. Instead, the reissued certificate is displayed on the certificate-reissuance page of this site. In order to enable installation, please cut/copy the contents of the reissued certificate into a text file and save it. Once you have created the certificate file you may go ahead and install the reissued certificate.

View Comments »

Posted in Apache, Security

Installing Lighttpd With PHP5 And MySQL Support On CentOS 5.0

January 24th, 2010

Installing Lighttpd With PHP5 And MySQL Support On CentOS 5.0

 Lighttpd is a secure, fast, standards-compliant web server designed for speed-critical environments. This tutorial shows how you can install Lighttpd on a CentOS 5.0 server with PHP5 support (through FastCGI) and MySQL support.

 

In this tutorial I use the hostname server1.example.com with the IP address 192.168.0.100. These settings might differ for you, so you have to replace them where appropriate.

2 Installing MySQL 5.0

 

First we install MySQL 5.0 like this:

yum install mysql mysql-server

Then we create the system startup links for MySQL (so that MySQL starts automatically whenever the system boots) and start the MySQL server:

chkconfig –levels 235 mysqld on
/etc/init.d/mysqld start

Create a password for the MySQL user root (replace yourrootsqlpassword with the password you want to use):

mysqladmin -u root password yourrootsqlpassword

Then check with

netstat -tap | grep mysql

on which addresses MySQL is listening. If the output looks like this:

tcp        0      0 localhost.localdo:mysql *:*                     LISTEN     2713/mysqld

which means MySQL is listening on localhost.localdomain only, then you’re safe with the password you set before. But if the output looks like this:

tcp        0      0 *:mysql *:*                     LISTEN     2713/mysqld
you should set a MySQL password for your hostname, too, because otherwise anybody can access your database and modify data:

mysqladmin -h server1.example.com -u root password yourrootsqlpassword

3 Installing Lighttpd

Lighttpd is not available from the official CentOS 5.0 repositories, but from the RPMforge repositories (see http://dag.wieers.com/rpm/FAQ.php#B2 for instructions). We install the RPMforge package for RHEL 5 which works for CentOS 5.0 as well:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

Afterwards, we can install Lighttpd like this:

yum install lighttpd

Then we create the system startup links for Lighttpd (so that Lighttpd starts automatically whenever the system boots) and start it:

chkconfig –levels 235 lighttpd on
/etc/init.d/lighttpd start

Now direct your browser to http://192.168.0.100, and you should see the Lighttpd placeholder page:

Lighttpd’s default document root is /srv/www/lighttpd on CentOS 5.0, and the configuration file is /etc/lighttpd/lighttpd.conf.

 

4 Installing PHP5

We can make PHP5 work in Lighttpd through FastCGI. Therefore we install the packages lighttpd-fastcgi and php-cli:

yum install lighttpd-fastcgi php-cli

 

5 Configuring Lighttpd And PHP5

To enable PHP5 in Lighttpd, we must modify two files, /etc/php.ini and /etc/lighttpd/lighttpd.conf. First we open /etc/php.ini and add the line cgi.fix_pathinfo = 1 right at the end of the file:

vi /etc/php.ini

[...]
cgi.fix_pathinfo = 1

Then we open /etc/lighttpd/lighttpd.conf and uncomment “mod_fastcgi”, in the server.modules stanza:

vi /etc/lighttpd/lighttpd.conf

[...]
server.modules              = (
#                               "mod_rewrite",
#                               "mod_redirect",
#                               "mod_alias",
                                "mod_access",
#                               "mod_cml",
#                               "mod_trigger_b4_dl",
#                               "mod_auth",
#                               "mod_status",
#                               "mod_setenv",
                                "mod_fastcgi",
#                               "mod_proxy",
#                               "mod_simple_vhost",
#                               "mod_evhost",
#                               "mod_userdir",
#                               "mod_cgi",
#                               "mod_compress",
#                               "mod_ssi",
#                               "mod_usertrack",
#                               "mod_expire",
#                               "mod_secdownload",
#                               "mod_rrdtool",
                                "mod_accesslog" )

 

and then , further down the file, there’s a fastcgi.server stanza which we uncomment as well – make sure you use /usr/bin/php-cgi instead of /usr/local/bin/php in the “bin-path” line::

[...]
#### fastcgi module
## read fastcgi.txt for more info
fastcgi.server             = ( ".php" =>
                               ( "localhost" =>
                                 (
                                   "socket" => "/tmp/php-fastcgi.socket",
                                   "bin-path" => "/usr/bin/php-cgi"
                                 )
                               )
                            )
[…]

 

Then we restart Lighttpd:

/etc/init.d/lighttpd restart

 

6 Testing PHP5 / Getting Details About Your PHP5 Installation

The document root of the default web site is /srv/www/lighttpd. We will now create a small PHP file (info.php) in that directory and call it in a browser. The file will display lots of useful details about our PHP installation, such as the installed PHP version.

vi /srv/www/lighttpd/info.php

<?php
phpinfo();?>

 

Now we call that file in a browser (e.g. http://192.168.0.100/info.php):

 

As you see, PHP5 is working, and it’s working through FastCGI, as shown in the Server API line. If you scroll further down, you will see all modules that are already enabled in PHP5. MySQL is not listed there which means we don’t have MySQL support in PHP5 yet.

7 Getting MySQL Support In PHP5

To get MySQL support in PHP, we can install the php-mysql package. It’s a good idea to install some other PHP5 modules as well as you might need them for your applications. You can search for available PHP5 modules like this:

yum search php

Pick the ones you need and install them like this:

yum install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc

Now restart Lighttpd:

/etc/init.d/lighttpd restart

Now reload http://192.168.0.100/info.php in your browser and scroll down to the modules section again. You should now find lots of new modules there, including the MySQL module:

 

 

To enable SSL for the whole server you have to provide a valid certificate and have to enable the SSL engine. If you want to use chained certificates you must also include the CA file, without it browsers will pop up an unknown certificate authority or some such error.

# cd /etc/lighttpd (go to this directory)
Generate the certificate
#openssl req -new -x509 -keyout server.pem -out server.pem -days 365 –nodes
(This certificate is valid for one year (365 days))

#vim /etc/lighttpd/lighttpd.conf

Locate ssl.engine in the /etc/lighttpd/lighttpd.conf and add the following

#### SSL engine
ssl.engine                 = “enable”
ssl.pemfile                = “/etc/lighttpd/server.pem” (Path of newly created certificate)

To check that your config is ok:
# lighttpd -t -f lighttpd.conf

You need to restart the web server, we can restart it by using the following command
/etc/init.d/lighttpd restart

For more details please check the below URLS

View Comments »

Posted in CentOS

IIS Security Checklist

January 24th, 2010

IIS Security Checklist 

There are several ways to enhance the security of computer publishing information on an intranet or the Internet. If you have concerns about the security of your system, review this checklist to determine if aspects of your security could be improved.

Windows Security

 The security features in IIS are built upon those in Windows. The following settings in Windows will help make your Web site secure.

File System

web_security

Services and Other Issues

web_security

Internet Information Services Security
IIS provides frontline security for your Web site, including authentication and Web permissions

 web_securityWeb Permissions

web_security

Physical Security
web_security
Personnel Security
web_security

View Comments »

Posted in Security

Ganglia Installation steps

January 24th, 2010

1. Download and install latest repository from the following URL:
rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

2. Install the rrdtool by using the following command:
yum install rrdtool*

3. rpm -ivh libconfuse-2.5-4.el5.i386.rpm
4. rpm -ivh libganglia-3_1_0-3.1.1-1.i386.rpm
5. rpm -ivh ganglia-gmond-3.1.1-1.i386.rpm

At Server End:
/etc/ganglia/gmetad.conf

The keyword ‘data_source’ must immediately be followed by a unique
# string which identifies the source, then an optional polling interval in
# seconds. The source will be polled at this interval on average.
# If the polling interval is omitted, 15sec is asssumed.
#
# A list of machines which service the data source follows, in the
# format ip:port, or name:port. If a port is not specified then 8649
# (the default gmond port) is assumed.
# default: There is no default value
#
# data_source “my cluster” 10 localhost my.machine.edu:8649 1.2.3.5:8655
# data_source “my grid” 50 1.3.4.7:8655 grid.org:8651 grid-backup.org:8651

data_source “ONAXER” localhost 192.168.1.100:8649

# The name of this Grid. All the data sources above will be wrapped in a GRID
# tag with this name.
# default: unspecified
gridname “ONAXER first Grid”

# If you want any host which connects to the gmetad XML to receive
# data, then set this value to “on”
# default: off
all_trusted on

# The port gmetad will answer requests for XML
# default: 8651
xml_port 8651

# The port gmetad will answer queries for XML. This facility allows
# simple subtree and summation views of the XML tree.
# default: 8652
interactive_port 8652

# The number of threads answering XML requests
# default: 4
server_threads 10

Client Side:

/* This configuration is as close to 2.5.x default behavior as possible
The values closely match ./gmond/metric.h definitions in 2.5.x */
globals {
daemonize = yes
setuid = yes
user = nobody
debug_level = 0
max_udp_msg_len = 1472
mute = no
deaf = no
host_dmax = 0 /*secs */
cleanup_threshold = 300 /*secs */
gexec = no
send_metadata_interval = 0
}

/* If a cluster attribute is specified, then all gmond hosts are wrapped inside
* of a tag. If you do not specify a cluster tag, then all will
* NOT be wrapped inside of a tag. */
cluster {
name = “ONAXER” (This should be same for all servers which are in the same grid)
#name = “unspecified”
owner = “onaxer”
latlong = “unspecified”
url = “unspecified”
}

/* The host section describes attributes of the host, like the location */
host {
location = “Onaxer” (This can be different for all servers so we can differentiate the servers )
}

/* Feel free to specify as many udp_send_channels as you like. Gmond
used to only support having a single channel */
udp_send_channel {
mcast_join = 192.168.1.10 (Ganglia server IP address)
port = 8649
ttl = 1
}

/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
port = 8649
}

/* You can specify as many tcp_accept_channels as you like to share
an xml description of the state of the cluster */
tcp_accept_channel {
port = 8649
}

/* Each metrics module that is referenced by gmond must be specified and
loaded. If the module has been statically linked with gmond, it does not
require a load path. However all dynamically loadable modules must include
a load path. */

Costumes message

gmetric -n “WOW” -v “It works” -t “string”
Install ganglia Server

1. Download and install latest repository from the following URL:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

Note: Please install php with fill lib. Support

2. Install the rrdtool by using the following command:
yum install rrdtool* , yum install php-* (php 5.x), yum install gcc* and yum install glib*

3. rpm -ivh libconfuse-2.5-4.el5.i386.rpm
4. rpm -ivh libconfuse-devel-2.5-4.el5.i386.rpm
5. rpm -ivh libganglia-3_1_0-3.1.1-1.i386.rpm
6. rpm -ivh ganglia-gmetad-3.1.1-1.i386.rpm
7. yum install apr*
8. rpm -ivh ganglia-devel-3.1.1-1.i386.rpm
9. rpm -ivh ganglia-3.1.1-1.src.rpm
10. rpm -ivh ganglia-debuginfo-3.1.1-1.i386.rpm
11. rpm -ivh ganglia-gmond-3.1.1-1.i386.rpm
12. rpm -ivh ganglia-web-3.1.1-1.noarch.rpm

There was an error collecting ganglia data (127.0.0.1:8652): fsockopen error: Connection refused

#setsebool -P httpd_can_network_connect 1

View Comments »

Posted in CentOS, Monitoring

Mysql Server Backup and Failover

January 24th, 2010

The solution will comprise of two database servers and an application server. At any instant of time one database server would act as active master allowing writes and both database servers will be able to serve reads. Application server will be writing and reading from preconfigured IPs for writes and reads.

mysql_backupMysql Server Backup and Failover

What we get is:
1) Data redundancy leading to data availability on another box in case of failure of one server
2) A secondary server that can be used for Database backups, without application downtime.

Automatic failover solution:

Even if we have the data copy being maintained over another server, MySQL Replication does not provide any mechanism for automatic failover to the next available server. For this we are currently setting up a Test Bed for evaluating a tool from Google, MySQL Multi-master Monitor (MMM).

Currently, we have installed it and are looking into some configuration problem regarding IPs using iproute2

Other solution discussed was getting the server monitoring service being integrated into the application code, which would mean a major change in application. So we are currently focusing on finding and formulating a solution that doesn’t changes the application much.
Other application changes being looked into:

With this solution we will have to introduce a server that will be hosting the files that are to be loaded into the database tables using the LOAD DATA INFILE queries and the database server monitoring service.
Herein application server has the files locally accessible and acts as a client to the database server (that allows writes) and the file contents will be sent over the network. This will mean that we have to assure that we validate all the currently running LOAD DATA INFILE queries to include a keyword “LOCAL”. Sending the contents of the files over the network could well be a performance issue and is being tested for.

Few concerns/ questions:

Will we be able to provide a separate network for communication between the three servers (App, CNS1 and CNS2) thus ensuring that we insulate the outer traffic from this network. This is being asked to see if we could ensure that even if file contents are moved along network, then also we get an optimal performance.

NOTE:

We have already configured and tested Master -2 and Master – slave MySQL Replication. Evaluation of the Automatic Failover Tool and the above mentioned application changes require a thorough testing for which we will be requiring another few days, after which we can only provide any time estimates and final equipment listing.

View Comments »

Posted in Mysql

Mysql Cluster Configuration

January 24th, 2010

Download the cluster rpms from the mysql website given below.
http://dev.mysql.com/downloads/cluster/6.2.html

There you will get the rpms for cluster storage engine,server,client,cluster storage engine management.
Download the rpms as per you setup configurations, i.e download storage engine rpms on storage server and management rpms on management server.

Mysql Cluster SQL node:

TheServer RPM (for example,MySQL-Cluster-gpl-server-6.2.15-0.rhel4.i386.rpm), which supplies the core files needed to run a MySQL Server withNDBCLUSTER storage engine support (that is, as a MySQL Cluster SQL node).
If you do not have your own client application capable of administering a MySQL server, you should also obtain and install theClientRPM (for example,MySQL-Cluster-gpl-client-6.2.15-0.rhel4.i386.rpm).

Cluster Storage Engine:

The Cluster storage engine RPM (for example,MySQL-Cluster-gpl-storage-6.2.15-0.rhel4.i386.rpm), which supplies the MySQL Cluster data node binary (ndbd).

Cluster Storage Engine Management:

The Cluster storage engine management RPM (for example, MySQL-Cluster-gpl-management-6.2.18-0.sles10.i586.rpm or MySQL-Cluster-gpl-management-6.3.24-0.sles10.i586.rpm), which provides the MySQL Cluster management server binary (ndb_mgmd).

In addition, you should also obtain the NDB Cluster – Storage engine basic tools RPM (for example, MySQL-Cluster-gpl-tools-6.2.15-0.rhel4.i386.rpm), which supplies several useful applications for working with a MySQL Cluster. The most important of these is the MySQL Cluster management client (ndb_mgm).

For more information you can refer this article:

http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-multi-install.html

Installation:

SQL node installation — RPM files. On each machine to be used for hosting a cluster SQL node, install theServer RPM by executing the following command as the system root user, replacing the name shown for the RPM as necessary to match the name of the RPM downloaded from the MySQL AB web site:

shell> rpm -Uhv MySQL-Cluster-gpl-server-6.3.24-0.sles10.i586.rpm

This installs the MySQL server binary (mysqld) in the /usr/sbin directory, as well as all needed MySQL Server support files. It also installs the mysql.server and mysqld_safe startup scripts in /usr/share/mysql and/usr/bin, respectively. The RPM installer should take care of general configuration issues (such as creating themysql user and group, if needed) automatically.

To administer the SQL node (MySQL server), you should also install the Client RPM, as shown here:
shell> rpm -Uhv MySQL-Cluster-gpl-client-6.3.24-0.sles10.i586.rpm
This installs the mysql client program.

Data node installation — RPM Files. On a computer that is to host a cluster data node it is necessary to install only the NDB Cluster – Storage engine RPM. To do so, copy this RPM to the data node host, and run the following command as the system root user, replacing the name shown for the RPM as necessary to match that of the RPM downloaded from the MySQL AB web site:
shell> rpm -Uhv MySQL-Cluster-gpl-storage-6.2.18-0.sles10.i586.rpm

The previous command installs the MySQL Cluster data node binary (ndbd) in the /usr/sbin directory.
Management node installation — RPM file. To install the MySQL Cluster management server, it is necessary only to use the NDB Cluster – Storage engine management RPM. Copy this RPM to the computer intended to host the management node, and then install it by running the following command as the system root user (replace the name shown for the RPM as necessary to match that of the Storage engine management RPM downloaded from the MySQL AB web site):

shell> rpm -Uhv MySQL-Cluster-gpl-management-6.3.24-0.sles10.i586.rpm

This installs the management server binary (ndb_mgmd) to the /usr/sbin directory.
You should also install the NDB management client, which is supplied by the Storage engine basic tools RPM. Copy this RPM to the same computer as the management node, and then install it by running the following command as the system root user (again, replace the name shown for the RPM as necessary to match that of theStorage engine basic tools RPM downloaded from the MySQL AB web site):
shell> rpm -Uhv MySQL-Cluster-gpl-tools-6.3.24-0.sles10.i586.rpm

The Storage engine basic tools RPM installs the MySQL Cluster management client (ndb_mgm) to the /usr/bindirectory.

Configuration:

For our n-node, n-host MySQL Cluster, it is necessary to write n no of configuration files, one per node host.

1. Each data node or SQL node requires a my.cnf file that provides two pieces of information: a connectstring that tells the node where to find the management node, and a line telling the MySQL server on this host (the machine hosting the data node) to enable the NDBCLUSTER storage engine.

2. The management node needs a config.ini file telling it how many replicas to maintain, how much memory to allocate for data and indexes on each data node, where to find the data nodes, where to save data to disk on each data node, and where to find any SQL nodes.

Configuring the Storage and SQL Nodes:

The my.cnf file needed for the data nodes is fairly simple. The configuration file should be located in the /etcdirectory and can be edited using any text editor. (Create the file if it does not exist.) For example:
shell> vi /etc/my.cnf

For each data node and SQL node in our example setup, my.cnf should look like this:
# Options for mysqld process:
[mysqld]
ndbcluster # run NDB storage engine
ndb-connectstring=192.168.0.10 # location of management server

# Options for ndbd process:
[mysql_cluster]
ndb-connectstring=192.168.0.10 # location of management server
After entering the preceding information, save this file and exit the text editor.

ImportantOnce you have started a mysqld process with the NDBCLUSTER and ndb-connectstringparameters in the [mysqld] in the my.cnf file as shown previously, you cannot execute any CREATE TABLE or ALTER TABLE statements without having actually started the cluster. Otherwise, these statements will fail with an error.

Configuring the management node. The first step in configuring the management node is to create the directory in which the configuration file can be found and then to create the file itself. For example (running asroot):

shell> mkdir /var/lib/mysql-cluster
shell> cd /var/lib/mysql-cluster
shell> vi config.ini

For our representative setup, the config.ini file should read as follows:

# Options affecting ndbd processes on all data nodes:
[ndbd default]
NoOfReplicas=2 # Number of replicas
DataMemory=80M # How much memory to allocate for data storage
IndexMemory=18M # How much memory to allocate for index storage
# For DataMemory and IndexMemory, we have used the
# default values. Since the “world” database takes up
# only about 500KB, this should be more than enough for
# this example Cluster setup.

# TCP/IP options:
[tcp default]
portnumber=2202 # This the default; however, you can use any port that is free
# for all the hosts in the cluster
# Note: It is recommended that you do not specify the port
# number at all and allow the default value to be used instead

# Management process options:
[ndb_mgmd]
hostname=192.168.0.10 # Hostname or IP address of management node
datadir=/var/lib/mysql-cluster # Directory for management node log files

# Options for data node “A”:
[ndbd]
id=2 # (one [ndbd] section per data node)
hostname=192.168.0.30 # Hostname or IP address
datadir=/usr/local/mysql/data # Directory for this data node’s data files

# Options for data node “B”:
[ndbd]
id=3
hostname=192.168.0.40 # Hostname or IP address
datadir=/usr/local/mysql/data # Directory for this data node’s data files

# SQL node options:
[mysqld]
hostname=192.168.0.20 # Hostname or IP address
# (additional mysqld connections can be
# specified for this node for various
# purposes such as running ndb_restore)

After all the configuration files have been created and these minimal options have been specified, you are ready to proceed with starting the cluster and verifying that all processes are running.

Initial Startup of MySQL Cluster:
Starting the cluster is not very difficult after it has been configured. Each cluster node process must be started separately, and on the host where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes:

1. On the management host, issue the following command from the system shell to start the management node process:
shell> ndb_mgmd -f /var/lib/mysql-cluster/config.ini

On each of the data node hosts, run this command to start the ndbdprocess

shell> ndbd
1. If you used RPM files to install MySQL on the cluster host where the SQL node is to reside, you can (and should) use the supplied startup script to start the MySQL server process on the SQL node.
If all has gone well, and the cluster has been set up correctly, the cluster should now be operational. You can test this by invoking the ndb_mgm management node client. The output should look like that shown here, although you might see some slight differences in the output depending upon the exact version of MySQL that you are using:

shell> ndb_mgm

– NDB Cluster — Management Client –
ndb_mgm> SHOW
Connected to Management Server at: localhost:1186
Cluster Configuration
———————
[ndbd(NDB)] 2 node(s)
id=2 @192.168.0.30 (Version: 5.1.32-ndb-6.3.24, Nodegroup: 0, Master)
id=3 @192.168.0.40 (Version: 5.1.32-ndb-6.3.24, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.0.10 (Version: 5.1.32-ndb-6.3.24)

[mysqld(API)] 1 node(s)
id=4 @192.168.0.20 (Version: 5.1.32-ndb-6.3.24)
The SQL node is referenced here as [mysqld(API)], which reflects the fact that the mysqld process is acting as a MySQL Cluster API node.

Note: The IP address shown for a given MySQL Cluster SQL or other API node in the output of SHOW is the address used by the SQL or API node to connect to the cluster data nodes, and not to any management node.

After installation we go to the sql node and create one memory based table and one disk based table.

Mem Based Table:

CREATE TABLE table1 (col1 int, col2 int, col3 int, col4 int,
PRIMARY KEY(col1), index(col1,col2))
ENGINE=ndb;

This will create memory based table.

Disk Based Table:

To create a disk based table we first have to create logfile group.The creation of a LOGFILE GROUP creates a file on each data node for storing UNDO logs. (Please
note in version 6.2, only one LOGFILE GROUP is supported, theoretically an unlimited amount of
UNDOFILE’s are supported however. The default value for the UNDO buffer is 8 MB.)

CREATE LOGFILE GROUP logfg1
ADD UNDOFILE ‘undofile1.dat’
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE = 1M
ENGINE=ndb;

View Comments »

Posted in Mysql

Checklist Websecurity

January 24th, 2010

1. Disable TRACE and TRACK in the main scope of httpd.conf
2. Disable directory listing or Path Traversal on Apache
3. Disable following options in the httpd.conf file
UseCanonicalName Off
ServerSignature Off
HostnameLookups Off
ServerTokens Prod
4. Disable the weaker SSLv2 protocol and to enable the stronger TLSv1 protocol
5. Disabling Undesirable Options in php.ini (In the all server where we are using the PHP)
allow_url_fopen = Off
register_globals = Off
enable_dl = Off ()
expose_php = Off
disable_functions = openlog
display_errors = Off
display_startup_errors = Off

6. Disabling manual pages listing like http://test.com/manual/?
7. Apache prevent hot linking or leeching of images using mod_rewrite
8. Enable TCP SYN Cookie Protection and enables TCP SYN Flood protection
9. Disable HTTP TRACE supports. Based on site requirements and policy, consider disabling HTTP TRACE support in web servers.
10. If possible enable mod_security or modsecurity (ModSecurity is an open source intrusion detection and prevention engine for web applications. It operates embedded into the web server, acting as a powerful umbrella – shielding applications from attacks. ModSecurity supports both branches of the Apache web server.)
11. Patch apache with latest and stable version.
12. Put secure/important pages in the SSL
13. Monitor Apache logs continuously and keep the backup of daily logs for future reference.
14. Stop uploading php scripts directly through the web sites.

View Comments »

Posted in Apache, Security

Boost Network Performance / Throughput

January 24th, 2010

Network Performance Document
Linux Configure Jumbo Frames to Boost Network Performance / Throughput
Most modern Linux distros (read as Linux Kernel 2.6.17+) does support frames larger than 1500 bytes. This can improve the performance. First, make sure your network driver supports custom MTU. Second you need to have a compatible gigabit NIC

Jumbo frames can reduce server overhead such as a big decrease in CPU usage when transferring larger file. Also you should see some increase in network throughput.

# ifconfig eth0 mtu 9000

Make changes permanent

Edit the network configuration file for eth0 interface – for example, /etc/sysconfig/network-script/ifcfg-eth0 (CentOS / RHEL / Fedora Linux):

# vi /etc/sysconfig/network-script/ifcfg-eth0

Append the following configuration directive, which specifies the size of the frame in bytes:

MTU=9000

# /etc/init.d/networking restart

To confirm the MTU used between two specific devices. use ip command as follows:

# ip route get {IP-address}
# ip route get 192.168.1.1

Output:
192.168.1.1 dev eth0 src 192.168.1.100 cache mtu 9000 advmss 1460 hoplimit 64

Tuning sysctl.conf

The sysctl.conf of a server is something that is seldom optimized for performance. You can get a tremendous boost in throughput by adjusting these settings. This configuration has been written by Steve from Rack911. I have applied this configuration to servers ranging from Celeron 1.7Ghz to Dual Xeon 2.8Ghz servers, and on the whole, the load on each lowered after making the changes.

First make a backup of your old /etc/sysctl.conf file by running the following command, logged in as root:
cp /etc/sysctl.conf /etc/sysctl.conf.bak
Now enter:
vi /etc/sysctl.conf
and replace the contents of the file with the following:

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Disables packet forwarding
net.ipv4.ip_forward=0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.eth0.log_martians = 0
# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
# Disables the magic-sysrq key
kernel.sysrq = 0
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
# Enable TCP SYN Cookie Protection
net.ipv4.tcp_syncookies = 1
# Enable ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts = 1
# Enable bad error message Protection
net.ipv4.icmp_ignore_bogus_error_responses = 1
# Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 1
# Increases the size of the socket queue (effectively, q0).
net.ipv4.tcp_max_syn_backlog = 1024
# Increase the tcp-time-wait buckets pool size
net.ipv4.tcp_max_tw_buckets = 1440000
# Allowed local port range
net.ipv4.ip_local_port_range = 16384 65536
CTRL + X to exit and save the file
To make your changes take effect immediately, type this command:
/sbin/sysctl -p
You can watch your server load by entering “uptime” command via SSH.There you have it, a quick few things you can do to your server to boost performance and lower CPU load.
TCP Tuning
For servers that are serving up huge numbers of concurrent sessions, there are some TCP options that should probably be enabled. In order to optimize TCP performance for this situation, I would suggest tuning the following parameters:
#echo 1024 65000 > /proc/sys/net/ipv4/ip_local_port_range
Tuning Syn Backlog
Set the Syn backlog to a high number so Squid will never be starved due to a kernel limiting network connection. The default value is 1024.
#echo 16384 > /proc/sys/net/ipv4/tcp_max_syn_backlog
Set up Network Bonding
Linux allows binding multiple network interfaces into a single channel/NIC using special kernel module called bonding. The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical “bonded” interface.

TCP Tuning Guide
Linux TCP Tuning
There are a lot of differences between Linux version 2.4 and 2.6, so first we’ll cover the tuning issues that are the same in both 2.4 and 2.6. To change TCP settings in, you add the entries below to the file /etc/sysctl.conf, and then run “sysctl -p”.
Like all operating systems, the default maximum Linux TCP buffer sizes are way too small. I suggest changing them to the following settings:
# increase TCP max buffer size setable using setsockopt()
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
# set max to at least 4MB, or higher if you use very high BDP paths
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
Note: you should leave tcp_mem alone. The defaults are fine.
Another thing you can try that may help increase TCP throughput is to increase the size of the interface queue. To do this, do the following:
ifconfig eth0 txqueuelen 1000
I’ve seen increases in bandwidth of up to 8x by doing this on some long, fast paths. This is only a good idea for Gigabit Ethernet connected hosts, and may have other side effects such as uneven sharing between multiple streams.
Also, I’ve been told that for some network paths, using the Linux ‘tc’ (traffic control) system to pace traffic out of the host can help improve total throughput.
________________________________________
Linux 2.4
Starting with Linux 2.4, Linux has implemented a sender-side autotuning mechanism, so that setting the opitimal buffer size on the sender is not needed. This assumes you have set large buffers on the recieve side, as the sending buffer will not grow beyond the size of the recieve buffer.
However, Linux 2.4 has some other strange behavior that one needs to be aware of. For example: The value for ssthresh for a given path is cached in the routing table. This means that if a connection has has a retransmition and reduces its window, then all connections to that host for the next 10 minutes will use a reduced window size, and not even try to increase its window. The only way to disable this behavior is to do the following before all new connections (you must be root):
sysctl -w net.ipv4.route.flush=1
More information on various tuning parameters for Linux 2.4 are available in the Ipsysctl tutorial .
________________________________________
Linux 2.6
Starting in Linux 2.6.7 (and back-ported to 2.4.27), linux includes alternative congestion control algoritms beside the traditional ‘reno’ algorithm. These are designed to recover quickly from packet loss on high-speed WANs.
Linux 2.6 also includes and both send and receiver-side automatic buffer tuning (up to the maximum sizes specified above). There is also a setting to fix the ssthresh caching weirdness described above.
There are a couple additional sysctl settings for 2.6:
# don’t cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
# recommended to increase this for 1000 BT or higher
net.core.netdev_max_backlog = 2500
# for 10 GigE, use this
# net.core.netdev_max_backlog = 30000
Starting with version 2.6.13, Linux supports pluggable congestion control algorithms . The congestion control algorithm used is set using the sysctl variable net.ipv4.tcp_congestion_control, which is set to cubic or reno by default, depending on which version of the 2.6 kernel you are using.
To get a list of congestion control algorithms that are available in your kernel, run:
sysctl net.ipv4.tcp_available_congestion_control
The choice of congestion control options is selected when you build the kernel. The following are some of the options are available in the 2.6.23 kernel:
• reno: Traditional TCP used by almost all other OSes. (default)
• cubic: CUBIC-TCP
• bic: BIC-TCP
• htcp: Hamilton TCP
• vegas: TCP Vegas
• westwood: optimized for lossy networks
For very long fast paths, I suggest trying cubic or htcp if reno is not is not performing as desired. To set this, do the following:

sysctl -w net.ipv4.tcp_congestion_control=htcp
More information on each of these algorithms and some results can be found here .
More information on tuning parameters and defaults for Linux 2.6 are available in the file ip-sysctl.txt, which is part of the 2.6 source distribution.
And finally a warning for both 2.4 and 2.6: for very large BDP paths where the TCP window is > 20 MB, you are likely to hit the Linux SACK implementation problem. If Linux has too many packets in flight when it gets a SACK event, it takes too long to located the SACKed packet, and you get a TCP timeout and CWND goes back to 1 packet. Restricting the TCP buffer size to about 12 MB seems to avoid this problem, but clearly limits your total throughput. Another solution is to disable SACK.
________________________________________
Linux 2.2
If you are still running Linux 2.2, upgrade! If this is not possible, add the following to /etc/rc.d/rc.local
echo 8388608 > /proc/sys/net/core/wmem_max
echo 8388608 > /proc/sys/net/core/rmem_max
echo 65536 > /proc/sys/net/core/rmem_default
echo 65536 > /proc/sys/net/core/wmem_default

Network Monitoring Tools

http://acs.lbl.gov/NCS/download/

View Comments »

Posted in CentOS, Linux, Network Performance