Blog Objective

This is a blog that attempts to make life easier by noting down the author's accrued knowledge and experiences.
The author has dealt with several IT projects (in Java EE and .NET) and is a specialist in system development.

06 May 2011

Apache Web Server Troubleshooting

How to troubleshoot Apache Web Server?

A few standard commands to know by hard:
  1. To confirm the modules that are or will be loaded in Apache: apachectl –M [-f <conf filename>]
  2. To check the virtual host load pattern: apachectl –S [-f <conf filename>]
  3. To test a httpd.conf file: apachectl –t [-f <conf filename>]
  4. To maintain the http server: apachectl start|stop|restart
  5. To start with a specified configuration file: apachectl –f <conf filename>
Create a minimalist httpd.conf. Something like:
Listen 80
ServerName myserver.com:80
ServerAdmin admin@myserver.com
#ServerRoot "." 
ServerRoot "/usr/local/apache2"
User nobody
Group nobody
#DocumentRoot "..\wwwroot" 
DocumentRoot "/home/data/html"
ErrorLog "logs/basic_apache.log"
LogLevel debug
#LoadModule log_config_module modules/mod_log_config.dll
LoadModule log_config_module modules/mod_log_config.so
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Host}i\" \"%{Referer}i\" %a %A" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog logs/basic_access.log combined
</IfModule>
Turn on log filter to debug

How to troubleshoot virtual directories?

Create a minimalist httpd.conf. Something like:
Listen 80
ServerName myserver.com:80
ServerAdmin admin@myserver.com
#ServerRoot "." 
ServerRoot "/usr/local/apache2"
User nobody
Group nobody
#DocumentRoot "..\wwwroot" 
DocumentRoot "/home/data/html"
ErrorLog "logs/basic_apache.log"
LogLevel debug
#LoadModule log_config_module modules/mod_log_config.dll
LoadModule log_config_module modules/mod_log_config.so
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Host}i\" \"%{Referer}i\" %a %A" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog logs/basic_access.log combined
</IfModule>
NameVirtualHost *:80
<VirtualHost *:80>
ServerName vhost1.com
#DocumentRoot "..\wwwroot" 
DocumentRoot "/home/data/html"
</VirtualHost>
<VirtualHost *:80>
ServerName vhost2.com
#DocumentRoot "..\wwwroot\vhost1" 
DocumentRoot "/home/data/html/vhost1"
</VirtualHost>
<VirtualHost *:80>
ServerName vhost2.com
#DocumentRoot "..\wwwroot\vhost2" 
DocumentRoot "/home/data/html/vhost2"
</VirtualHost>
<VirtualHost *:80>
ServerName vhost3.com
#DocumentRoot "..\wwwroot\vhost3" 
DocumentRoot "/home/data/html/vhost3"
</VirtualHost>
Create the necessary DocumentRoot for each virtual host with files therein for basic troubleshooting

How to troubleshoot WebLogic plug-in module?

  1. Turn the proxy parameters to debug mode:
    1. debug on
    2. ShowConfigInfo yes
  2. The debug file by default can be found in /tmp/wl_proxy.log

The HTTP error code thrown by the plug-in depends on the situation. Plug-in will return the HTTP error code 500 in the following conditions:
  • Neither WebLogicCluster nor WebLogicPort was specified in the httpd.conf file.
  • Unable to resolve the WebLogicHost parameter specified in the httpd.conf file.
  • Port number specified by WebLogicPort, in the httpd.conf file, exceeds 65535.
  • Unsuccessful in parsing the request while applying the PathTrim property.
  • The request header is of type Unknown Transfer-Encoding.
  • Failed to read the chunked request.
  • Encountered an error reading POST data from client.
  • Failed to open a temporary(temp) file.
  • Failed to write POST data to the temp file.
  • Encounetered an error reading POST data from the temp file.
  • POST timed out.
  • SSL was specified without the parameter trustedCAFile.

On the other hand, the HTTP error code 503 is returned when:

  • The maximum number of retries is exceeded. This value is computed by dividing ConnectTimeoutSecs by ConnectRetrySecs.
  • Idempotent is OFF.

Personal Experience


I had the privilege of troubleshoot Apache with WL proxy. Strangely, the proxy works with the non-clustered setting (by specifying the WebLogicHost <hostname>) but does not work with the clustered setting (WebLogicCluster <address>)

As far as I’m aware, there is nothing in WebLogic administration parameter setting required for a web server proxy to WLS cluster.

After playing with a couple of parameters (e.g. request timeout, retries, IO timeout) and giving up as nothing works, I finally looked in the wl_proxy.log and discovered the issue! We had used IP address to route traffic to from Apache but the WLS cluster was configured with the hostname. Apparently, the proxy was not able to route traffic in the cluster.

No comments: