Posts

Showing posts with the label troubleshoot

Performance Counters for .NET

A little dated by now. From Windows 2000 Resource Kit: http://technet.microsoft.com/en-us/library/cc938609.aspx http://technet.microsoft.com/en-us/library/cc940375.aspx http://technet.microsoft.com/en-us/library/cc938586.aspx http://technet.microsoft.com/en-us/library/bb734903.aspx For .NET http://www.symantec.com/business/support/index?page=content&id=HOWTO9722 From Windows 2003: http://technet.microsoft.com/en-us/library/cc779038(WS.10).aspx

Troubleshooting Common .NET HTTP Connection Errors

The first is to identify whether the error is with the client or server (or even intermediaries). Most of the errors begin with “ The underlying connection was closed: “… Indications of client error An unexpected error occurred on a send – Could be due to: antivirus software installed on the client machine   Indications of intermediary error The remote name could not be resolved or The proxy name could not be resolved – Could be due to: DNS issue inability to access the hosts file Unable to connect to the remote server – Likely to have gotten through the DNS but hit a connection glitch due to: proxy firewall network authentication   Indications of server error or intermediaries (e.g. load balancer, proxy, etc.) An unexpected error occurred on a receive – Server or intermediary unexpectedly closes the TCP connection. May be due to: Server or intermediary timeout values set too low ( TODO : increase the client’s request timeout & also the server...

System Deterioration

Machineries need oiling after some time; building structure deteriorates due to natural forces; facades need to be repainted; the Golden Gate bridge needs to be repainted; vehicles need to be maintained; what does that mean for systems? I’m inclined to believe that software systems, like all other things, need to be constantly maintained, oiled, cleaned, before they deteriorate. How does deterioration look like? Deterioration may appear in these forms: systems become slower progressively systems crash/ become unavailable more often systems become more bloated (larger codebase, more storage space required, etc.) The system was originally deployed and tested fine so what went wrong? How does deterioration happen? It typically happens due to the following forces: user-base increased post-deployment to a number that was not intended/ tested for smart users found ways to use the system that was not originally intended for operation/ support team did not make it a point to upkeep t...

Logging Levels

Most popular logging frameworks (e.g. Log4J, SLF4J, Log4Net, EntLib) allow for several logging levels to be instrumented. A common question is: What logging levels are appropriate under what scenarios? The following lists some common log levels are their usage: Fatal - Service/ application/ process about to terminate. Unable to proceed with normal operation. Force a shutdown to prevent (further) data loss. Examples: No more system resource (OOM) Error - Unhandled or unexpected system error has occurred. Current operation/ thread to be aborted. Issue needs to be fixed. Still able to continue with normal operation (at least for other users/ sessions) otherwise. Examples: database deadlock Unexpected/ unhandled exception Runtime errors Transaction aborted Can't create file Warn - Things are generally working fine. Recoverable conditions. Transient environmental conditions happened. Something happened (but was handled) that may potentially ...

Minimum (Lean) System Documentation

Let’s admit it: any form of system documentation is not up-to-date . The moment we start producing it, it is out-of-date. If that is the case, we should do with minimum (perhaps, lean) system documentation that is kept current. This begs the next question: How little is enough? If I imagine myself taking over a system from someone else, I believe the minimum/ lean system documentation should contain the following: Solution Design description of the high-level process flow description of the main modules/ services in the system what are the architecturally significant use cases or main functions of the system? what processes/ components make up the system? E.g. Are there web-based applications? Are there batch processes? what are the databases in use? What are the primary (entity) tables in use? what output is generated by the system? E.g. Are there printed output? Are there output for system integration? Are there messages (emails/ SMS) sent? what systems are integra...

Oracle Weblogic Server States

Image
With respect to the Weblogic server, there are several runtime states that are interesting. In addition, certain events/ commands lead to the transitions to other known states. I attempted to capture the state transitions and events into a UML state diagram for easy reference. States in brown are end-states while those in yellow are transitional.

Apache Web Server Troubleshooting

How to troubleshoot Apache Web Server? A few standard commands to know by hard: To confirm the modules that are or will be loaded in Apache: apachectl –M [-f <conf filename>] To check the virtual host load pattern: apachectl –S [-f <conf filename>] To test a httpd.conf file: apachectl –t [-f <conf filename>] To maintain the http server: apachectl start|stop|restart To start with a specified configuration file: apachectl –f <conf filename> Create a minimalist httpd.conf. Something like: Listen 80 ServerName myserver.com:80 ServerAdmin admin@myserver.com #ServerRoot " . " ServerRoot " /usr/local/apache2 " User nobody Group nobody #DocumentRoot " ..\wwwroot " DocumentRoot " /home/data/html " ErrorLog " logs/basic_apache.log " LogLevel debug #LoadModule log_config_module modules/mod_log_config.dll LoadModule log_config_module modules/mod_log_config.so <IfModule log_config_module> LogFormat " %h %l...

Monitoring & Troubleshooting ASP.NET Applications on IIS

Toolbox for the IIS server net start sc query tasklist /svc netstat –ano wfetch/ wget/ curl procmon/ filemon/ regmon eventvwr mmc.exe (reliability & performance monitoring) process time & % .NET CLR ASP.NET Requests Wait Time Requests Current Requests Queued Requests Rejected Toolbox for the browser machine ping telnet/ portcheck wfetch/ wget/ curl When should application pools be turned into web gardens? Web gardens can only be used if the application doesn’t use in-process session variables but rather out-of-process ones (e.g. session state service or database session state). Drivers to using web gardens are: Application makes long-running synchronous requests Application is low in availability and crashes often Application creates high CPU load on work process Non-IIS Settings & Bottlenecks Optimum paging file size setting: 1.5x the RAM for 32-bit OS system-managed for 64-bit OS Disk queue length should always average les...