As every system administrator knows, hackers are forever trying to break into
your system. Just look at your error logs — they are filled with references
File does not exist:" followed by some filename
and/or directory which you've never heard of and likely isn't even relevant
to your operating system. In the meantime, the valid error messages about which
you need to do something are lost in the flood of messages due to Nimda and
other attackers. It's time to take back your error logs.
There is a way to filter out all those hacker-generated error messages —
it's called piped error logs. The
directive in the Apache documentation mentions that you can pipe the error
messages to a script, but doesn't describe the process in any more detail, and
alas, the devil is in the details.
A common way to invoke such a script is to set the
directive as follows:
ErrorLog "|/path/to/script/script >>/path/to/logfile/logfile"
for example, I use
ErrorLog "|/www/cgi-src/errorlog.pl >>/www/logs/error.log"
Note the leading "
|" symbol which says to send
(in technical terms pipe, as a verb) each error message to the named
script, which in my case is a Perl script called
The script receives the error message via Standard Input (
and sends its result to Standard Output (
we follow the call to the script in the
>>/www/logs/error.log", the script's
output is appended to the file
However, there are two gotchas.
The first gotcha concerns how the Apache Server interacts with the script.
First, whenever the server starts, it invokes the script. Now, if your script
were to wait for an error message from
STDIN, act on it
and then terminate, the script wouldn't be around to process the next error
message. This means that the script must be in a continuous loop waiting for
STDIN to respond, terminating only when the server sends
STDIN. Thus, the server controls
the script entirely through
STDIN. The script starts out
waiting for a line of input from
STDIN, processes the line,
and then waits for another line.
Using the scripting language Perl, this translates to surrounding the non-static portion of the script with something like
When the server has an error message to display, it sends it to
the script wakes up and does its thing. Then the script goes back to the
statement and waits for more input which occurs only when the server has another
error message. At that point, the code inside the
loop repeats until the server terminates at which point the script has an opportunity
to clean up.
In the simplest case, you might think that the entire program can be as short as
print while <STDIN>;
However, were you to try this program you would find that it appears not to work, which brings us to the second gotcha: buffered output. In short, Perl (as with many other languages) doesn't display its output immediately — instead, the output is cached (in a buffer) and is spit out only when the buffer fills (or is flushed). Thus, the program appears not to be working only because there is no output. In fact, it is working, but like a squirrel with a nut in its cheek, it just hasn't displayed anything as yet. The solution is to tell Perl to use unbuffered output, adding one more statement which, finally, produces a working program:
$|=1; # Use unbuffered output
print while <STDIN>;
Now that we know how the script works, let's add some code to filter out unwanted error messages. At this point, the problem is down to how to identify unwanted messages, for example,
[Sat Jan 1 06:10:30 2005] [error] [client 22.214.171.124] user not
One way is to examine the file mentioned at the end of the message. For example,
you might be running a Unix-like system and the files typical to a Windows system
have no meaning on your system. Thus you might want to filter out messages which
More specifically, you might want to filter out files with explicit names, such
formmail.pl, to name a few.
Another way is to filter out error messages which name a specific directory,
The following code implements these ideas. However, you should review your error logs regularly to see what new tricks the black hat hackers have up their sleeves. The following is my current collection:
# Skip those error messages ending with any of the following terms:
$end = 'cmd\.exe|root\.exe|formmail\.pl|formmail|owssvr\.dll|cltreq\.asp|wpad\.dat|'
. 'Ne0\d:\.exe|EPSON\.exe|HP LaserJet\.exe|NETLOGON|author\.exe';
# Skip those error messages which reference any of the following directories:
$dir = 'ads tour slv html\.ng js\.ng passport m download adi adj us\.yimg\.com free '
. 'banners dynamic img caes ice config espana\.starmedia\.com galeria imag '
. 'producto w3c officescan';
$dir = join '/|/', split ' ', "/$dir/"; # Replace spaces
with interior '/|/', and
# put in leading and trailing directory markers
Use unbuffered output
while (<STDIN>) #
Loop through STDIN
$Msg = $_; # Capture the line of input
if ($Msg !~ m!($end)$!io # If it doesn't match the forbidden endings,
&& $Msg !~ m!($dir)!io) # and it doesn't match the forbidden directories, ...
print $Msg; # Print it
Although the above discussion uses Perl as the scripting language, any other programming language may be substituted. The two key elements are an infinite loop around a call to Standard Input and unbuffered output. Thereafter, regular expressions can easily filter out unwanted error messages.
For example, using PHP, the short version of a working program looks like
$stdin = fopen ('php://stdin', 'r');
ob_implicit_flush (true); // Use unbuffered output
while ($line = fgets ($stdin))
Keep in mind that each error message is a wake up call about an attack. Before you choose to discard an error message, you also need to be completely sure you have guarded against such an attack.
For example, you might know that a call to
a particular directory is bogus because it doesn't exists in that directory,
but does it exist in a different directory from which it might be a threat?
Might you at some later time install, say, FrontPage which will create the
file? Why are hackers targeting that file anyway — is there something about
it you don't know which causes it to be a security risk?
Be sure you answer these and other questions before adopting this technique.
TransferLogusing a script tailored to the particular log type and its contents.
httpd.conffile and have a spare virtual domain name, it is a good idea to test changes to your script there, rather than on a live system.
"|perl /www/..."), don't forget to set execute permissions on the script (
chmod 0755 errorlog.pl).
This page was created by Bob Smith -- please any questions or comments about it to me.
Thanks for Rex Swain for his helpful comments.
|© Sudley Place Software 1997-2017.|
|Comments or suggestions? Send them to .|