I thought I’d take a moment to post about some web security tools I use
pretty often, which help as a security consultant when responding to
various web hacking related incidents. These tools have helped me write
my own scripts whenever I’m in a jam and need something good and quick
to do the job.

Application Log File Forensics: The Hard Way

The first thing a security professional or administrator usually think
of when handling an application security incident is to check the logs
for the applications, databases, and other application-tiers involved.
Often, these logs are either on the servers that run the applications
themselves, or possibly in a central logging location. If a certain
attacker tool can be identified from the log files (or other sources
such as full packet-capture), then it may be of interest to run that
exact same tool against your own application-under-target (preferably in
a mocked-up lab or test environment, if it mirrors production well
enough).

The most popular web servers, Apache httpd and Microsoft IIS, do create
local log files by default. According to most compliance regulations and
standards (e.g. COBIT, HIPAA, GLB, PCI-DSS, FISMA, EU Directive on
Privacy and Electronic Communications, ISO 17799/27002, CA SB1386 and
similar), logging must be centrally located, or may have other required
provisions. This may include application-layer information, such as the
log information from Apache and IIS. It may be very likely that your
organization already has centralized logging where this information is
available.

If centralized logging does not exist, it may be a good time to start up
a project to enable it. The Apache Cookbook,
2E, is the best place to go in order to
configure httpd to start sending syslog information. It’s about as
simple to add "ErrorLog syslog:user" into the httpd.conf file, but this
only logs error messages, not authentication/access_log messages. The
book gives two prescriptions, one using "AccessLog "|/usr/bin/logger"
combined" if your OS supports the logger command properly. The other is
to run a custom log message through a Perl script, as seen below:

CustomLog |/usr/local/apache/bin/apache_syslog combined

#!/usr/bin/perl
use Sys::Syslog
qw( :DEFAULT setlogsock );
setlogsock('unix');
openlog('apache', 'cons', 'pid', 'user');

while ($log = <STDIN>) {
    syslog('notice', $log);
}

closelog;

Microsoft IIS will need to go through the Event Log, which can be
converted to syslog messages using a third-party software package such
as Snare or
MonitorWare Agent. If IIS logs can also be
converted to w3c standard log format, then Apache log analyzer tools
such as AWStats could also be used. W3C also
has their own log analysis tool that also does HTML validation, called
the Log Validator. These
may be useful to run following your own scan of the application using
the same or similar attacker
tool,
as they will not only point out where in your application the scan/tool
covered, but also where you may have the most errors or lack of
quality/security controls.

The book Practical Information Security
Monitoring also makes some suggestions
for log collections, including the use of
Sawmill or Splunk to
sort/search log messages and gain further information and detail. There
may also be further adjustments you will want to do at the application
(or other tier) layer, such as logging POST data. We discussed logging
HTTP referrers on our old post: Using Google Analytics to Subvert
Privacy.
Practical Information Security Monitoring talks about Oracle audit
logging, but there is also a detailed article on Pete Finnigan’s blog on
Oracle forensics and
UKOUG. At
the recent BlackHat DC conference, David Litchfield gave a talk on The
Forensic Investigation of a Compromised Oracle Database
Server,
which may also be of interest (once the slides are available). There are
also some new books coming out on the topic of Oracle Forensics in the
next few months / year.

Web Application Incident Handling: The Easy Way

Most of the logfile "digging" takes time, even when consolidated and
using expert tools and analysis. There are some very easy approaches
that we’ve come up with, or seen others using and talking about. These
tools integrate well at the HTML and Script layers.

Over a year ago, Mario Heiderich started the PHP-IDS
project, as a way to build protection and
monitoring capabilities into PHP applications. Several side projects
spurred up as a direct result of the incredible work that was put into
PHP-IDS, mainly its default_filter.xml regular
expressions.
This XML file of regular expressions provides capabilities to detect a
vast range of attacks, including XSS, CSRF, SQL Injection, Directory
Traversal, Local/Remote File Execution, DoS, and Information Disclosure.
Part of the success behind the PHP-IDS project, was the constant testing
and attacking of PHP-IDS regex filters, which can be reviewed
extensively in this sla.ckers.org
thread. More info on
PHP-IDS can be found in the PHP-IDS FAQ.

Romain Gaucher, wrote
Scalp,
an Apache log analyzer in Python, which leverages PHP-IDS’
default_filter.xml to detect attack strings in logs. I’ve used Scalp on
numerous occasions, including a recent attack attempt on
tssci-security.com. By nature, Scalp cannot examine POST content because
Apache logs do not contain POST data. (See PHP-IDS or mod_security for
those purposes)

Simply use Scalp by running it as follows (keep in mind there may be
false positives with regards to the attack type, though it is very good
at pulling attack queries from the log):

./scalp.py --log access_log --filters ./default_filter.xml \
--html --tough --exhaustive

Arshan Dabirsiaghi recently released
OWASP
Scrubbr.
Scrubbr works by detecting input data in a specified database that does
not match up with a specified AntiSamy policy file. Because Scrubbr uses
an AntiSamy policy to validate data, does not mean it necessarily
detects XSS in your database. Note, one does not require AntiSamy to be
implemented in an application to use Scrubbr. Using Scrubbr, you have
the capability of validating each and every column capable of holding
strings of every row of every table in a database.

Together, Scalp and Scrubbr make for excellent web application security
forensic tools. Scalp can help detect attacks in Apache logs, and
Scrubbr can help you clean your database of content that does not match
your site’s policy.

*** This is a Security Bloggers Network syndicated blog from tssci-security authored by Marcin Wielgoszewski. Read the original post at: http://www.tssci-security.com/archives/2009/02/23/web-application-security-incident-handling