This week's installment of Detecting Malice with ModSecurity will discuss how to detect potential open proxy abuse. HTTP Proxies excerpt section of Robert 'Rsnake' Hansen's book 'Detecting Malice' -
In analyzing typical web-based attacker methodology, for example as part of the WASC Distributed Web Honeypot Project, SpiderLabs sees that the vast majority of attackers use proxy chaining techniques. Proxy chaining is merely connecting to more than one proxy and then onto to the intended destination. Attackers can use as many proxy servers as they want. The more they have, the more anonymous they will be. By using proxy chaining the http path may look like this:
attacker's computer => proxy1 => proxy2 => ... => proxy X => target web-site
In order to maximize the potential traceback challenges of security personnel, the savy attackers will choose to distribute their open proxy chaining across multiple geographic regions as this would complicate the process of obtaining log data. It is this key fact that we can use to our advantage for detecting open proxy abuse.
Due to the high use of international proxy chaining by attackers, we can look for GeoIP Country Code mismatches between the final host that is connecting to our web server and the first IP address listed in any X-Forwarded-For request header (as the first IP address is the origin of the request). In a previous blog post - Detecting Malice with ModSecurity: GeoLocation Data - we showed that ModSecurity supports using geolocation data through the integration with the free MaxMind GeoLite Country or GeoLite City databases.
The first step is to download the database of your choice and put it somewhere on the local filesystem where ModSecurity can use it (for example in the same directory as the Core Rule Set).
Before you can use the GeoIP data, you first have to use the SecGeoLookupDb directive:
Once the SecGeoLookupDb directive is active, you can then use the following rule set to identify GeoIP Country Code Mismatches between the X-Forwared-For Request Header and the REMOTE_ADDR of the client connecting directly to your web server:
The first SecRule inspects the first IP address listed in any X-Forwarded-For headers and captures the data in a TX collection to be evaluated later against the REMOTE_ADDR fo the client. The regular expression used in this rule uses a negative look-behind to exclude internal/private IP ranges in order to avoid false positives. The 2nd SecRule takes the saved data and resolves it against the GeoIP DB and stores the Country Code data. The 3rd SecRule resoves the REMOTE_ADDR against the GeoIP DB and then the 4th SecRule compares these two values together and if they do not match then it alerts.
Let's say that your web server receives the following request as shown by the ModSecurity audit log format entry:
The bolded data under Section (A) is the REMOTE_ADDR of the client and the bolded data under Section (B) is the X-Forwarded-For data. This request would You would generate an alert message like the following:
This experimental rule set has been added to the OWASP ModSecurity CRS trunk repository. If you would like to try out these latest CRS, you should follow these steps:
HTTP Proxies
HTTP proxies are specialized server applications designed to make HTTP requests on behalf of their users. As a result, web servers that receive such requests by default get to see only IP addresses of proxies, and not those of the real users. In many cases the requests will contain the identifying information, but you will need to actively look for it and configure your servers to log such information.
In HTTP, request meta-data is transported in request headers, so it is no surprised that there you may also find the information identifying the users of proxies. The following request headers are commonly used for this purpose:
Via
Via – part of HTTP and designed to track HTTP transactions as they travel through proxy servers. The header can contain information on one or several proxies.
X-Forwarded-For – a non-standard header introduced by the developers of the Squid proxy before the Via header was added to HTTP. X-Forwarded-For is very widely supported today.
Note: In addition to the two request headers above, reverse proxies (which typically route requests from many users to a single server) will set a number of different requests headers to convey the original IP address and, less often, SSL information. Such headers include X-Forwarded-Host, X- Forwarded-Hostname, Client-IP, and others. You will very rarely see such information unless you setup a reverse proxy yourself. If you do see one of these headers in your requests you are advised to investigate, because their presence indicates an unusual—and thus suspicious—setup.
...
X-Forwarded-For
The X-Forwarded-For header is simpler, and it is designed to only contain the IP addresses of the proxies that relayed a HTTP transaction. Here are some examples:
10.255.39.116, 203.144.32.176
10.140.37.84, unknown
212.23.82.100, 212.23.64.155, 195.58.1.134
209.131.37.237
unknown
Proxy Spoofing
The headers that proxies use to relay real user information are not at all required for normal web operation. While most modern proxies are pretty good about including this information, they all allow themselves to be configured not to send it—so you’ll be at the administrators’ mercy, essentially. You should also be aware that anyone not using a proxy can pretend to be a proxy, as the headers can be trivially spoofed.
Open Proxy Abuse - Proxy Chaining for Anonymity
In analyzing typical web-based attacker methodology, for example as part of the WASC Distributed Web Honeypot Project, SpiderLabs sees that the vast majority of attackers use proxy chaining techniques. Proxy chaining is merely connecting to more than one proxy and then onto to the intended destination. Attackers can use as many proxy servers as they want. The more they have, the more anonymous they will be. By using proxy chaining the http path may look like this:
attacker's computer => proxy1 => proxy2 => ... => proxy X => target web-site
In order to maximize the potential traceback challenges of security personnel, the savy attackers will choose to distribute their open proxy chaining across multiple geographic regions as this would complicate the process of obtaining log data. It is this key fact that we can use to our advantage for detecting open proxy abuse.
Detecting Open Proxy Abuse
Due to the high use of international proxy chaining by attackers, we can look for GeoIP Country Code mismatches between the final host that is connecting to our web server and the first IP address listed in any X-Forwarded-For request header (as the first IP address is the origin of the request). In a previous blog post - Detecting Malice with ModSecurity: GeoLocation Data - we showed that ModSecurity supports using geolocation data through the integration with the free MaxMind GeoLite Country or GeoLite City databases.
The first step is to download the database of your choice and put it somewhere on the local filesystem where ModSecurity can use it (for example in the same directory as the Core Rule Set).
Before you can use the GeoIP data, you first have to use the SecGeoLookupDb directive:
SecGeoLookupDb
Description: Defines the path to the geographical database file.
Syntax:SecGeoLookupDb /path/to/db
Example Usage:SecGeoLookupDb /usr/local/geo/data/GeoLiteCity.dat
Processing Phase: N/A
Scope: Any
Version: 2.5.0
Dependencies/Notes: Check outmaxmind.comfor free database files.
Once the SecGeoLookupDb directive is active, you can then use the following rule set to identify GeoIP Country Code Mismatches between the X-Forwared-For Request Header and the REMOTE_ADDR of the client connecting directly to your web server:
SecGeoLookupDb /usr/local/apache/conf/modsec_current/base_rules/GeoLiteCity.dat
SecRule REQUEST_HEADERS:X-Forwarded-For "^\b\d{1,3}(?<!192|127|10)\.\d{1,3}\.\d{1,3}\.\d{1,3}\b" "chain,phase:1,t:none,capture,block,msg:'Potential Open Proxy Abuse - GeoIP Country Code Mismatch of X-Forwarded-For Request Header and Client REMOTE_ADDR',logdata:'IP Country is: %{geo.country_code} and X-Forwarded-For is: %{tx.geo_x-forwarded-for}'"
SecRule TX:0 "@geoLookup" "chain,setvar:tx.geo_x-forwarded-for=%{geo.country_code}"
SecRule REMOTE_ADDR "@geoLookup" "chain,t:none"
SecRule GEO:COUNTRY_CODE "!@streq %{tx.geo_x-forwarded-for}" The first SecRule inspects the first IP address listed in any X-Forwarded-For headers and captures the data in a TX collection to be evaluated later against the REMOTE_ADDR fo the client. The regular expression used in this rule uses a negative look-behind to exclude internal/private IP ranges in order to avoid false positives. The 2nd SecRule takes the saved data and resolves it against the GeoIP DB and stores the Country Code data. The 3rd SecRule resoves the REMOTE_ADDR against the GeoIP DB and then the 4th SecRule compares these two values together and if they do not match then it alerts.
Example Usage
Let's say that your web server receives the following request as shown by the ModSecurity audit log format entry:
--1f9d1914-A-- [24/Mar/2011:10:19:05 --0400] TYtS1cCoAWgAAQFSdscAAAAA 66.162.147.51 62641 ::1 80 --1f9d1914-B-- GET /path/to/foo.php HTTP/1.1 Host: www.example.com Connection: keep-alive User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.151 Safari/534.16 Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 X-Forwarded-For: 194.30.232.180, 196.2.126.171 --1f9d1914-F-- HTTP/1.1 200 OK Content-Length: 1221 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/plain; charset=iso-8859-1
The bolded data under Section (A) is the REMOTE_ADDR of the client and the bolded data under Section (B) is the X-Forwarded-For data. This request would You would generate an alert message like the following:
Message: Warning. Match of 'streq %{tx.geo_x-forwarded-for}' against 'GEO:COUNTRY_CODE' required. [file '/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf'] [line '37']
[msg 'Potential Open Proxy Abuse - GeoIP Country Code Mismatch of X-Forwarded-For Request Header and Client REMOTE_ADDR'] [data 'IP Country is: US and X-Forwarded-For is: GR' Added to OWASP ModSecurity Core Rule Set (CRS)
This experimental rule set has been added to the OWASP ModSecurity CRS trunk repository. If you would like to try out these latest CRS, you should follow these steps:
- Create a local svn directory -
$ mkdir svn/crs - Go into the the svn directory -
$ cd svn - Use SVN to clone the CRS trunk -
$ svn co https://mod-security.svn.sourceforge.net/svnroot/mod-security/crs/trunk crs
