
                                  --=--
                                  p.0.f
                                  --=--

                    "Dr. Jekyll had something to Hyde"
		    
                      passive OS fingerprinting tool
                              version 1.8.3

     (C) Copyright 2000 - 2003 by Michal Zalewski <lcamtuf@coredump.cx>
      (C) Copyright 2001 - 2003 by William Stearns <wstearns@pobox.com>

                        http://www.stearns.org/p0f/


--------------
Project Status
--------------

  This program is now hosted and maintained by William Stearns
  <wstearns@pobox.com>.  It was originally written by Michal Zalewski
  <lcamtuf@coredump.cx>.  Feel free to mail William or both of us with
  bug-fixes, ideas, etc. =)

-----------------
Special thanks to
-----------------

  * Lance Spitzner for his white paper on passive OS fingerprinting:
    http://project.honeynet.org/papers/finger/

  * tf8 for initial piece of libpcap support and packet parsing

  * teso/security.is/b0f/#hax for ideas and testing

  * Jeremy Weatherford, Chris Wilson and Szilveszter Adam for
    portability testing/patches, bug-fixes and ideas,

  * other BUGTRAQ readers for OS fingerprints and useful patches

  * other people involved (or not) in this project

  * very, very special thanks to el- :*


----------  
Background
----------

  * What is passive OS fingerprinting?

  The passive OS fingerprinting technique is based on information coming
  from a remote host when it tries to establish a connection to your
  system.  Captured packet parameters contain enough information to
  identify the remote OS.  In contrast to active scanners such as nmap
  and queSO, p0f does this without sending anything to the remote host.

  If you're looking for more information on this approach, read
  Spitzner's white paper (mentioned above). :)

  In short, there are certain TCP/IP flag settings specific for given
  systems. Usually initial TTL (8 bits), window size (16 bits), maximum
  segment size (16 bits), don't fragment flag (1 bit), sackOK option (1
  bit), nop option (1 bit), window scaling option (8 bits), and initial
  packet size (16 bits) vary from one TCP stack implementation to
  another.  Together, they give a unique, 67-bit signature for every
  system.

  Some portions of the p0f code are currently used by IDS systems and
  sniffer software.
  
  * What are the main advantages?

  Passive OS fingerprinting can be done on huge amounts of input data -
  for example, information gathered on a firewall, proxy, routing device
  or Internet server - without causing any network activity.  You can
  launch passive OS detection software on such a machine and leave it
  for days, weeks or months, collecting really interesting statistical
  information about your customers, attackers, other servers, etc. 
  Since packet filtering firewalls, network address translation and so
  on are almost always transparent to p0f-alike software, you're able to
  obtain information about systems behind the firewall.  Also, such
  software can determine the distance between a remote host and your
  system, allowing you to generate network structure maps for
  firewalled/structural networks.  All this can be done without sending
  a single packet.  It is especially nice for IDSes.

-----------  
Limitations
-----------

  Proxy firewalls and other high-level proxy devices are not transparent
  to any TCP-level fingerprinting software. The device itself will be
  fingerprinted, not actual source hosts.

  In order to obtain information required for fingerprinting, you have
  to receive at least one SYN packet initiating a TCP connection to your
  machine or network.  Note: you don't have to respond to this
  particular SYN.  Of course, it's impossible to perform any kind of OS
  detection without receiving any information.

  It is possible to perform passive fingerprinting on a live TCP
  connection, or on a connection established by you to a remote host.
  However, these techniques are less reliable (many implementations copy
  parameters from the first SYN packet; other parameters change rapidly
  with time).

---------------------------------------------
Is there anything special about this program?
---------------------------------------------

  There is another passive OS detection utility, called 'siphon'.  It's
  a pretty good piece of proof-of-concept software, but it isn't
  perfect.  Well, p0f isn't perfect for sure, but features some
  improvements:

  - it's single-threaded and pretty clean,

  - works properly on Linuxes (siphon has a problem with bpf on 2.2), as
    well as on BSD systems and SunOS/Solaris,

  - has pretty large and detailed fingerprints database,

  - uses more information for fingerprinting (42 extra bits),

  - it's more accurate,

  - you can define your own filtering rules in the tcpdump flavour:
    p0f 'src host 1.2.3.4' or p0f 'gateway 1.2.3.4 and port 80', and
    listening interface (using option -i).

  What more?  Dunno. :) Simply, check it out.

------------
Not working!
------------

  Probably p0f isn't working well on every platform in the world.  First
  of all, you'll need libpcap 0.4 or newer; sometimes pcap.h is placed
  in /usr/include/pcap instead of /usr/include/ (for example, in the
  broken Red Hat 6.1 package).  In this case, simply issue:

       ln -s /usr/include/pcap/pcap.h /usr/include/
       ln -s /usr/include/pcap/net/bpf.h /usr/include/net/

  NOTE:  if p0f recognized the system incorrectly or cannot recognize it
  at all, please send the OS signature and system description to the
  author.  Thanks. :)

  Tested platforms:

  - NetBSD  
  - FreeBSD
    in the ports collection
  - OpenBSD
  - Linux 2.0/2.2/2.4
    http://www.stearns.org/p0f/
  - Solaris 2.6-2.7
  - LinuxPPC
    http://rpmfind.net/linux/RPM/linuxPPC/contrib/software/Applications/Networking/p0f-1.7-0.ppc.html
  - Win32 (experimental)
    http://www.stearns.org/p0f/ soon after the 1.8 release comes out.
  
  Requires: libpcap 0.4 or newer; GNU cc 2.7.x or newer; GNU make 3.7x or
            BSD make; GNU egrep (for proper Makefile processing)


-------------
MySQL support
-------------
p0f MySQL quickstart (Courtesy of Marion Mates, <Marion.Bates@Dartmouth.EDU> )

0. Install MySQL, set up account that p0f can use to write to the db,
launch MySQL and configure such that it can accept network connections
from the p0f account (unless your db server is the same as your p0f
collector). 

1. As per the README's instructions, edit p0f-mysql.conf (in the mysql
dir in the p0f install dir) to reflect the location and account/password
of your MySQL db. Copy this file to /etc or some other safe place. 

2. Create database in MySQL called "pof" (or whatever, as long as it
matches the db name specified in /etc/p0f-mysql.conf.

3. Create tables in p0f db to match the specifications in the db.sql
file (in the mysql dir in the p0f install dir). Copy and paste the
"CREATE" statements. I'm sure there is a way to just load that file into
MySQL and have it create the tables automatically, but I didn't bother
to look that up.  :)

4. make p0f-mysql && (sudo) make install

5. Populate the database with fingerprints (only do this once):

/usr/sbin/p0f -m /etc/p0f-mysql.conf -g /etc/p0f.fp

	Using -m and -g parameters both at the same time will insert
p0f.fp to your db. Afterwards -m parameter is enough to get fprints from
db, process them and put the results into db.

6. Start p0f. /etc/rc.d/init.d/p0f start .  If you'd rather start it by
hand, make sure you add the "-m /etc/p0f-mysql.conf" command line
parameter.

Tips:

The database contains two tables, os and pool. The os table stores the
fingerprint data (pulled from the p0f.fp file). The pool table stores
the actual data feeds. Pool has a field called "OS" which is an ID
number linking to the field "osid" in the os table. This sounds more
complicated than it is. 

Example: Here are three lines of data from the pool table.
 
mysql> select * from pool;
+-------------+------+----+-------+-------+----------------+
| IP          | Hops | OS | sport | dport | time           |
+-------------+------+----+-------+-------+----------------+
| 192.168.1.2 |    1 | 11 | 50324 |    80 | 20020513132823 |
| 192.168.1.3 |    1 | 90 | 43808 |    80 | 20020513133646 |
| 10.10.10.5  |    1 | 89 |  1615 |    80 | 20020513134509 |
+-------------+------+----+-------+-------+----------------+
3 rows in set (0.00 sec)


The visitor with IP 192.168.1.2 had an OS fingerprint with ID 11. (For
now, this corresponds to line numbers in p0f.fp, starting from the
bottom of the list. This will change soon.) So if I want to know what OS
that is, I look it up:

mysql> select * from os where osid = 11;
+------+---------------+-------+------+------+------+--------+------+------+------+
| osid | osname        | win   | ttl  | mss  | df   | wscale | sok  | nop  | size |
+------+---------------+-------+------+------+------+--------+------+------+------+
|   11 | Mac OS X 10.1 | 32768 |   64 | 1460 |    1 |      0 |    0 |    1 |   60 |
+------+---------------+-------+------+------+------+--------+------+------+------+
1 row in set (0.00 sec)


So, visitor 192.168.1.2 generated a Mac OS X 10.1 fingerprint.

You can combine these lookups in an easier way, by doing an inner join:

mysql> select pool.IP, pool.Hops, pool.sport, pool.dport, pool.time, os.osname from pool, os where os.osid = pool.OS;

+--------------+------+-------+-------+----------------+--------------------------+
| IP           | Hops | sport | dport | time           | osname                   |
+--------------+------+-------+-------+----------------+--------------------------+
| 192.168.1.2  |    1 | 50324 |    80 | 20020513132823 | Mac OS X 10.1            |
| 192.168.1.3  |    1 | 43808 |    80 | 20020513133646 | Linux 2.4.2 - 2.4.14 (1) |
| 10.10.10.5   |    1 |  1615 |    80 | 20020513134509 | Mac OS 9 (1)             |
+--------------+------+-------+-------+----------------+--------------------------+
3 rows in set (0.00 sec)


-------------
Configuration
-------------

  The database of OS fingerprints is usually kept in /etc/p0f.fp or
  ./p0f.fp .  Its format is described below:

#
# p0f - passive OS fingerprinting
# -------------------------------
# (C) Copyright 2000 by Michal Zalewski <lcamtuf@coredump.cx>
# Maintained by William Stearns <wstearns@pobox.com>
#
# Every entry in this file is a description of unique TCP parameters 
# specific for the first SYN packet sent by a remote party while 
# establishing a connection. Those parameters include: window size (wss),
# maximum segment size (mss), don't fragment flag (DF), window scaling 
# (wscale), sackOK flag, nop flag, initial time to live (TTL), declared
# SYN packet size.
#
# Normally, p0f reports unknown OSes providing you with all parameters,
# so you can simply find out what system your party runs, and then,
# add appropriate rule to this file. There's only thing you have to do
# - determine initial TTL of a packet. Well, usually it is equal to the first
# power of 2 greater than TTL you're seeing, given that your remote party is
# not too far away (if traceroute shows more than 20-25 hosts, be careful).
# So, for example, if you get TTL of 55 in a fingerprint returned by p0f,
# initial TTL probably was 64. NOTE: it is better to overestimate it (will
# affect distance prediction) than to underestimate (will not work at all in
# some cases).
#
# There are some brain-damaged devices, like network printers, that
# have initial TTLs set to values like 60. However, if you see HP LaserJet
# trying to connect your server, you probably should have a break :)
#
# Format:
#
# wwww:ttt:mmm:D:W:S:N:OS Description
#
# wwww - window size
# ttt  - time to live
# mmm  - maximum segment size
# D    - don't fragment flag  (0=unset, 1=set) 
# W    - window scaling (-1=not present, other=value)
# S    - sackOK flag (0=unset, 1=set)
# N    - nop flag (0=unset, 1=set)
# I    - declared packet size (-1 = irrelevant)
#

--------------------
What should be done?
--------------------

  - Colorful interface, of course ;)
  - Packet sizes added for old fingerprints
  - Other user-friendly features

-------------------
License, disclaimer
-------------------

  The p0f utility and related utilities are free software; you can
  redistribute it and/or modify it under the terms of the GNU Library
  General Public License as published by the Free Software Foundation;
  either version 2 of the License, or (at your option) any later
  version.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
  MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  NON-INFRINGEMENT.  IN NO EVENT SHALL MICHAL ZALEWSKI, OR ANY OTHER
  CONTRIBUTORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
  WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
  OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  THE SOFTWARE.

--------------------------
Logcheck Integration Notes
--------------------------

From: John Sage <jsage@finchhaven.com>
To: wstearns@pobox.com
Subject: Adding p0f to LogCheck report

I just put on your latest p0f version, after having enjoyed running the
previous version for several weeks.
  
Great little proggie!
  
I decided to see if I could add p0f awarness to LogCheck
(http://psionic.com/abacus/logcheck/) and it seems to have worked like a
charm:
 
 
Unusual System Events
=-=-=-=-=-=-=-=-=-=-=
Jan 22 07:46:16 greatwall snort: [1:0:0] Potential CodeRed/Nimda probe {TCP}
 a.b.c.d:3621 -> w.x.y.z:80
Jan 22 07:46:19 greatwall snort: [1:0:0] Potential CodeRed/Nimda probe {TCP}
 a.b.c.d:3621 -> w.x.y.z:80
 
Jan 22 07:46:16 greatwall kernel: Packet log: input DENY ppp0 PROTO=6 a.b.c.d:3621
 w.x.y.z:80 L=48 S=0x00 I=7278 F=0x4000 T=117 SYN (#62)
Jan 22 07:46:19 greatwall kernel: Packet log: input DENY ppp0 PROTO=6 a.b.c.d:3621
 w.x.y.z:80 L=48 S=0x00 I=7692 F=0x4000 T=117 SYN (#62)
 
<Tue Jan 22 07:46:16 2002> 12.149.215.61 [12 hops]: Windows 2000 (9)
 a.b.c.d:3621 -> w.x.y.z:80
<Tue Jan 22 07:46:19 2002> 12.149.215.61 [12 hops]: Windows 2000 (9)
 a.b.c.d:3621 -> w.x.y.z:80
 
 
In logcheck.sh I added:

#
$LOGTAIL /var/log/snort/p0f.log >> $TMPDIR/check.$$
#

and in logcheck.violations I added:

: UNKNOWN [
hops]:

which is the regex-like pattern that picks up the p0f entries..


- John

--------------------------------
Links to OS fingerprinting sites
--------------------------------

http://www.sys-security.com/html/papers.html
http://project.honeynet.org/papers/finger/
http://www.siteware.ch/webresources/useragents/db.html
http://www.linuxjournal.com/article.php?sid=4750
http://ettercap.sourceforge.net/
http://www.securiteam.com/securitynews/5NP0C153PI.html
http://www.securitybugware.org/mUNIXes/4680.html
http://www.cgisecurity.net/papers/fingerprinting-2.txt
http://www.team-teso.net/data/ldistfp-auth-fingerprints
http://www.team-teso.net/releases.php
	and, of course,
http://www.stearns.org/p0f/
	

