How do I use nmap XML?

Recently, in the nmap-dev mailing list, Fyodor asked:

In what ways do you use the Nmap XML output? Do you parse it from within a higher level program, transform it to HTML with XSLT, use it to populate a database, use XPath to parse the results from the command-line in a way that is as easy as awk/sec/cut/etc. on the normal output, or something else entirely?

I’ll share here my approach to nmap output parsing.

For my automated scans I use a combination of Python, Bash and AWK scripts. I always keep nmap scans in XML even if these will be used by some Bash/Awk scripts.

With Python I just parse the XML with libxml’s Python bindings.

With Bash and/or AWK I transform the XML output into PYX format with a custom made utility called xmltopyx.

For those not familiar with PYX, it is a way of converting XML documents into a more grep/AWK friendly format. More information about it can be found here and here.

An example of xmltopyx + AWK usage:

$ xmltopyx nmap-sample-tcpudp-portscan.xml | awk -f getports.awk
tcp 21 open ftp
tcp 22 open ssh
tcp 53 open domain
udp 53 open|filtered domain
tcp 111 open rpcbind
udp 111 open|filtered rpcbind
udp 608 open|filtered sift-uft
tcp 611 open npmp-gui
udp 636 open|filtered
tcp 639 open
udp 664 open|filtered
udp 667 open|filtered
tcp 670 open
tcp 953 open rndc
tcp 2049 open nfs
udp 2049 open|filtered nfs
tcp 3128 open squid-http
udp 3130 open|filtered squid-ipc
udp 3401 open|filtered squid-snmp
udp 4827 open|filtered squid-htcp
udp 32768 open|filtered omad
udp 32771 open|filtered sometimes-rpc6

Then, using getports.awk together with a while read proto port state service; do … ; done loop in Bash is very simple.