Fruit: Why you so low?
Network Recon 2011AD
Hack.lu 2011
Oh, Hi. ●
I'm Metlstorm (Adam to my mum) ●
●
I hack stuff. ●
●
Usually with python, bacon, vim, unix and beer.
Roll with Brett Moore's Insomnia Security ●
●
Based in Wellington, New Zealand
Previously of Immunity, Security-Assessment.com
On (double-award winning) weekly infosec news podcast Risky.biz
Proprietor, Kiwicon (est 2007)
^^^^ Still the best dressed hacker, even while in NZ!
Triforce Journey ●
This talk is nominally about Network Reconnaissance ●
●
But really, its about a journey
Three, entertwined journies ●
The LHKF project
●
Network reconnaissance as a whole
●
My journey, as a hacker
Network Reconnaissance ●
Traditional tools ●
Portscanners, banner grabbers, fingerprinting
●
Netcat, some-worm.c, commercial tools
●
●
Distributed ●
●
Nmap 5.x == state of the art; fast, flexible, applayer, scriptable Unicorn scan, RIP Jack.
Modern tools ●
Flexible, protocol layer scanning
●
Searchable web interface
Hang on, isnt this just V-A ●
Well, yes. But have you tried asking Qualys to scan a Class B? ●
Not only is it expensive, but your machine will die rendering the 50000 page pdf report, ha ha.
●
Ditto nessus or whatever
●
Metasploit + DB might... ●
●
But even New Zealand has 6.8M IPs. :/
None of the tools scale well
So I Wrote Another One ●
Geo-targeted network recon data acquisition system ● ●
●
●
With a web interface Automated, fire-and-forget-and-go-to-the-pub operation That scales properly
Changelog ●
v1.0 “Low Hanging Kiwifruit” for Kiwicon ]I[ ●
●
●
580k hosts in 6.2M IPs (.nz)
v2.0 “Low Scuttling Chillicrab” for SyScan 2010 ●
360k hosts in 4.8M IPs (.sg)
●
New acquisition engine
V2.1 “Now with added Luxembourg” ●
(also I accidentally a whole Belgium)
●
840k (.nz) + 414k (.be) + 52k (.lu)
●
New db schema, search engine
What's it good for? ●
Target location ●
Exploit-centric targeting (script kiddie-ing)
●
Pre-seeding your “warhol worm”
●
Scope expansions
●
National sitrep
●
In lieu of data breach disclosure laws
●
Security Consultancy
●
Lulz...
The Innards ●
●
●
v1.0 was an exercise to see how plausible it was to “just scan everything and grep” Nmap, python ghetto-queue, lotsa shellscripts, and manglethis2that.py glued together with some 1980s style curses gui. It looked something like this:
Re-enactment
The Innards ● ●
●
Which worked surprisingly well And taught me the necessary lessons about how to scale it up v2.0 re-engineered the acquisition portion ●
(pretty much a coupla weekend's work)
●
looks something like this
metlstrm@lhkf:~$ python >>> from lhkf.acquisition import scanCountry >>> scanCountry(“lu”, [22,23,25,80,110...])
lhkf.scanCountry(“sg”, [21,22,23,25,80...]) GeoIP Message Bus Target Generation
Queue
Bulk Scanner Pool
Queue
App Scanner Pool
Queue
Disk Grinding Pool
MongoDB MongoDB The Internets
Webserver
(Enterprise) Architecture ●
Hip, cloud web2.0 stylin'
●
MongoDB “nosql” main data store
●
Erlang/RabbitMQ message bus
●
Python/Celery MQ/Job dispatch engine ●
Workflow rules to sort everything out
●
PostgreSQL for relational data
●
Python/Django frontend
●
GridFS distributed filestore for bulk data (e.g. images)
Target Selection ●
What's a country in cyberspace? ● ●
●
Domains that end in .nz/.lu/.be? Netblocks announced at some domestic peering exchanges?
●
Address registry allocations?
●
GeoIP?
They're all valid answers, you just gotta pick ●
I chose GeoIP; outsource the problem to maxmind
●
Misses out dns names hosted overseas
●
Thats okay; simplifies our “jurisdictional issues”
Acquisition ●
High rate nmap TCP SYN scans, tuned well ●
●
Tried with unicorn scan; if anything its too fast, and sadly unmaintained
●
Typically sit at 4kpps (16 Class C/sec...)
●
Pushing 30kpps makes my ISP sad :(
Custom python protocol aware banner grabbing framework ●
●
plug in python libs, external binaries, Xservers, whatever necessary to get app data ~20 specific protocols at present, including “graphical banners”
Correlation ●
With DNS PTR
●
Address registry “whois” info
●
DNS
●
●
With DNS CNAME / A / MX / NS (NZ zone files)
●
Bing ip: lookup “unlimited API calls” :)
Store all historical data to track changes over time
Storage ●
●
(580k + 360k hosts) * avg 15 ports/host + applayer data ~= 1.4B rows. per scan refresh Classic data-mine style problem ●
●
Dataset is search/read heavy, very insert light, near zero updates.
●
Optimise for retreival; denormalise, index.
●
Relational DB wrong solution.
MongoDB “document store” database ●
Auto sharding/replicating to scale out
●
Easy as hell to use
Open Cast Data Mining ●
●
There is just, well, a lot of it. What do you want? ●
Old unix boxen?
●
Things with self-signed certs? Wildcard certs?
●
Cisco Switches? Blade chassis?
●
SunRPC services? Writable SMB shares?
●
.gov/.mil/.spooks?
Search by ●
Banners, SSL Cert DN, 302 targets, , and other protocol stuff (smb, ldap, mysql, mssql....)
IDS Avoidance ●
Corps spend mega fat-cash on IDSes and Security Operations Centres ●
●
So best be careful to avoid them, right?
One port at a time across the whole country, randomise ●
Tune for detection rate across above average netblock size (say, /16)
IDS Who-Gizzashit ●
Scanning .nz ●
●
7 abuse@ mails
Scanning .sg ●
1 abuse@ mail
●
And it was hilarious! –
●
(the “eCop” detected my “horizontal and vertical” scans!)
Scanning .lu, .be ●
No abuse mails :D
“Hack the planet!”
IDS Baiting ●
So, noone's watching, right? Hack the planet? ● ●
●
Just check out the DNS PTR backscatter if you don't believe me.
Portscans just aren't interesting in 2010AD ●
●
Not quite. People are watching.
So how do we make 'em interesting?
Pro Tip #437: Don't have a few beers on Friday night, then do this ......
...in-addr.arpa. IN PTR scanner03.ccip.govt.nz.
ewps.
Yeeeah, about that... ● ●
●
●
...don't. My poor ISP got a call from the spooks at 0910 Monday morning, Poor spooks probably had to fill out all sorts of forms, in triplicate. So apparently people are watching :) ●
Hi there!
IN PTR not.really.the.CCIP.terribly.sorry.about.the.confusion.
But Not Good For ●
●
Actually doing something about it ●
I did try, for a while
●
But like software full disclosure, it's a waste of time.
The Digital Pearl Harbour? ●
Open it up! Use it for hacker tourism!
●
Invite all the .tr and .br kidz to come own us all up!
●
All the low-hanging shit gets owned, it hurts for a bit, but eventually herd health will improve
●
Be a stronger, better high-tech economy
●
… yeah, no. :/
Breakin' the Law ●
Portscanning & preauth banner grabbing is pretty much legal in most jurisdictions ● ●
● ●
●
I obey all warning banners telling me to disconnect Scanner is tuned to avoid causing DoS to any single IP or netblocok
Aggregating & searching public data is legal Providing info that can be used to “access in excess of your authority” is possibly illegal in .nz, but there's no case law (and is also stupid) Making this data illegal only helps the badguys ●
Because they already have it.
However ●
I've chosen at this time not to make LHKF general public access ●
●
●
●
Instead, providing access on a case-by-case to infosec industry people, CERTs, .gov, and anyone who sounds legit enough to me. Like you guys, amirite? (l: haxor.lu p: giraffe)
I spose I could monetise it, but that sounds like actual work instead of fun And besides, there is already a public one of these...
What About Shodan ●
●
Shodan is the same thing, but with breadth rather than depth focus, and public ●
4 ports (21,22,23,80)
●
Whole world as target
LHKF approx contemporary with Shodan ●
●
●
Shodan went public ~4 days before LHKF did at Kiwicon 3
In terms of raw data, about similar size ●
My .sg + .nz ~= shodan's * in host/port tuples
●
But: .nz: shodan: 24k hosts, LHKF: 580k
Shodan's interface is much more hip, web2.0
So What Does It All Mean ●
Search engines are a force multiplier ●
●
Building a system like this is easy, fun and entirely too feasible ●
●
Public data + aggregation & search = power
Engineering time is a few weekends
If I have, others have ●
If you're a cyber*.mil and you don't have one of these, you're doing your cyber-thing wrong.
But isn't portscanning stuff just so 1997AD?
Network Recon ●
Recon matters ●
Active recon (scanning) less than it used to –
●
Easy to do
Passive recon (sniffing, traffic analysis) more than it used to – –
(And not N-IDS/IPS) Scales up well if you're a telco, IX, or intelligence agency
Passive ●
Sniff for C&C, data exfiltration from your net to detect compromise ●
●
Acquire botnet data from someone ●
DNS sinkholes (ala Shadowserver)
●
Darknets (ala CYMRU)
●
●
Something in your organisation is owned; anything else is statistically infeasible
Other shady crowds (Endgame, CyberEIS, Damballa, Unveillance)
Pretty much the only new tool in the defence arsenal lately
Targeting ●
Targeting is under-estimated; ●
Look at both Francois & Fred, Phillippe yesterday; both are powerful attack classes, facilitated by targeting.
Assertion: ●
Targeting info approaches 0day in value. ●
This is one of the things that made me stop and think...
Endgame.us pricelist from HBGary's mailspool (big kthx to aaron barr for awesome passwd management)
Value ●
25 x 0day = $2.5M
●
Botnet telemetry = $2M
●
Active recon info = $2M
●
And you get these all correlated.
Target Acquisition ●
Targeting is the main function Warehouse all the info, so you can search one db for each new tasking/target/mission ●
Find the thing you need to own –
●
Target org, its ISP, its outsourcer, its bank, its arms vendor, its scada vendor...
Or the thing you already own (same diff, really; given incremental cost of owning something) –
Or the thing some botnet owns, and that you can buy or steal
Vector ●
●
0day are bad weapons ●
Shelf life hard to predict
●
Every time you use it, you risk burning it
Utilising botnets makes more sense ●
More predictable/stable/weaponisable
●
Can outsource the crime to herders, JIT acquire
●
More efficient use of 0day (10s of k new hosts for a flash 0day, vs blowing your USB 0day on a single stuxnetting)
End game ●
●
A large scale recon map relating: ●
Target organisations
●
Their trust partners
●
Vulnerability
●
Existing compromises to reuse
== massive force multiplier
The Personal Journey ●
●
I'm a trad hacker; unix, networks, enterprise apps, trust expansion The world has changed around us ●
Its not about “this box is vulnerable to statdx” –
●
Its “your operational patch management policy is bad”
I thought scanning whole countries was pretty bad-ass 4-5 years ago. ●
I was wrong. It's passé. Everyone does it.
●
But why its relevant now is … “cyber”.
Cyber, the verb. ●
Cyber changes everything ●
●
●
●
Traditional private sector infosec - AV, pentests, code reviews, arch reviews, policy Is irrelevant in the world of Stuxnet, of massive state-sponsored cyber-espionage, of Diginotar, of multi-terabit of BGP rerouting into .cn. We simply cannot defend against multi-million dollar offensive tech budgets Plus, all the talent, bugs, info is being vacuumed up into the cyber-mil-industrial complex –
And if you dont...
So I whittled a giraffe I hope you like it. www.lowhangingkiwifruit.com ●
Go explore .lu, .be and .nz.
●
Creds are:
● ●
●
Login: haxor.lu / Pass: giraffe
●
It'll be live for a week or two
Be good, don't use your powers for evil The performance will probably suck with everyone using it, so be patient too
KTHX & Questions Good luck. You'll need it.
metlstorm (at) storm.net.nz Also, come to Kiwicon V in Wellington, New Zealand Nov 5-6 2011
