Linuxworld Toronto 2007 Presentation Open Source S

Open Source Security Tools : 

Open Source Security Tools Linux World / Network Toronto Tutorial Part II

Agenda: 

Agenda Review of Security Applications Building a complete NIDS reporting/analysis system Snort Installation Snort Operation BASE Installation and operation Building a complete Vulnerability Assessment System Nesssus/OpenVAS Installation Nessus/Open VAS Operation NCC and the Birth of an Open Source Project Graphical Wireless Auditing with Netstumbler/Stumbverter The changing face of Open Source Software

Your Presenter: 

Your Presenter Tony Howlett President of Network Security Services, Inc. CISSP, GSNA Author of “Open Source Security Tools” 18 years of experience building and managing networks

Pre-speech Caveats: 

Pre-speech Caveats Contains updates to material in book; a preview of 2nd edition OS Requirements; Linux or BSD, Reference OS is Mandrake 10.1 Choice of Security apps are my favorites, yes there are others, these are MY favorites Please feel free to attempt installation during talk, but don’t ask me to wait as you catch up or ask OS specific installation questions during the talk Please hold questions till of seminar

Security Tool Warnings: 

Security Tool Warnings DO NOT run these tools on any machines that don’t belong to you without WRITTEN permission Be very careful when running them on production machines at work Failure to abide by this could result in getting fired, legal action or losing your ISP connection.

Network Intrusion Detection Systems: 

Network Intrusion Detection Systems NIDS systems are complicated sniffers that check all data packets on the network against a database of signatures Some IDS systems also do anomalous activity detection and Intrusion Prevention (taking action on the results)

Building a Complete Intrusion Analysis and Reporting System: 

Building a Complete Intrusion Analysis and Reporting System Required Elements: IDS Database back-end for alerts Web front-end for analysis and reporting

Slide8: 

A powerful, lightweight open source NIDS Large default rule set (several thousand) Highly customizable and configurable Offers a scripting language to write custom rules to match your network Works on Linux/BSD/Windows

Installing Libpcap: 

Installing Libpcap Packet capture library required for network sniffing operations Must be installed before Snort can work Download from: www.tcpdump.org

Installing the Pig: 

Installing the Pig Un-tar source files Enter the following commands: ./configure make make install

Snort Operation Modes: 

Snort Operation Modes Packet Sniffer Mode Packet Logger Mode Intrusion Detection Mode

Packet Sniffer Mode: 

Packet Sniffer Mode Similar to Tcpdump or any other sniffer Useful to test to make sure snort is seeing packets Type “snort –v” at command line Other switches -d displays application layer -e displays datalink layer

Packet Logging Mode: 

Packet Logging Mode Same as packet sniffer mode but output is logged Use switch “-l logfile” where logfile is pathname and file to log to Additional switch “-h homenet” where homenet identifies your home network so that Snort will log all packets relative to it

Intrusion Detection Mode: 

Intrusion Detection Mode Where we start using a intrusion signature rule set to decide what gets logged/alerted Same as packet logging mode except you add “-c configfile” where configfile is the path and name of your Snort config file By default, located in /etc/snort/snort.conf

The Snort Config file: 

The Snort Config file Key variables to set: Home_net Internal servers Output modules – decide how you want Snort to do with the output Syslog Database (mySQL, oracle, postgreSQL) Unified Alert Format

Other Snort Settings: 

Other Snort Settings Snort Decoders and Preprocessors Handle high volume processing and other special handling not done by rule sets Examples: Ports scans, etc Tampering with these is not for the light hearted but can give you big performance increases

The Snort Rules Files: 

The Snort Rules Files Similar to signatures in an antivirus program Each rule file contains multiple statements that the sensor tries to match to network traffic When a match occurs, Snorts takes action (logs, alerts, etc) Separated into classes, each is a different file called by config file Backdoor.rule, bad-traffic.rule, chat.rule, ddos.rule, dos.rule, icmp.rule, etc

Disabling/Writing Snort Rules: 

Disabling/Writing Snort Rules To disable whole classes of rules, simple place a # in front of the line for that class in the snort.conf file To disable a single rule, find it within that .rule file and place a comment in front of it Or use a graphical interface such as Snort Webmin To add a custom rule, simply insert the rule into the appropriate class (using the right syntax of course)

Implementation tips: 

Implementation tips Store logs on different server (separate sensors and log database) for better performance and security Use a database for output and optional analysis tools Fine tune your rule sets Place several sensors at different points around network (DMZ, Internal, Outside the firewall) Defense in depth: Use Snort with other defense measures (firewall, anti virus, anti spam ,etc)

Fine-tuning your pig: 

Fine-tuning your pig Lets snort run for a few days wide open and see what the network is generating. Analyze what is normal traffic and what is abnormal for your network Spend some time on this, it is crucial to having a useful NIDS

Snort Analysis and Report Tools: 

Snort Analysis and Report Tools This is where BASE comes in Snort is virtually useless in the long term without this program or one like it.

Basic Analysis and Security Engine (BASE): 

Basic Analysis and Security Engine (BASE) Formerly Analysis Console for Intrusion Detection (ACID) Web front end and database interface for Snort This program allows you to really tap into the power of Snort Imports alerts into a database and allows viewing through a web-based interface This allows analysis of your Snort data and helps with tuning your Snort sensors

BASE Requirements: 

BASE Requirements MySQL, PostGreSQL or Oracle database PHP-enabled web server One or more snort sensors to gather data from Important: You must build your Snort boxes with the “--with-mysql” or appropriate parameter and edit the snort.conf files correctly. Download BASE from: http://secureideas.sourceforge.net/

Installing BASE: 

Installing BASE Install Apache, PHP, MySQL or other DB server as well as at least one Snort Sensor Edit the Snort.conf file to give the parameters for your database so the snort box will insert its alerts in the database It’s a good idea to run Base on a separate box from your sensor For better security, the logs don’t reside on the sensor.. Can even be a different location For better performance. With multiple sensors the Snort box will bog down the database Download and untar the program Place the untarred files in your /www/htdocs directory or appropriate subdirectory below that

Installing BASE: 

Installing BASE Edit the Base.conf.php file The lines starting with $ are variables you need to define. The comment lines will give you information on how to fill these out Enter the appropriate variables and save the file

BASE Variables: 

BASE Variables $DBType Type of database you are using $alert_dbname The name of the database Snort uses $alert_host Hostname or IP of the database $alert_port Port used by the database $alert_user The database user that BASE will log the data with $alert_password Password for the same

Other BASE Variables: 

Other BASE Variables $Archive_database Same as main alert db but for archiving $chart_files Optional module used for reports (must have jpgraph loaded)

BASE Logical Layout: 

BASE Logical Layout

Installing BASE: 

Installing BASE Once your variables are in place, save the file Go to the URL in your browser Click on “Create AG” and BASE will create the database for your sensors Go to the URL to see the data on your sensors You can protect these pages with Basic Authentication and .htaccess files

BASE Demo: 

BASE Demo

Ways to Use BASE Data: 

Ways to Use BASE Data Who’s Being Attacked? (run a VA scan on those boxes, make sure they are hardened) Whose doing the attacking? (block those IPs at the firewall, file complaints with their ISPs) What’s being attacked (check most popular protocol used, possibility block those at firewall) Follow Multi-day trends (24 hour/72 hour) for worst attackers

Building a Complete Vulnerability Assessment Analysis & Reporting system: 

Building a Complete Vulnerability Assessment Analysis & Reporting system                                                                           Required Elements: A VAS scanner An Analysis program A Reporting program

Benefits of OpenVAS and other Nessus Forks: 

Benefits of OpenVAS and other Nessus Forks                                                                           Client-Server architecture. Supports multiple platforms for clients (Linux, Windows, Web-based, Java) Has a built in scripting language (NASL) for writing custom security tests Well documented and supported from legacy Best of all, absolutely FREE! For now, we use Nessus 2.2.5, the last open source version

Nessus / OpenVAS Requirements: 

Nessus / OpenVAS Requirements                                                                           LibPcap Libraries (www.tcpdump.org) Gimp Took Kit (GTK) ftp.gimp.org/pub/gtk/v1.2 Nmap (if you want to use it as your port scanner) OpenSSL (if you want to connect securely www.openssl.org)

Installing Nessus 2.2.5: 

Installing Nessus 2.2.5                                                                           Download the software package from www.nessus.org If its not available on that site, look on mirrors or from the CD in the book There are four separate tar files Nessus-libraries Libnasl Nessus-core Nessus-plug-ins You must untar and install each module in the order above Using the following commands: ./configure make make install

Installing Nessus 2.2.5: 

Installing Nessus 2.2.5          Put the following in your path statement: /usr/local/sbin:/usr/local/bin Edit a file called ld.so.conf in /etc and add the following line /usr/local/lib Then type “ldconfig”

Installing Nessus 2.2.5: 

Installing Nessus 2.2.5 Creating the certificates and initializing the installation Type “nessus-mkcert” This creates a certificate for your Nessus installation Type “nessus-adduser” This will create an initial user to give you access to the Nessus server

Using Nessus 2.2.5: 

Using Nessus 2.2.5 Fire up the Nessus server by typing ./nessusd & to run it as a daemon Fire up the Unix Client by typing “./nessus” Or Use NessusWX, the window client to access the server Use one of the web clients out there (NCC which is next)

NCC and the Birth of an Open Source Project: 

NCC and the Birth of an Open Source Project                                                                           We had a need that wasn’t fulfilled by commercial software or open source alternatives Needs Management platform for multiple VA Scans Web based interface for scheduling and running VA scans Recurrence Different customer sets Web based reporting mechanism Database back end

The Birth of an Open Source Project: 

The Birth of an Open Source Project                                                                           Questions to ask: Is there something already out there? Look on Sourceforge, Google, mailing lists Is there a broader need for your program (otherwise may as well develop internally) Do you have permission to release code as open source (if doing work on company time or under a non compete)

Benefits of OS development: 

Benefits of OS development                                                                           Building on existing programs to cut down development time (version.9 of NCC used NPI, an open source PHP interface to Nessus) Use outside talent (larger set of developers) Give back to the OS community 

Downsides of OS development: 

Downsides of OS development                                                                           Can be frustrating. (like herding cats) Might take longer than traditional development cycle Can be chaotic and unpredictable (use OS tools to manage your project)

Open Source Platform Decisions: 

Open Source Platform Decisions                                                                           LAMP Linux for OS Apache for web servcies MySQL for database Perl/PHP (LAMPP?)

Other useful OS tools: 

Other useful OS tools                                                                           CVS or Subversion (SVN) for version control Crucial with a large number of developers Allows roll back, maintains integrity and consistency of code versions

Netscan Command Console (NCC): 

Netscan Command Console (NCC) A web based interface and database backend for managing multiple .nsr-based vulnerability scans Allows for scheduling of future scans, recurrence and storing of results in a database Web based analysis tool for results Multi-user, multi group (good for consultants) Version 2.1 coming out in July (sneak preview today!) Co-written by yours truly 

NCC Elements: 

NCC Elements                                                                           Perl Scripts to do interaction with VAS Scanner (simply using command line interface) Check schedules Start scans Import results into database Update schedules, queue Sends out herald and report complete emails

NCC Elements: 

NCC Elements                                                                           MySQL database to store VAS scan results and schedule and report data Table Structure: Groups (Supports virtual interfaces for consultants/partners) Companies (to allow for each company to have their own interface) Users: 7 different user levels (system admin to regular user) Targets (finite groups of target IPs with scan config) Schedules (one time, recurring, how long) Scans False Positives Reports (false positives, Scan history)

NCC Elements: 

NCC Elements                                                                           PHP scripts to create web interface for database Main.php for main interface Sub pages for each menu choice Scan summary page Reports page PDF generation

NCC Elements: 

NCC Elements                                                                           Text files Install.pl Install script README, INSTALL, LICENSE, etc (someone has to write all those files)

NCC Logical Diagram: 

NCC Logical Diagram                                                                          

Netscan Command Console (NCC): 

Netscan Command Console (NCC) Written using Perl and PHP Based on the LAMP platform (Linux, Apache, MySQL and PHP (Perl too!) Backwards compatible with OS Nessus Requirements: MySQL: Version 3.2352 or higher PHP: Version 4.32 or higher Perl: Version 5.8 or higher OpenVAS/Nessus: Version 2.07 or higher Apache: Version 2.0.47 or higher Get it at www.netsecuritysvcs.com/ncc

NCC version 2.1: 

NCC version 2.1 Major improvements including Completely rewritten PHP interface (no longer based on NPI) Better queue management New System reporting capabilities (usage & activity reports) .nbe,.nsr importing capabilities (for internal scans) Many bugs flushed out Due out in July… Sneak preview today!

Installing NCC: 

Installing NCC First Steps: Install apache Install PHP Install Perl Install MySQL Nessus or other .nsr based VAS (between version 2.0 & 3.0, not higher) The rest is easy!

NCC Variables: 

NCC Variables Before running the install script make sure you have the values of the following variables ready: NCC user: a system OS user that will run NCC functions (recommended that you create a special user for NCC) Install directory NCC admin email “From” address banner for results (important for spam filters) Host name or IP Name of your MySQL server

NCC Variables: 

NCC Variables Name of the NCC database to create MySQL user ( create a valid user beforehand) MySQL password (Security warning!) Hostname or IP of Nessus server Nessus Port (default is 1241) Nessus user name Nessus Password (security warning!) Nessus Path (weather or not you run Nessus locally, you will need the executables locally for the client and import process) Temp directory (where result files are staged for importation)

NCC Variables: 

NCC Variables Name of the NCC Admin user and password Chose this carefully as this will be the administrator of the whole installation and cant be changed afterwards

Installing NCC: 

Installing NCC Run install.pl It checks for the prereqs., particularly the Perl modules needed for NCC) Initializes database and copies file into the appropriate places Answer the prompts that come up and the install scripts will complete the installation

Installing NCC: 

Installing NCC Once the install script has run successfully, create a symbolic link from your public web directories to the root NCC install directory This will connected your public html directories to the public NCC files as well as protect non public NCC files You are now ready to start using NCC

NCC Demo: 

NCC Demo

Using NCC: 

Using NCC Main screen give you user level, group and company membership and activity summary Menus are user level sensitive Quick buttons (new for version 2.0)for access to commonly used functions QuickScan (don’t have to create a new company, user, etc) View Reports

System Admin Functions: 

System Admin Functions Group Management (separate directories for the images etc) Users Management Queue Management System Reports (new in version 2.0!) Import Functions (new in version 2.0!)

Group Management Functions: 

Group Management Functions For virtual Interfaces Company management (add, modify, delete) Users management (add, modify delete)

Company Management Functions: 

Company Management Functions User Management Target Management Schedule Management

User functions: 

User functions Available to all user levels Manage Profile (change password, address, etc) View reports (only of your membership level)

Wireless Networking: 

Wireless Networking Great! Access from anywhere! Terrible.. Access to your network from.. Anywhere! All your firewalls and external defenses can be undone by one unsecured access point About 30% of access points surveyed are unsecured Down from last year but still high Even APs using security can be cracked (WEP only) In some areas, over saturation can lead to signal degradation loss It makes sense to do a wireless survey of your “airspace” for both security and wireless network performance

The Dangers of Wifi: 

The Dangers of Wifi Eavesdropping (wireless sniffing) Access to wireless PCs (once they have an IP, they can be attacked, liking having a public IP (no firewall between you and attackers Access to your internal LAN for further exploration Anonymous Internet access (great for hacking, virus release, spam)

The perimeter before Wifi: 

The perimeter before Wifi

The perimeter after Wifi: 

The perimeter after Wifi

802.11x Specific Vulnerabilities: 

802.11x Specific Vulnerabilities Default SSIDs (you can tell the owner didn’t do much to configure it, bonus, you are automatically logged on) Beacon Broadcast (allows us to find it regardless of SSID) Unencrypted by default (getting better) Weak encryption protocol (also getting better with WPA, LEAP)

Creating a Graphical Wireless Survey: 

Creating a Graphical Wireless Survey Goal: to graphically show the active Wifi signals in the area and their various attributes (signal strength, encryption, etc) Good for seeing saturation of signals, finding rogue/unsecured access points

Tools Needed:: 

Tools Needed: Laptop or palmtop with Wifi card A GPS receiver with PC input Note: Netstumbler only supports serial interface, newer laptops will need a USB-Serial bridge Netstumbler Software Stumbverter Software MS Mappoint 2004 or later

NetStumber: 

NetStumber Not purely Open Source, but Freeware License Available at www.netstumbler.com along with lots of other good wireless resources Gives lots of useful information (signal strengths, type of equipment, AP settings, GPS data) Requirements: Windows W2K or later Prism2, Hermes or Atmel chip-based Wifi card Optional external antenna for better results Optional GPS receiver for map data

Stumbverter: 

Stumbverter Open Source add-on to Netstumbler; also works with Kismet files Converts your Netstumbler files captured with GPS data into MS Map Point Makes a nice graphical representation of the APs in your area Good for finding areas of signal interference, channel choices Surveying for rogue access points Requirements: Netstumbler GPS sensor with PC interface Licensed version of MS Map Point 2004 or later Get it at http://www.sonar-security.com/sv.html

Do your survey: 

Do your survey Roam the Area with Netstumbler on and your GPS plugged and enabled Use a palmtop if you need more mobility Use an external antenna for better reception In car or on foot… go beyond where you think the best reception would be Save the .nsi file

Import into Stumverter: 

Import into Stumverter Start up Stumbverter Chose “New Map” and create a new North America Map Chose Import and select your .nsi file to be imported It will show APs on the map with Signal Strength (the number of bars emanating from it Encryption (green or red color) Other detail (SSID, MAC, etc) by clicking on it

Slide76: 

Stumbverter Map

Example Wifi Survey: 

Example Wifi Survey Area surrounding a local mall Yes, we had permission, our client was the developer of the property Mall is a perfect location for war driving Surrounded by high rise buildings (several owned by our client) Hospital, businesses inside the mall, residential neighbor hood Easy to park for hours and gather data to crack WEP keys

What we found…: 

What we found… 103 Access points in the immediate area 32 access points within their building 57 without encryption (42%) 42 unsecured by anything (31% allowed surfing, and other activity) SSIDs of “emergency” “statcare” “surgery” Clients WEP key was cracked in 15.5. hours They have since switched to Enterprise WPA

Tips for Effective and Ethical Wireless Auditing: 

Tips for Effective and Ethical Wireless Auditing Get permission (legality of war driving is still vague) Inform your neighbors of any unsecured access points Audit under multiple conditions (rain, fog, winter, summer can affect results)

Open Source Software Needs: 

Open Source Software Needs Opportunities to contribute and gain experience and reputation Better Windows Tools! Encryption Windows VAS Windows Snort tools OpenVAS project or side projects (like NCC!) Better Unix wireless tools

The Changing Face of Open Source Software: 

The Changing Face of Open Source Software IT Providers (both hardware and software) have embraced the open source concept Why? Its trendy Lower development costs Makes them appear to be less “proprietary” Some have done it as a defensive move (Oracle, Microsoft)

The Changing Face of Open Source Software: 

The Changing Face of Open Source Software All the major hardware manufactures now support Linux on their machines Some make a major marketing effort behind it (HP, IBM (who’d a thought?) Embedded Linux (the penguin in your cell phone, gas pump, etc) Is this a good thing???

The Changing Face of Open Source Software: 

The Changing Face of Open Source Software Pros: More money for development of OS Promotes corporate adoption Raises legitimacy of the OS platforms Cons In some cases, future versions are being closed source or released under more restrictive licenses Driving quality developers from OS? No longer the free love, fight the man movement. The penguin has been coopted!

REPEATSecurity Tool Warnings: 

REPEAT Security Tool Warnings DO NOT run these tools on any machines that don’t belong to you without WRITTEN permission Be very careful when running them on production machines at work Failure to abide by this could result in getting fired, legal action or losing your ISP connection.

Questions / Comments?: 

Questions / Comments? Email me at thowlett@netsecuritysvcs.com Most of the tools mentioned and more can be found in my book “Open Source Security Tools”, available from your bookseller or online