Snort uses rules to analyze network traffic discover potential threats or network anomalies. Alerts can be dispatched to an analyst or trigger remediation scripts or other actions.

Snort is widely used by Blue Teams protecting networks of all sizes and is considered a robust part of network security infrastructure. Cisco purchased the snort project in 2013 and incorporated it in its Sourcefire line of products. The core snort software remains open source with a GPL2+ license.

Common Use Cases for Snort

Snort can be used in a variety of scenarios to protect networks from cyber threats. Some practical use cases for Snort include:

Detecting and blocking network attacks

Snort can be used to detect and block network-based attacks, such as denial of service (DoS) attacks, SQL injection or network service attacks such as the well known ETERNALBLUE exploit. Snort will analyze network traffic in real-time, alerting and potentially taking action to prevent the attack from succeeding.

Monitoring network traffic for suspicious activity

Snort can be used to monitor network traffic for any suspicious activity, such as an unusually high amount of traffic; think multiple Microsoft Remote Desktop (RDP) logins or High number of HTTP POST requests. This can help identify potential security threats allowing the network administrator assess a potential incident.

Detecting and blocking malware

Snort can be configured to use a set of rules that are designed to detect known implants or malware signatures. Common examples would be Cobal Strike (installer / C2 traffic) and the Metasploit based Meterpreter. When malware is detected, Snort can alert the network administrator or trigger actions to mitigate damage from the malware.

These are the most common use cases for a snort deployment. It should be kept in mind that due to the ability to create custom rules, the possibilities for what Snort can monitor and alert on is endless.

Installing Snort 2.9 on Ubuntu

In order to get started with Snort easily, we recommend starting with Snort 2.9 which is available in the Ubuntu 22.04 repositories. Installation is a simple matter of the standard apt-get install.

:-$ sudo apt install snort

Using this method ensures you have a production ready version that is easy to maintain and update when required through the standard update processes.

:-$ snort --version

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.15.1 GRE (Build 15125) 
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014-2019 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.10.1 (with TPACKET_V3)
           Using PCRE version: 8.39 2016-06-14
           Using ZLIB version: 1.2.11

Snort 3 with Docker

Using the Cisco Talos docker container is the fastest way to get Snort 3 up and running. Primarily suited for initial testing, the docker container has a full snort installation and can be used to quickly process a network capture (pcap) within a few minutes.

Snort 3 comes with a number of new capabilities and features. Jump in with the following docker commands.

:-$ sudo docker pull ciscotalos/snort3
:-$ sudo docker run --name snort3 -h snort3 -u snorty -w /home/snorty -d -it ciscotalos/snort3 bash
:-$ sudo docker exec -it snort3 bash

Installing Snort 3 on Ubuntu

As snort 3 does not come as packaged binaries it is necessary to install from source to deploy on Ubuntu.

The full installation guide is available from the snort.org website. Specifically for Ubuntu deployments you will need the following required packages.

:-$ sudo apt install build-essential libpcap-dev libpcre3-dev libnet1-dev zlib1g-dev luajit hwloc libdnet-dev \
libdumbnet-dev bison flex liblzma-dev openssl libssl-dev pkg-config libhwloc-dev cmake cpputest libsqlite3-dev uuid-dev \
libcmocka-dev libnetfilter-queue-dev libmnl-dev autotools-dev libluajit-5.1-dev libunwind-dev libfl-dev

Getting the Rules

The rules can be downloaded from snort.org and are available as the Community Rule set, as well as the official Cisco rules. The official rules require a free registration (30 day delay) or a paid subscription for immediate access to newly released rules.

While the community rules are an excellent resource the official rules are essential for getting good coverage and registration or a subscription should be done.

In addition there are excellent rules available from Emerging Threats (Proofpoint) with the option of Free or a Paid for offering.

Oinkcodes - Automate Rule Downloads

The Oinkcode is an API key associated with a registered account. Using the oinkcode you are able to access the rule updates programatically using a tool such as Pulled Pork.

Working Snort 3 Installation

Whichever version or method you are using running the following confirms that snort is installed and ready to go:

snorty@snort3:~$ snort --version

   ,,_     -*> Snort++ <*-
  o"  )~   Version 3.0.0 (Build 267)
   ''''    By Martin Roesch & The Snort Team
           http://snort.org/contact#team
           Copyright (C) 2014-2019 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using DAQ version 3.0.0
           Using LuaJIT version 2.1.0-beta3
           Using OpenSSL 1.1.1d  10 Sep 2019
           Using libpcap version 1.8.1
           Using PCRE version 8.39 2016-06-14
           Using ZLIB version 1.2.11
           Using Hyperscan version 5.1.0 2019-01-31
           Using LZMA version 5.2.4

snorty@snort3:~$ 

Practical Examples

These examples show a number of practical uses for snort as a command line tool and demonstrates how the system works in a hands on capacity.

1. Capture on Local Interface with Snort

In this mode, Snort reads packets from the network interface and compares them to the set of rules specified in the configuration file.

:~$ snort -c /etc/snort/snort.conf -i eth0

2. Analyse Packets from a PCAP File

You can use Snort to read packets from a PCAP file.

:~$ snort -r file.pcap -c /etc/snort/snort.conf

3. Test Snort Configuration File

This command tests your Snort configuration and rules for errors.

:~$ snort -T -c /etc/snort/snort.conf

4. Log Traffic to a pcap File

Output options are configured in the snort.conf file. Logging to pcap can be configured in the file or we can use the command line option below to write the pcap.

Read packets from the configured network interface and write to a pcap file.

:~$ snort -b -L packets.pcap

5. A simple test rule to ensure Snort is working as expected

To test everything is working and to understand how the alerting / logging works lets create a simple rule that we can trigger at any time.

Edit the file /etc/snort/rules/local.rules and put the following line at the end.

alert icmp any any -> any any (msg:"ICMP connection attempt"; sid:1000010; rev:1;)

This rule will detect any use of the icmp protocol (second entry in rule). That matches the source / dest (any -> any variable), and will then use the msg: as the alert text.

The following example is a bit different to previous. It says to print the alerts to the console (-A console) and uses the (-q) parameter to be quiet. Stopping the debugging and startup information from appearing and providing clean output. We can specify the local.rules file as the config or the snort.conf (as it should be including the local.rules file).

:~$ snort -q -A console -c /etc/snort/local.rules

If you ping the host or network that is listening you should see the alerts printed to the console.

05/25-10:50:00.887913  [**] [1:1000002:1] ICMP connection attempt [**] [Priority: 0] {ICMP} 10.1.1.33 -> 10.1.1.44
05/25-10:50:00.888003  [**] [1:1000002:1] ICMP connection attempt [**] [Priority: 0] {ICMP} 10.1.1.44 -> 10.1.1.33

6. Reject and Drop Rules

Using our previous test rule for icmp we are able to demonstrate the drop and reject options for rules. To demostrate we will simply replace the alert with reject. The sid will also be incremented otherwise there will be an error when starting with two rules with the same sid.

reject icmp any any -> $HOME_NET any (msg:"ICMP connection attempt"; sid:1000011; rev:1;)

Restarting snort and running the same ping -c 2 10.1.1.44 we will receieve the following output:

:~$ ping -c 2 10.1.1.44
PING 10.1.1.44 (10.1.1.44) 56(84) bytes of data.
64 bytes from 10.1.1.44: icmp_seq=1 ttl=64 time=1.25 ms
From 10.1.1.44 icmp_seq=1 Destination Port Unreachable

The first packet gets a response, however the subsequent packet is rejected with an icmp port unreachable.

The rule options are available here -> http://manual.snort.org/node29.html

Using the reject option causes snort to send a TCP reset or an ICMP port unreachable packet, that will break the session. Using drop and sdrop will only work if Snort is running inline as it does as advertised and simply will drop the packets in this mode.

7. Filtering on the Command Line with BPF

Similar to tcpdump we can provide BPF filters on the command line to limit the traffic we are inspecting and capturing. The following example limits captured traffic to a single host, that can be the source or destination.

:~$ snort -q -A console -c /etc/snort/snort.conf host 10.1.1.33

8. Enable app-detect.rules and Know the Network

After copying the official rules into the /etc/snort/rules/, quite a lot of rules are actually disabled. This is due to the fact that the default configuration is trying to balance alert noise vs coverage. It is up to the administrator to enable many of the rules.

An interesting set of rules to look at when getting started is the app-detect.rules these detect many types of application on the network - many of those that have remote control features often used by attackers but also legitimitaly.

:~$ sudo grep app-detect /etc/snort/snort.conf 
#include $RULE_PATH/app-detect.rules

Firstly the configuration file has the rule file disabled. Furthermore the app-detect.rules rules are disabled by default.

# alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"APP-DETECT VNC server response"; flow:established; content:"RFB 0"; depth:5; content:".0"; depth:2; offset:7; metadata:ruleset community; classtype:misc-activity; sid:560; rev:9;)

This is an interesting rule, VNC is an application that allows GUI access to a console. While VNC can be used by administrators it is also used by attackers. An example is the payloads for VNC found within Metasploit.

So this is an example of the app-detect.rules that we want to enable by removing the '#' from the start of the line.

9. Enable malware rules

Another set of rules that are disabled by default in the Ubuntu package are the malware-rules. We want to enable these as they will provide coverage of attacker favorites such as Cobalt Strike beacons or installers.

:~$ sudo grep malware /etc/snort/snort.conf 
#include $RULE_PATH/malware-backdoor.rules
#include $RULE_PATH/malware-cnc.rules
#include $RULE_PATH/malware-other.rules
#include $RULE_PATH/malware-tools.rules

Remove the comment from the start of these lines to enable the use of the malware rules.

These rules contain detections for interesting tools such as Cobalt Strike and Meterpreter. If these are triggering on the internal network you will certainly want to know about it.

Conclusion

Snort has been around for 25 years and is still a powerful and effective tool for those who defend networks from threats. The above tutorial and examples are not intended to cover everything but to give you a practical starting point from which to build up your Snort skillset and build some key knowledge for when planning a deployment.

Even if you do not plan on throwing it on a network immediately, being able to quickly spin up a docker container or an install can be very helpful. Run it over some pcaps from the network or an incident and you may just find some bread crumbs to follow.

In recent years the trend has moved from Network Intrusion Detection (nids) to Endpoint Detection and Response (edr). This makes sense with increasingly encrypted network traffic. However, snort and other network tools still give visibility to a great deal of interestings on the wire and not everything runs an EDR client.

The post Snort Tutorial and Practical Examples appeared first on HackerTarget.com.

In this recon-ng tutorial, discover open source intelligence and easily pivot to new results. Using a modular approach, collect and dig deeper into extracted data.

What is Recon-ng?

Recon-ng is a reconnaissance / OSINT tool with an interface similar to Metasploit. Running recon-ng from the command line speeds up the recon process as it automates gathering information from open sources.

Recon-ng has a variety of options to configure, perform recon, and output results to different report types.

Recon-ng Installation

Installing Recon-ng is very simple and there are a few common ways. Below are a few examples;

Kali:

At the time of this article version 5.1.2 comes pre-installed with Kali Linux. Having said that, its good to run apt-get update && apt-get install recon-ng to ensure latest dependencies installed.

Ubuntu:

Requires git and pip installed.

test@ubuntu:~/$ git clone https://github.com/lanmaster53/recon-ng.git
test@ubuntu:~/$ cd recon-ng
test@ubuntu:~/recon-ng/$ pip install -r REQUIREMENTS

Next to run recon-ng;

test@ubuntu:~/recon-ng/$ ./recon-ng

The Recon-NG console is now loaded.

    _/_/_/    _/_/_/_/    _/_/_/    _/_/_/    _/      _/            _/      _/    _/_/_/
   _/    _/  _/        _/        _/      _/  _/_/    _/            _/_/    _/  _/       
  _/_/_/    _/_/_/    _/        _/      _/  _/  _/  _/  _/_/_/_/  _/  _/  _/  _/  _/_/_/
 _/    _/  _/        _/        _/      _/  _/    _/_/            _/    _/_/  _/      _/ 
_/    _/  _/_/_/_/    _/_/_/    _/_/_/    _/      _/            _/      _/    _/_/_/    
                                                                                        

                                          /\
                                         / \\ /\
    Sponsored by...               /\  /\/  \\V  \/\
                                 / \\/ // \\\\\ \\ \/\
                                // // BLACK HILLS \/ \\
                               www.blackhillsinfosec.com

                  ____   ____   ____   ____ _____ _  ____   ____  ____
                 |____] | ___/ |____| |       |   | |____  |____ |
                 |      |   \_ |    | |____   |   |  ____| |____ |____
                                   www.practisec.com

                      [recon-ng v5.1.2, Tim Tomes (@lanmaster53)]                       

[*] No modules enabled/installed.

[recon-ng][default] > 

Using recon-ng

From the console it is easy to get help and get started with your recon.

[recon-ng][default] > help

Commands (type [help|?] ):
---------------------------------
back            Exits the current context
dashboard       Displays a summary of activity
db              Interfaces with the workspace's database
exit            Exits the framework
help            Displays this menu
index           Creates a module index (dev only)
keys            Manages third party resource credentials
marketplace     Interfaces with the module marketplace
modules         Interfaces with installed modules
options         Manages the current context options
pdb             Starts a Python Debugger session (dev only)
script          Records and executes command scripts
shell           Executes shell commands
show            Shows various framework items
snapshots       Manages workspace snapshots
spool           Spools output to a file
workspaces      Manages workspaces

Recon-ng begins with an empty framework. No modules enabled or installed.

[*] No modules enabled/installed.

How to use Recon-ng:

Create a Workspace

There is a lot of options when using this OSINT tool. Maintaining collected information and notes organised is a necessary part of any OSINT investigation. Creating a workspaces keeps things orderly and easy to find. When using Recon-ng workspaces, all data located and collected is saved within a database in that workspace.

[recon-ng][default] >  workspaces create example_name 

[recon-ng][default] > workspaces create example_name
[recon-ng][example_name] > 

The command recon-ng -w example_name opens or returns directly to that workspace.

test@ubuntu:~/$ recon-ng -w example_name 

[recon-ng][example_name] > 

Recon-ng Marketplace and Modules

Here again the help comes in handy marketplace help shows commands for removing modules, how to find more info, search, refresh and install.

[recon-ng][default] > marketplace help
Interfaces with the module marketplace

Usage: marketplace info|install|refresh|remove|search [...] 

Typing marketplace search displays a list of all the modules. From which you can start following the white rabbit exploring and getting deeper into recon and open source intelligence.

Recon-ng modules

Modules are grouped together under various categories and can be found searching on marketplace

- discovery
- exploitation
- import
- recon
- reporting

Each of the above have sub categories as shown in the table below. Use marketplace search for a full table providing information on version, status (installed or not-installed), date updated, dependencies or require keys.

[recon-ng][example_name] > marketplace search

  +---------------------------------------------------------------------------------------------------+
  |                        Path                        | Version |     Status    |  Updated   | D | K |
  +---------------------------------------------------------------------------------------------------+
  | discovery/info_disclosure/cache_snoop              | 1.1     | not installed | 2020-10-13 |   |   |
  | discovery/info_disclosure/interesting_files        | 1.2     | not installed | 2021-10-04 |   |   |
  | exploitation/injection/command_injector            | 1.0     | not installed | 2019-06-24 |   |   |
  | exploitation/injection/xpath_bruter                | 1.2     | not installed | 2019-10-08 |   |   |
  | import/csv_file                                    | 1.1     | not installed | 2019-08-09 |   |   |
  | import/list                                        | 1.1     | not installed | 2019-06-24 |   |   |
  | import/masscan                                     | 1.0     | not installed | 2020-04-07 |   |   |
  | import/nmap                                        | 1.1     | not installed | 2020-10-06 |   |   |
  | recon/companies-contacts/bing_linkedin_cache       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/companies-contacts/censys_email_address      | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-contacts/pen                       | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/companies-domains/censys_subdomains          | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/companies-domains/pen                        | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/companies-domains/viewdns_reverse_whois      | 1.1     | not installed | 2021-08-24 |   |   |
  | recon/companies-domains/whoxy_dns                  | 1.1     | not installed | 2020-06-17 |   | * |
  | recon/companies-hosts/censys_org                   | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-hosts/censys_tls_subjects          | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/companies-multi/github_miner                 | 1.1     | not installed | 2020-05-15 |   | * |
  | recon/companies-multi/shodan_org                   | 1.1     | not installed | 2020-07-01 | * | * |
  | recon/companies-multi/whois_miner                  | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/contacts-contacts/abc                        | 1.0     | not installed | 2019-10-11 | * |   |
  | recon/contacts-contacts/mailtester                 | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/contacts-contacts/mangle                     | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/contacts-contacts/unmangle                   | 1.1     | not installed | 2019-10-27 |   |   |
  | recon/contacts-credentials/hibp_breach             | 1.2     | not installed | 2019-09-10 |   | * |
  | recon/contacts-credentials/hibp_paste              | 1.1     | not installed | 2019-09-10 |   | * |
  | recon/contacts-domains/migrate_contacts            | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/contacts-profiles/fullcontact                | 1.1     | not installed | 2019-07-24 |   | * |
  | recon/credentials-credentials/adobe                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/credentials-credentials/bozocrack            | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/credentials-credentials/hashes_org           | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-companies/censys_companies           | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/domains-companies/pen                        | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/domains-companies/whoxy_whois                | 1.1     | not installed | 2020-06-24 |   | * |
  | recon/domains-contacts/hunter_io                   | 1.3     | not installed | 2020-04-14 |   | * |
  | recon/domains-contacts/metacrawler                 | 1.1     | not installed | 2019-06-24 | * |   |
  | recon/domains-contacts/pen                         | 1.1     | not installed | 2019-10-15 |   |   |
  | recon/domains-contacts/pgp_search                  | 1.4     | not installed | 2019-10-16 |   |   |
  | recon/domains-contacts/whois_pocs                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-contacts/wikileaker                  | 1.0     | not installed | 2020-04-08 |   |   |
  | recon/domains-credentials/pwnedlist/account_creds  | 1.0     | not installed | 2019-06-24 | * | * |
  | recon/domains-credentials/pwnedlist/api_usage      | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-credentials/pwnedlist/domain_creds   | 1.0     | not installed | 2019-06-24 | * | * |
  | recon/domains-credentials/pwnedlist/domain_ispwned | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-credentials/pwnedlist/leak_lookup    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-credentials/pwnedlist/leaks_dump     | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-domains/brute_suffix                 | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/domains-hosts/binaryedge                     | 1.2     | not installed | 2020-06-18 |   | * |
  | recon/domains-hosts/bing_domain_api                | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/domains-hosts/bing_domain_web                | 1.1     | not installed | 2019-07-04 |   |   |
  | recon/domains-hosts/brute_hosts                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/builtwith                      | 1.1     | not installed | 2021-08-24 |   | * |
  | recon/domains-hosts/censys_domain                  | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/domains-hosts/certificate_transparency       | 1.2     | not installed | 2019-09-16 |   |   |
  | recon/domains-hosts/google_site_web                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/hackertarget                   | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/domains-hosts/mx_spf_ip                      | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/netcraft                       | 1.1     | not installed | 2020-02-05 |   |   |
  | recon/domains-hosts/shodan_hostname                | 1.1     | not installed | 2020-07-01 | * | * |
  | recon/domains-hosts/spyse_subdomains               | 1.1     | not installed | 2021-08-24 |   | * |
  | recon/domains-hosts/ssl_san                        | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/threatcrowd                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-hosts/threatminer                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/domains-vulnerabilities/ghdb                 | 1.1     | not installed | 2019-06-26 |   |   |
  | recon/domains-vulnerabilities/xssed                | 1.1     | not installed | 2020-10-18 |   |   |
  | recon/hosts-domains/migrate_hosts                  | 1.1     | not installed | 2020-05-17 |   |   |
  | recon/hosts-hosts/bing_ip                          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-hosts/censys_hostname                  | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/censys_ip                        | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/censys_query                     | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/hosts-hosts/ipinfodb                         | 1.2     | not installed | 2021-08-24 |   | * |
  | recon/hosts-hosts/ipstack                          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-hosts/resolve                          | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/reverse_resolve                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/ssltools                         | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/virustotal                       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-locations/migrate_hosts                | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-ports/binaryedge                       | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/hosts-ports/shodan_ip                        | 1.2     | not installed | 2020-07-01 | * | * |
  | recon/locations-locations/geocode                  | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-locations/reverse_geocode          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-pushpins/flickr                    | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/locations-pushpins/shodan                    | 1.1     | not installed | 2020-07-07 | * | * |
  | recon/locations-pushpins/twitter                   | 1.1     | not installed | 2019-10-17 |   | * |
  | recon/locations-pushpins/youtube                   | 1.2     | not installed | 2020-09-02 |   | * |
  | recon/netblocks-companies/censys_netblock_company  | 2.0     | not installed | 2021-05-11 | * | * |
  | recon/netblocks-companies/whois_orgs               | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-hosts/censys_netblock              | 2.0     | not installed | 2021-05-10 | * | * |
  | recon/netblocks-hosts/reverse_resolve              | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-hosts/shodan_net                   | 1.2     | not installed | 2020-07-21 | * | * |
  | recon/netblocks-hosts/virustotal                   | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/netblocks-ports/census_2012                  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/netblocks-ports/censysio                     | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/ports-hosts/migrate_ports                    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/ports-hosts/ssl_scan                         | 1.1     | not installed | 2021-08-24 |   |   |
  | recon/profiles-contacts/bing_linkedin_contacts     | 1.2     | not installed | 2021-08-24 |   | * |
  | recon/profiles-contacts/dev_diver                  | 1.1     | not installed | 2020-05-15 |   |   |
  | recon/profiles-contacts/github_users               | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/namechk                    | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/profiler                   | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/profiles-profiles/twitter_mentioned          | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-profiles/twitter_mentions           | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/profiles-repositories/github_repos           | 1.1     | not installed | 2020-05-15 |   | * |
  | recon/repositories-profiles/github_commits         | 1.0     | not installed | 2019-06-24 |   | * |
  | recon/repositories-vulnerabilities/gists_search    | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/repositories-vulnerabilities/github_dorks    | 1.0     | not installed | 2019-06-24 |   | * |
  | reporting/csv                                      | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/html                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/json                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/list                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/proxifier                                | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/pushpin                                  | 1.0     | not installed | 2019-06-24 |   | * |
  | reporting/xlsx                                     | 1.0     | not installed | 2019-06-24 |   |   |
  | reporting/xml                                      | 1.1     | not installed | 2019-06-24 |   |   |
  +---------------------------------------------------------------------------------------------------+

  D = Has dependencies. See info for details.
  K = Requires keys. See info for details.

Marketplace search brings up the full table, however you can be more specific in your search, a couple of examples

recon-ng][default] >marketplace search ssl
[*] Searching module index for 'ssl'...

  +----------------------------------------------------------------------------+
  |             Path            | Version |     Status    |  Updated   | D | K |
  +----------------------------------------------------------------------------+
  | recon/domains-hosts/ssl_san | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/hosts-hosts/ssltools  | 1.0     | not installed | 2019-06-24 |   |   |
  | recon/ports-hosts/ssl_scan  | 1.1     | not installed | 2021-08-24 |   |   |
  +----------------------------------------------------------------------------+

  D = Has dependencies. See info for details.
  K = Requires keys. See info for details.

[recon-ng][default] > 

To find out more info on a specific module

[recon-ng][default] > marketplace info ssltools 

  +---------------------------------------------------------------------------------------+
  | path          | recon/hosts-hosts/ssltools                                                                                                                                                                                 |
  | name          | SSLTools.com Host Name Lookups                                                                                                                                                                             |
  | author        | Tim Maletic (borrowing from the ssl_san module by Zach Graces)                                                                                                                                             |
  | version       | 1.0                                                                                                                                                                                                        |
  | last_updated  | 2019-06-24                                                                                                                                                                                                 |
  | description   | Uses the ssltools.com site to obtain host names from a site's SSL certificate metadata to update the 'hosts' table.  Security issues with the certificate trust are pushed to the 'vulnerabilities' table. |
  | required_keys | []                                                                                                                                                                                                         |
  | dependencies  | []                                                                                                                                                                                                         |
  | files         | []                                                                                                                                                                                                         |
  | status        | not installed                                                                                                                                                                                              |
  +------------------------------------------------------------------------------------+

[recon-ng][default] > 

As noted above Hackertarget has a module. This will be used as an example on how to use recon-ng.

Recon-ng example

As an example on how to use Recon-ng, hackertarget has a module to gather subdomains recon/domains-hosts/hackertarget. This module uses the Hackertarget API and hostname search.

Install module

To install this module use the following:

[recon-ng][default] > marketplace install hackertarget
[*] Module installed: recon/domains-hosts/hackertarget
[*] Reloading modules...
[recon-ng][default] > 

Load module

[recon-ng][default] > modules load hackertarget
[recon-ng][default][hackertarget] > 

Module Help

The help command from within a loaded module has different options to the global 'help'.
When you are ready to explore more modules use 'back'.

[recon-ng][default][hackertarget] > help

Commands (type [help|?] ):
---------------------------------
back            Exits the current context
dashboard       Displays a summary of activity
db              Interfaces with the workspace's database
exit            Exits the framework
goptions        Manages the global context options
help            Displays this menu
info            Shows details about the loaded module
input           Shows inputs based on the source option
keys            Manages third party resource credentials
modules         Interfaces with installed modules
options         Manages the current context options
pdb             Starts a Python Debugger session (dev only)
reload          Reloads the loaded module
run             Runs the loaded module
script          Records and executes command scripts
shell           Executes shell commands
show            Shows various framework items
spool           Spools output to a file

[recon-ng][default][hackertarget] > 

Set source

Using show options, brings a table showing the source current value set at default.

[recon-ng][default][hackertarget] > show options

  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  default        yes       source of input (see 'show info' for details)

Now, set the source to the name of the domain investigating. This example uses tesla.com as they have a published big bounty.

Use command options set SOURCE tesla.com

[recon-ng][default][hackertarget] > options set SOURCE tesla.com
SOURCE => tesla.com

Use command info. This shows current value has changed to tesla.com

[recon-ng][default][hackertarget] > info

Options:
  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  tesla.com      yes       source of input (see 'info' for details)

Source Options:
  default      SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
  string       string representing a single input
  path         path to a file containing a list of inputs
  query sql    database query returning one column of inputs

Use input to see

[recon-ng][default][hackertarget] > input

  +---------------+
  | Module Inputs |
  +---------------+
  | tesla.com     |
  +---------------+

Run the module

Type run to execute the module.

[recon-ng][default][hackertarget] > run

---------
TESLA.COM
---------
[*] Host: tesla.com
[*] Ip_Address: 104.119.104.74
[*] --------------------------------------------------
[*] Host: o7.ptr6980.tesla.com
[*] Ip_Address: 149.72.144.42
[*] --------------------------------------------------
[*] Host: vpn1.tesla.com
[*] Ip_Address: 8.45.124.215
[*] --------------------------------------------------
[*] Host: apacvpn1.tesla.com
[*] Ip_Address: 8.244.131.215
[*] --------------------------------------------------
[*] Host: cnvpn1.tesla.com
[*] Ip_Address: 114.141.176.215
[*] --------------------------------------------------
[*] Host: vpn2.tesla.com
[*] Ip_Address: 8.47.24.215
[*] --------------------------------------------------
[*] Host: model3.tesla.com
[*] Ip_Address: 205.234.27.221
[*] --------------------------------------------------
[*] Host: o3.ptr1444.tesla.com
[*] Ip_Address: 149.72.152.236
[*] --------------------------------------------------
[*] Host: o2.ptr556.tesla.com
[*] Ip_Address: 149.72.134.64
[*] --------------------------------------------------
[*] Host: o5.ptr8466.tesla.com
[*] Ip_Address: 149.72.172.170
[*] --------------------------------------------------
[*] Host: o6.ptr9437.tesla.com
[*] Ip_Address: 168.245.123.10
[*] --------------------------------------------------
[*] Host: o4.ptr1867.tesla.com
[*] Ip_Address: 149.72.163.58
[*] --------------------------------------------------
[*] Host: marketing.tesla.com
[*] Ip_Address: 13.111.47.196
[*] --------------------------------------------------
[*] Host: o1.ptr2410.link.tesla.com
[*] Ip_Address: 149.72.247.52
[*] --------------------------------------------------
[*] Host: referral.tesla.com
[*] Ip_Address: 72.10.32.90
[*] --------------------------------------------------
[*] Host: mta2.email.tesla.com
[*] Ip_Address: 13.111.4.231
[*] --------------------------------------------------
[*] Host: mta.email.tesla.com
[*] Ip_Address: 13.111.14.190
[*] --------------------------------------------------
[*] Host: xmail.tesla.com
[*] Ip_Address: 204.74.99.100
[*] --------------------------------------------------
[*] Host: comparison.tesla.com
[*] Ip_Address: 64.125.183.133
[*] --------------------------------------------------
[*] Host: apacvpn.tesla.com
[*] Ip_Address: 8.244.67.215
[*] --------------------------------------------------
[*] Host: cnvpn.tesla.com
[*] Ip_Address: 103.222.41.215
[*] --------------------------------------------------
[*] Host: emails.tesla.com
[*] Ip_Address: 13.111.18.27
[*] --------------------------------------------------
[*] Host: mta2.emails.tesla.com
[*] Ip_Address: 13.111.88.1
[*] --------------------------------------------------
[*] Host: mta3.emails.tesla.com
[*] Ip_Address: 13.111.88.2
[*] --------------------------------------------------
[*] Host: mta4.emails.tesla.com
[*] Ip_Address: 13.111.88.52
[*] --------------------------------------------------
[*] Host: mta5.emails.tesla.com
[*] Ip_Address: 13.111.88.53
[*] --------------------------------------------------
[*] Host: mta.emails.tesla.com
[*] Ip_Address: 13.111.62.118
[*] --------------------------------------------------
[*] Host: click.emails.tesla.com
[*] Ip_Address: 13.111.48.179
[*] --------------------------------------------------
[*] Host: view.emails.tesla.com
[*] Ip_Address: 13.111.49.179
[*] --------------------------------------------------
[*] Host: itanswers.tesla.com
[*] Ip_Address: 204.74.99.100
[*] --------------------------------------------------
[*] Host: events.tesla.com
[*] Ip_Address: 13.111.47.195
[*] --------------------------------------------------
[*] Host: www-uat.tesla.com
[*] Ip_Address: 199.66.9.47
[*] --------------------------------------------------
[*] Host: shop.eu.tesla.com
[*] Ip_Address: 205.234.27.221
[*] --------------------------------------------------
[*] Host: mfamobile-dev.tesla.com
[*] Ip_Address: 205.234.27.209
[*] --------------------------------------------------
[*] Host: mfauser-dev.tesla.com
[*] Ip_Address: 205.234.27.209
[*] --------------------------------------------------


-------
SUMMARY
-------
[*] 35 total (35 new) hosts found.

Show hosts

Now we have begun to populate our hosts. Typing show hosts will give you a summary of the resources discovered.

[recon-ng][default][hackertarget] > show hosts
 +----------------------------------------------------------------------------------------------------------------------+
  | rowid |            host         |    ip_address   | region | country | latitude | longitude | notes |    module    |
  +----------------------------------------------------------------------------------------------------------------------+
  | 1   | tesla.com                 | 104.119.104.74  |        |         |          |           |       | hackertarget |
  | 2   | o7.ptr6980.tesla.com      | 149.72.144.42   |        |         |          |           |       | hackertarget |
  | 3   | vpn1.tesla.com            | 8.45.124.215    |        |         |          |           |       | hackertarget |
  | 4   | apacvpn1.tesla.com        | 8.244.131.215   |        |         |          |           |       | hackertarget |
  | 5   | cnvpn1.tesla.com          | 114.141.176.215 |        |         |          |           |       | hackertarget |
  | 6   | vpn2.tesla.com            | 8.47.24.215     |        |         |          |           |       | hackertarget |
  | 7   | model3.tesla.com          | 205.234.27.221  |        |         |          |           |       | hackertarget |
  | 8   | o3.ptr1444.tesla.com      | 149.72.152.236  |        |         |          |           |       | hackertarget |
  | 9   | o2.ptr556.tesla.com       | 149.72.134.64   |        |         |          |           |       | hackertarget |
  | 10  | o5.ptr8466.tesla.com      | 149.72.172.170  |        |         |          |           |       | hackertarget |
  | 11  | o6.ptr9437.tesla.com      | 168.245.123.10  |        |         |          |           |       | hackertarget |
  | 12  | o4.ptr1867.tesla.com      | 149.72.163.58   |        |         |          |           |       | hackertarget |
  | 13  | marketing.tesla.com       | 13.111.47.196   |        |         |          |           |       | hackertarget |
  | 14  | o1.ptr2410.link.tesla.com | 149.72.247.52   |        |         |          |           |       | hackertarget |
  | 15  | referral.tesla.com        | 72.10.32.90     |        |         |          |           |       | hackertarget |
  | 16  | mta2.email.tesla.com      | 13.111.4.231    |        |         |          |           |       | hackertarget |
  | 17  | mta.email.tesla.com       | 13.111.14.190   |        |         |          |           |       | hackertarget |
  | 18  | xmail.tesla.com           | 204.74.99.100   |        |         |          |           |       | hackertarget |
  | 19  | comparison.tesla.com      | 64.125.183.133  |        |         |          |           |       | hackertarget |
  | 20  | apacvpn.tesla.com         | 8.244.67.215    |        |         |          |           |       | hackertarget |
  | 21  | cnvpn.tesla.com           | 103.222.41.215  |        |         |          |           |       | hackertarget |
  | 22  | emails.tesla.com          | 13.111.18.27    |        |         |          |           |       | hackertarget |
  | 23  | mta2.emails.tesla.com     | 13.111.88.1     |        |         |          |           |       | hackertarget |
  | 24  | mta3.emails.tesla.com     | 13.111.88.2     |        |         |          |           |       | hackertarget |
  | 25  | mta4.emails.tesla.com     | 13.111.88.52    |        |         |          |           |       | hackertarget |
  | 26  | mta5.emails.tesla.com     | 13.111.88.53    |        |         |          |           |       | hackertarget |
  | 27  | mta.emails.tesla.com      | 13.111.62.118   |        |         |          |           |       | hackertarget |
  | 28  | click.emails.tesla.com    | 13.111.48.179   |        |         |          |           |       | hackertarget |
  | 29  | view.emails.tesla.com     | 13.111.49.179   |        |         |          |           |       | hackertarget |
  | 30  | itanswers.tesla.com       | 204.74.99.100   |        |         |          |           |       | hackertarget |
  | 31  | events.tesla.com          | 13.111.47.195   |        |         |          |           |       | hackertarget |
  | 32  | www-uat.tesla.com         | 199.66.9.47     |        |         |          |           |       | hackertarget |
  | 33  | shop.eu.tesla.com         | 205.234.27.221  |        |         |          |           |       | hackertarget |
  | 34  | mfamobile-dev.tesla.com   | 205.234.27.209  |        |         |          |           |       | hackertarget |
  | 35  | mfauser-dev.tesla.com     | 205.234.27.209  |        |         |          |           |       | hackertarget |
  +----------------------------------------------------------------------------------------------------------------------+

[*] 35 rows returned

[recon-ng][default][hackertarget] > 

--------------------------------------------------------------

Add API keys to Recon-ng

It is a simple matter to add API keys to recon-ng. Shodan with a PRO account is a highly recommended option. This will enable queries to open ports on your discovered hosts without sending any packets to the target systems.

How to add shodan API key

Create or login to your Shodan account, Go to 'Account" in top right corner. The API Key is listed here on the Account Overview page.

Recon-ng shows the syntax to add an API key is below

[recon-ng][default] > keys add 
Adds/Updates a third party resource credential

Usage: keys add name value

[recon-ng][default] keys add shodan_api bbexampleapikey33 

.recon-ng configuration files

When you install recon-ng on your machine, it creates a folder in your home directory called .recon-ng. Contained in this folder is keys.db. If you are upgrading from one version to another or changed computers, and have previous modules that require keys to work, copy this file from the old version on your system and move it on the new one. You do not have to start all over again.

test@test-desktop:~/.recon-ng$ ls

keys.db  
modules  
modules.yml  
workspaces

test@test-desktop:~/.recon-ng$ 

Conclusion

Recon-ng is a powerful tool that can be further explored by viewing the list of modules. The help within the console is clear, and with a bit of playing around it won't take long to become an expert.

The rise of bug bounties allows you to play with new tools and explore Organizations' every expanding attack surface footprint. Have fun. Don't break the rules.

The post Recon-NG Tutorial appeared first on HackerTarget.com.

Gobuster Installation

Written in the Go language, this tool enumerates hidden files along with the remote directories. Using the command line it is simple to install and run on Ubuntu 20.04.

For version 2 its as simple as:

$ sudo apt install gobuster 

The Linux package may not be the latest version of Gobuster. Check Repology: the packaging hub, which shows the package of Gobuster is 2.0.1 (at the time of this article). The Github repository shows a newer version V3.1.0. https://github.com/OJ/gobuster.git

Under "Easy installation" on the github page the options to install are binary releases, a Go install, and Building from source. For this install lets play around with the Go install. Gobuster needs Go to be at least v1.16

Setting up a Go environment (optional)

Download the GO install from here: https://go.dev/dl/

change to the directory where Downloads normally arrive and do the following;

--> extract
$ sudo tar xvzf go1.17.7.linux-amd64.tar.gz 
--> change permissions
$ sudo chown -R root:root ./go
--> move to local directory
$ sudo mv -v go /usr/local

A local environment variable called $GOPATH needs to be set up. Since Go 1.8 this is not essential, though still recommended as some third party tools are still dependent on it.

Add the following to the .bash_profile Locate in home directory with ls -la .

export GOPATH=/usr/local/go
export PATH=$PATH:/usr/local/go/bin

To check its all worked and the Go environment is set up:

$ go version
go version go1.17.7 linux/amd64 

Now with the Go environment confirmed. Its simply a matter of using the following command to install Gobuster.

$ go install github.com/OJ/gobuster/v3@latest

check Gobuster is installed with:

$ gobuster version
3.1.0

How to use Gobuster

Gobuster is now installed and ready to use. The rest of the tutorial is how to use Gobuster to brute force for files and directories.

Gobuster modes and flags

Gobuster has a variety of modes/commands to use as shown below. This tutorial focuses on 3: DIR, DNS, and VHOST.

To see a general list of commands use: gobuster -h Each of these modes then has its own set of flags available for different uses of the tool.

 $ gobuster -h 
Usage:
  gobuster [command]

Available commands:
  dir         Uses directory/file enumeration mode
  dns         Uses DNS subdomain enumeration mode
  fuzz        Uses fuzzing mode
  help        Help about any command
  s3          Uses aws bucket enumeration mode
  version     shows the current version
  vhost       Uses VHOST enumeration mode

Flags: 
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
  -h, --help              help for gobuster
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patters
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist
  

Wordlists

Gobuster needs wordlists. One of the essential flags for gobuster is -w . Wordlists can be obtained from various places. Depending on the individual setup, wordlists may be preinstalled or found within other packages, including wordlists from Dirb or Dirbuster. The ultimate source and "Pentesters friend" is SecLists - https://github.com/danielmiessler/SecLists which is a compilation of numerous lists held in one location.

Gobuster DIR command

The DIR mode is used for finding hidden directories and files.

To find additional flags available to use gobuster dir --help

$ gobuster dir --help

Uses directory/file enumeration mode

Usage:
  gobuster dir [flags]

Flags:
  -f, --add-slash                       Append / to each request
  -c, --cookies string                  Cookies to use for the requests
  -d, --discover-backup                 Upon finding a file search for backup files
      --exclude-length ints             exclude the following content length (completely ignores the status). Supply multiple times to exclude multiple sizes.
  -e, --expanded                        Expanded mode, print full URLs
  -x, --extensions string               File extension(s) to search for
  -r, --follow-redirect                 Follow redirects
  -H, --headers stringArray             Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2'
  -h, --help                            help for dir
      --hide-length                     Hide the length of the body in the output
  -m, --method string                   Use the following HTTP method (default "GET")
  -n, --no-status                       Don't print status codes
  -k, --no-tls-validation               Skip TLS certificate verification
  -P, --password string                 Password for Basic Auth
      --proxy string                    Proxy to use for requests [http(s)://host:port]
      --random-agent                    Use a random User-Agent string
  -s, --status-codes string             Positive status codes (will be overwritten with status-codes-blacklist if set)
  -b, --status-codes-blacklist string   Negative status codes (will override status-codes if set) (default "404")
      --timeout duration                HTTP Timeout (default 10s)
  -u, --url string                      The target URL 
  -a, --useragent string                Set the User-Agent string (default "gobuster/3.1.0")
  -U, --username string                 Username for Basic Auth
      --wildcard                        Force continued operation when wildcard found

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

Flags

The 2 flags required to run a basic scan are -u -w. This example uses common.txt from the SecList wordlists.

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/common.txt  

Example results
===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://example.com
[+] Method:                  GET
[+] Threads:                 10
[+] Wordlist:                /wordlists/Discovery/Web-Content/common.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.1.0
[+] Timeout:                 10s
===============================================================
2022/03/01 10:34:16 Starting gobuster in directory enumeration mode
===============================================================
/assets              
/css                  
/download             

Not too many results and was quite heavy on the system processess. Results depend on the wordlist selected. It is worth working out which one is best for the job. The length of time depends on how large the wordlist is. It can also be worth creating a wordlist specific to the job at hand using a variety of resources.

Threads

Gobuster is fast, with hundreds of requests being sent using the default 10 threads. This speeds can create problems with the system it is running on. It could be beneficial to drop this down to 4.

Additionally it can be helpful to use the flag --delay duration Time each thread waits between requests (e.g. 1500ms). For example --delay 1s in other words, if threads is set to 4 and --delay to 1s, this will send 4 requests per second.

$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt -t 4 --delay 1s -o results.txt

Results

Results are shown in the terminal, or use the -o option to output results to a file example -o results.txt

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/directory-list-2.3-small.txt -t 4 --delay 1s -o results.txt

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:                     https://example.co.uk/
[+] Method:                  GET
[+] Threads:                 4
[+] Delay:                   1s
[+] Wordlist:                /wordlists/Discovery/Web-Content/directory-list-2.3-small.txt
[+] Negative Status codes:   404
[+] User Agent:              gobuster/3.1.0
[+] Timeout:                 10s
===============================================================
2022/03/08 12:12:19 Starting gobuster in directory enumeration mode
===============================================================
/admin
/aux
===============================================================
2022/03/08 12:46:57 Finished
===============================================================

Took a while, but by filtering the results to an output file its easy to see and retain for future enumerating, what was located. . A few more interesting results this time.

Other DIR flag examples

The results above show status codes. To exclude status codes use  -n 

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt  -n  -t 4 --delay 1s -o results.txt

An example of another flag to use is the  -x  File extension(s) to search for. This is for the times when a search for specific file extension or extensions is specified. Such as, -x .php or other only is required.

user@matrix:$ gobuster dir -u https://example.com -w /wordlists/Discovery/Web-Content/big.txt  -x .php, .txt  -t 4 --delay 1s -o results.txt

Continue enumerating

Continue to enumerate results to find as much information as possible. Run gobuster again with the results found and see what else appears. Keep digging to locate those hidden directories.

$ gobuster dir -u https://example.com/aux -w /wordlists/Discovery/Web-Content/big.txt -t 4 --delay 1s -o results.txt

Gobuster DNS command

Use the DNS command to discover subdomains with Gobuster. To see the options and flags available specifically for the DNS command use: gobuster dns --help

user@matrix:$ gobuster dns --help
Uses DNS subdomain enumeration mode

Usage:
  gobuster dns [flags]

Flags:
  -d, --domain string      The target domain
  -h, --help               help for dns
  -r, --resolver string    Use custom DNS server (format server.com or server.com:port)
  -c, --show-cname         Show CNAME records (cannot be used with '-i' option)
  -i, --show-ips           Show IP addresses
      --timeout duration   DNS resolver timeout (default 1s)
      --wildcard           Force continued operation when wildcard found

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

DNS example

$ gobuster dns -q -r 8.8.8.8 -d example.com -w wordlists/Discovery/DNS/subdomains-top1million-5000.txt -t 4 --delay 1s -o results.txt"	 

Breaking this down.

dns mode
-q --quiet : Don't print the banner and other noise
-r --resolver string : Use custom DNS server (format server.com or server.com:port)
-d --domain string
-w --wordlist string : Path to the wordlist
-t --threads
--delay -- delay duration
-o --output string : Output file to write results to (defaults to stdout)

Using another of the Seclists wordlists /wordlists/Discovery/DNS/subdomains-top1million-5000.txt.

Results

In this case, as the flag -q for quiet mode was used, only the results are shown, the Gobuster banner and other information are removed.

Found: www.example.com
Found: nagios.example.com
Found: dev.example.com   
Found: auto.example.com                                

The same search without the flag -q obviously gives the same results - and includes the banner information.

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Domain:     example.com
[+] Threads:    4
[+] Delay:      1s
[+] Resolver:   8.8.8.8
[+] Timeout:    1s
[+] Wordlist:   /home/wordlists/subdomains-top1million-5000.txt
===============================================================
2022/03/18 16:20:35 Starting gobuster in DNS enumeration mode
===============================================================

Found: www.example.com
Found: nagios.example.com
Found: dev.example.com   
Found: auto.example.com  
                              
===============================================================
2022/03/18 16:20:37 Finished
===============================================================

Gobuster VHost command

The vhost command discovers Virtual host names on target web servers. Virtual hosting is a technique for hosting multiple domain names on a single server.

Exposing hostnames on a server may reveal supplementary web content belonging to the target. Vhost checks if the subdomains exist by visiting the formed URL and cross-checking the IP address.

To brute-force virtual hosts, use the same wordlists as for DNS brute-forcing subdomains.

Similar to brute forcing subdomains eg. url = example.com, vhost looks for dev.example.com or beta.example.com etc.

For options and flags available use gobuster vhost --help

user@matrix:$ gobuster vhost --help

Uses VHOST enumeration mode

Usage:
  gobuster vhost [flags]

Flags:
  -c, --cookies string        Cookies to use for the requests
  -r, --follow-redirect       Follow redirects
  -H, --headers stringArray   Specify HTTP headers, -H 'Header1: val1' -H 'Header2: val2'
  -h, --help                  help for vhost
  -m, --method string         Use the following HTTP method (default "GET")
  -k, --no-tls-validation     Skip TLS certificate verification
  -P, --password string       Password for Basic Auth
      --proxy string          Proxy to use for requests [http(s)://host:port]
      --random-agent          Use a random User-Agent string
      --timeout duration      HTTP Timeout (default 10s)
  -u, --url string            The target URL
  -a, --useragent string      Set the User-Agent string (default "gobuster/3.1.0")
  -U, --username string       Username for Basic Auth

Global Flags:
      --delay duration    Time each thread waits between requests (e.g. 1500ms)
      --no-error          Don't display errors
  -z, --no-progress       Don't display progress
  -o, --output string     Output file to write results to (defaults to stdout)
  -p, --pattern string    File containing replacement patterns
  -q, --quiet             Don't print the banner and other noise
  -t, --threads int       Number of concurrent threads (default 10)
  -v, --verbose           Verbose output (errors)
  -w, --wordlist string   Path to the wordlist

As shown above the Global flags are the same as for the all modes. Again, the 2 essential flags are the -u URL and -w wordlist. Not essential but useful -o output file and -t threads, -q for quiet mode to show the results only.

Vhost example

user@matrix:$ gobuster vhost -u https://example.com -t 50 -w /wordlists/Discovery/DNS/subdomains-top1million-5000.txt 

Results

===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url:          https://example.com
[+] Method:       GET
[+] Threads:      4
[+] Wordlist:     /wordlists/subdomains-top1million-5000.txt
[+] User Agent:   gobuster/3.1.0
[+] Timeout:      10s
===============================================================
2022/03/22 10:21:38 Starting gobuster in VHOST enumeration mode
===============================================================
Found: auto.example.com (Status: 200) [Size: 162]
Found: beta.example.com (Status: 200) [Size: 162]
Found: apache.example.com (Status: 200) [Size: 162]
                                                        
===============================================================
2022/03/22 10:21:39 Finished
===============================================================

Conclusion

Gobuster is a useful tool for recon and increasing the knowledge of the attack surface. Start with a smaller size wordlist and move to the larger ones as results will depend on the wordlist chosen. Keep enumerating. Don't stop at one search, it is surprising what is just sitting there waiting to be discovered.

The post Gobuster tutorial appeared first on HackerTarget.com.

Nessus 10.0 even has support for Raspberry Pi allowing it to be deployed anywhere.

A Hacker Target team member grabbed a copy of Nessus Essentials 10.0.0 and installed it on a clean Ubuntu 20.04 system. NB: This is just a quick look at the product. We do not use it commercially as part of the work done by HackerTarget.com as this would require Nessus Pro and, as we favour the Open Source OpenVAS vulnerability scanner.

Register for an Activation Code

Although free, Nessus essentials require the user to register for an activation code.

This activation does not expire however it is one use only. So, if you want to install Nessus on another machine or reinstall it, you need to register for another code.

Check your email for a message from Tenable with the activation code inside.

Download and Install

We are going to run Nessus on Ubuntu Debian. Head to the downloads page

Install takes less than a minute. It is fast and easy as you can seen below.

user@acidburn:~$ sudo dpkg -i Downloads/Nessus-10.0.0-ubuntu1110_amd64.deb
[sudo] password for user:

Selecting previously unselected package nessus.
(Reading database ... 343156 files and directories currently installed.)
Preparing to unpack Nessus-10.0.0-ubuntu1110_amd64.deb ...
Unpacking nessus (10.0.0) ...
Setting up nessus (10.0.0) ...
Unpacking Nessus Scanner Core Components...
Created symlink /etc/systemd/system/nessusd.service -> /lib/systemd/system/nessusd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/nessusd.service -> /lib/systemd/system/nessusd.service.  

 - You can start Nessus Scanner by typing /bin/systemctl start nessusd.service
 - Then go to https://192.168.1.123:8834/ to configure your scanner

At the prompt, start the Nessus scanner start nessusd.service

user@acidburn:~$ /bin/systemctl sudo start nessusd.service

Now check the status to see if it is active/running with systemctl status nessusd

user@acidburn:~$ systemctl status nessusd

After setting an admin password during the web based configuration, I promptly managed to forget it. Reset the Nessus admin password easily using the nessuscli utility:

user@acidburn:~$ sudo /opt/nessus/sbin/nessuscli chpasswd admin

Nessus Management Console

After logging in the first time you are presented with this popup. Launch a host discovery scan to identify what hosts on our network are available to scan. You can choose to do this or close this pop up and come back to to the "Host discovery scan" under the "New Scans" page

It's a simple interface and straight-forward to create a new scan. There is an option to create a New Folder allowing you to keep your scans organised. Once created this folder will appear on the left side under My Scans.

Settings

Settings provide information on the version, last update, license expiration, and Licensed Hosts - indicating how many of the 16 free scans you have used. From the menu on the left, you can access a variety of things such as Proxy Server, Password Management, and 'My Account'. There are a lot more settings to look through but for this post nothing needed to be configured and stuck with the default setup.

Create a Nessus Scan

Creating a scan is easy enough. Click on the  + New Scan  button or "Create a new scan".
This takes you to the Scan Templates screen. Many users only use the default scan option, so clearly defining the available options as Basic Network Scan, Advanced Scan, Web App Tests and Malware Scan helps the user understand the scope of the test. Scrolling down the page shows other options, including those that require an upgrade to Professional to access.

An example is the Basic Network Scan option - see screenshot below - it's easy to see what information goes where. In this instance sticking with the default setup, added the info, and saved the scan.

For the scan target, you could enter a host-name, IP address or a network range.

Launch a Nessus Scan

From this page we can see previously run scans and the newly added scan. The last modified column shows run time of previously launched scans, here you can also select the play button to run the new scan (appearing at the bottom of the list) or re-run a previous scan. Once launched the scan jumps to the top row and goes about its scanning.

Nessus Report and Detection

The test scan did a good of detecting missing updates on the test Ubuntu system. For those unfamiliar with vulnerability scanners, I recommend you take a look at the options to customise the scan policies. One of the most important configurations options for basic network scans is to ensure you are performing credential scan. This allows much more in-depth coverage of the target as the scanner is able to interrogate the system for installed software and packages. Giving valuable information to the scan engine such as patch levels of the system for both Windows or Linux based targets.

Malware detection with Nessus

An interesting feature of Nessus is the known malware detection feature. Malware has been a problem since the days of the first boot loader virus's, however in today's world of ransomware, the threats are wide-spread and potentially devastating to an organisation.

The security industry is creating all manner of network based anomaly detection products to discover unknown malware. This capability seems quite simple and one I suspect will be beneficial to many organisations.

As the Nessus scanner performs a credential based scan of a system it can collect hashes of all the running processes and compare these to an online database that is effectively a clone of a system such as VirusTotal. So it immediately adds a new layer of defence to your Anti-virus capability. If your primary AV client misses a piece of malware; when you run your regular Nessus scan you may still catch the unknown malware. Understand however that like any AV detection it will also not find everything.

Nessus Command Line Scanning

So, the question is - with Nessus Essentials, can a scan be run from the command line? The short answer is no. Nor can it be done with a Nessus Pro account. In previous versions there was more that could be done via the command line, now, other than the initial setup and changing passwords, all scans are done via the GUI.

Nessus API access

What can we do with the API? --> Scans can't be run via the API. According to the answer on the community Q&A - this function is only available to Tenable.sc or Tenable.io consoles.

Conclusion

Overall, Nessus Essentials 10 is relatively easy to deploy and configure. You can be up and running within 15-20 minutes. This has been a quick review, further testing would be required to see how it scales on a large network and how comprehensive the vulnerability detection plugins are.

It is preferable to have multiple vulnerability scanner options available. Having more than one assists in correlation and provides an assurance that a vulnerability missed by one scanner may be picked up by the second. Our online OpenVAS scan based on the Greenbone Vulnerability Manager is an effective second assessment option, particularly when reviewing Internet-facing systems.

The post Nessus 10 On Ubuntu 20.04 Install And Mini Review appeared first on HackerTarget.com.

In this post, I detail how to easily deploy additional Ubuntu Linux-based servers into the DetectionLab environment.

The idea is to be able to quickly deploy a Linux server that has agents (ossec / osquery) pre-installed for security monitoring, with the agents sending data back to the DetectionLab logger host.

Vagrant Configuration File

The basis for adding additional hosts to the DetectionLab network is the Vagrant configuration file. If you have not used Vagrant, then getting things working in a test environment is not difficult. A little bit of knowledge will go a long way, and you will find yourself using Vagrant for other tasks as it simply saves you time.

Digging into the Vagrant configuration and deployment, we find that the configuration file is configured to:

1. Deploy an Ubuntu server that connects to the DetectionLab Network (192.168.38.0/24)
2. Installs osquery agent and connects to the Fleet Manager
3. Install ossec agent, configured to send logs to UDP 514 Syslog on Splunk
4. Updates Splunk to enable the UDP Syslog Input
5. Splunk is now collecting security events from the Ubuntu Server (indexing both osquery and ossec events)

Hopefully, this sample configuration highlights the time saving benefits of using Vagrant within your lab environment. All these tasks are performed automatically and after the operating system is deployed, with only a few configuration files.

Using this configuration it will be straightforward to modify the parameters and shell commands to generate custom systems that suit your own environment.

Reading through the configuration file there is nothing overly complex. One point to highlight is the curl commands that retrieve a JWT token from the fleetdm server (using default credentials). Using these credentials the second curl command downloads the client certificate so that osquery can connect to Fleet.

Deploying the Ubuntu Host

To get started grab the Vagrantfile, secret.txt, ossec.conf and flagfile.txt from Github. Modify the host name and IP address as required if you are deploying multiple systems.

Hostname: ubuntu200
IP Address: 192.168.38.200

test@vbox:~/$ git clone https://github.com/hackertarget/DetectionLabUbuntuAddon/
test@vbox:~/$ cd DetectionLabUbuntuAddon
test@vbox:~/DetectionLabUbuntuAddon$ vagrant up

This will start up the host, add it to the network and install ossec and osquery. Log into the fleetdm console to see if the host (ubuntu200) has connected successfully.

And in this screenshot we can see ossec alerts being indexed in Splunk from the ubuntu200 host.

The post Extend DetectionLab with Linux Endpoints appeared first on HackerTarget.com.

DetectionLab and Vagrant

DetectionLab by Chris Long makes this initial configuration a piece of cake. It is quite simply amazing; if DetectionLab was thrown into a 1RU box with blinking lights (Lockeed/Boeing/Raython) would probably sell this thing as a Cyber Range for 7+ figures.

The DetectionLab will run on many operating systems and Hypervisors, but, in this quick start guide, we will use a clean Ubuntu 20.04 host, VirtualBox and Vagrant.

Included Software Components

There is a lot packaged together into this popup network.

While this is a quick way to spin up a testing environment for immediate testing of the installed software components. It is also very easy to expand the network using Vagrant; Throw in Kali Linux or other attack tools , Malware Analysis or an endless array of offensive / defensive scenarios.

Installing DetectionLab

Ok, enough with the introduction - start the stopwatch. Lets go!

Due to the number of virtual hosts, running the lab will require a minimum of an i5 with 16GB of memory. If you are planning on adding additional hosts, then more memory would be very helpful.

test@vbox:~/$ sudo apt install virtualbox virtualbox-ext-pack

This will install Virtualbox 6.1.6 from the Ubuntu repository. Keeping things fast and supported with easy updates. The name of the game in building our cyber security lab is to minimise hassle.

We will also install the latest vagrant from Hashicorp (2.2.9 is recommended - Ubuntu 20.04 ships with 2.2.6).

test@vbox:~/$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

test@vbox:~/$ sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

test@vbox:~/$ sudo apt-get update && sudo apt-get install vagrant

Download the Deployment Files

Download the required files for deployment from github and kick off the prepare.sh script.

test@vbox:~/$ git clone https://github.com/clong/DetectionLab.git 
test@vbox:~/$ cd DetectionLab/Vagrant
test@vbox:~/DetectionLab/Vagrant$ ./prepare.sh 
[+] Checking for necessary tools in PATH...
  [-] Packer was not found in your PATH.
  [-] This is only needed if you plan to build you own boxes, otherwise you can ignore this message.
  [?] Vagrant was found in your PATH
  [?] Your version of Vagrant (2.2.16) is supported
  [?] Curl was found in your PATH

[+] Checking if any boxes have been manually built...
  [?] No custom built boxes found

[+] Checking for disk free space...
  [?] You have more than 80GB of free space on your primary partition

[+] Checking if any Vagrant instances have been created...
  [?] No Vagrant instances have been created yet

[+] Checking if the vagrant-reload plugin is installed...
  [?] The vagrant-reload plugin is currently installed

[+] Enumerating available providers...
Available Providers:
  [?] virtualbox

To get started building DetectionLab, run vagrant up.
If you run into any issues along the way, check out the troubleshooting and known issues page: 
https://www.detectionlab.network/deployment/troubleshooting/

Before starting the vagrant builds, check that you are in a terminal that can launch the VirtualBox Guest GUI's. This means if you are connected to your test machine remotely using ssh you will want to use X forwarding in the session (ssh -X ...) and you will not want to be in tmux.

Essentially what happens with vagrant up is that each machine will be deployed and during the build the VirtualBox GUI will pop up as the machines are started. It is possible to build the hosts individually such as vagrant up logger but launching the following will build all 4 hosts. Depending on the bandwidth and CPU speed this will take some time, however here is the best part; no further interaction is required.

From a terminal the vagrant up process will need to launch the VirtualBox GUI, so we need to run this locally, or in an X forwarded session, if the host is remote & not in tmux:

In the terminal launch:

test@vbox:~/DetectionLab/Vagrant/$ vagrant up

During the installation and provisioning occasionally something may break as there are lots of moving parts. If a machine does not boot correctly check the error but if it is not obvious, you can simply try and re-provision the machine.

To reattempt the provisioning try vagrant reload wef --provision. If this fails you can simply remove the broken machine from the VirtualBox console and rerun vagrant up wef, this will often just work.

These basic functions of vagrant are good to know. If you mess up a machine during testing or when something breaks; rebuilding a single machine is a piece of cake.

Further Resources

The DetectionLab project site has great documentation and resources for further information. Including the excellent network layout diagram linked below and introductory YouTube video.

@Chris put together this great overview that includes credentials for access.
DetectionLab Network

Nmap Scan of the DetectionLab Network

Lets take a quick look at the services available on our new host only network. This 192.168.38.0/24 is configured on the host and is isolated. Note: with these networks, outbound traffic is allowed as the network has a NAT gateway. Keep that in mind if you plan on testing any malicious software.

Starting Nmap 7.80 ( https://nmap.org ) at 2021-07-30 13:08 AEST
Stats: 0:02:19 elapsed; 0 hosts completed (4 up), 4 undergoing Service Scan
Nmap scan report for 192.168.38.102
Host is up (0.00053s latency).
Not shown: 93 filtered ports
PORT     STATE SERVICE       VERSION
53/tcp   open  domain?
| fingerprint-strings: 
|   DNSVersionBindReqTCP: 
|     version
|_    bind
88/tcp   open  kerberos-sec  Microsoft Windows Kerberos (server time: 2021-07-30 03:08:10Z)
135/tcp  open  msrpc         Microsoft Windows RPC
139/tcp  open  netbios-ssn   Microsoft Windows netbios-ssn
389/tcp  open  ldap          Microsoft Windows Active Directory LDAP (Domain: windomain.local, Site: Default-First-Site-Name)
445/tcp  open  microsoft-ds  Windows Server 2016 Standard Evaluation 14393 microsoft-ds (workgroup: WINDOMAIN)
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: DC
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: dc.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.14393
|_  System_Time: 2021-07-30T03:10:25+00:00
| ssl-cert: Subject: commonName=dc.windomain.local
| Not valid before: 2021-07-27T06:16:20
|_Not valid after:  2022-01-26T06:16:20
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: Host: DC; OS: Windows; CPE: cpe:/o:microsoft:windows

Host script results:
|_nbstat: NetBIOS name: DC, NetBIOS user: , NetBIOS MAC: 08:00:27:a9:3b:cf (Oracle VirtualBox virtual NIC)
| smb-os-discovery: 
|   OS: Windows Server 2016 Standard Evaluation 14393 (Windows Server 2016 Standard Evaluation 6.3)
|   Computer name: dc
|   NetBIOS computer name: DC\x00
|   Domain name: windomain.local
|   Forest name: windomain.local
|   FQDN: dc.windomain.local
|_  System time: 2021-07-30T03:10:25+00:00
| smb-security-mode: 
|   account_used: guest
|   authentication_level: user
|   challenge_response: supported
|_  message_signing: required
| smb2-security-mode: 
|   2.02: 
|_    Message signing enabled and required
| smb2-time: 
|   date: 2021-07-30T03:10:25
|_  start_date: 2021-07-28T06:16:29

Nmap scan report for 192.168.38.103
Host is up (0.00027s latency).
Not shown: 99 filtered ports
PORT     STATE SERVICE       VERSION
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: WEF
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: wef.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.14393
|_  System_Time: 2021-07-30T03:10:26+00:00
| ssl-cert: Subject: commonName=wef.windomain.local
| Not valid before: 2021-07-29T00:33:11
|_Not valid after:  2022-01-28T00:33:11
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: OS: Windows; CPE: cpe:/o:microsoft:windows

Nmap scan report for 192.168.38.104
Host is up (0.00089s latency).
Not shown: 98 filtered ports
PORT     STATE SERVICE       VERSION
135/tcp  open  msrpc         Microsoft Windows RPC
3389/tcp open  ms-wbt-server Microsoft Terminal Services
| rdp-ntlm-info: 
|   Target_Name: WINDOMAIN
|   NetBIOS_Domain_Name: WINDOMAIN
|   NetBIOS_Computer_Name: WIN10
|   DNS_Domain_Name: windomain.local
|   DNS_Computer_Name: win10.windomain.local
|   DNS_Tree_Name: windomain.local
|   Product_Version: 10.0.18362
|_  System_Time: 2021-07-30T03:10:25+00:00
| ssl-cert: Subject: commonName=win10.windomain.local
| Not valid before: 2021-07-29T00:43:18
|_Not valid after:  2022-01-28T00:43:18
|_ssl-date: 2021-07-30T03:11:05+00:00; 0s from scanner time.
Service Info: OS: Windows; CPE: cpe:/o:microsoft:windows

Nmap scan report for 192.168.38.105
Host is up (0.00023s latency).
Not shown: 95 closed ports
PORT     STATE SERVICE    VERSION
22/tcp   open  ssh        OpenSSH 7.6p1 Ubuntu 4ubuntu0.3 (Ubuntu Linux; protocol 2.0)
| ssh-hostkey: 
|   2048 1f:9d:24:3e:07:2b:74:0c:6a:26:76:4f:0f:db:0f:01 (RSA)
|   256 60:2b:09:3a:f4:17:42:b9:54:8f:9b:37:df:1b:31:b0 (ECDSA)
|_  256 25:c4:3c:ee:32:1f:a5:88:65:ad:9e:01:8e:51:bc:b3 (ED25519)
111/tcp  open  rpcbind    2-4 (RPC #100000)
| rpcinfo: 
|   program version    port/proto  service
|   100000  2,3,4        111/tcp   rpcbind
|   100000  2,3,4        111/udp   rpcbind
|   100000  3,4          111/tcp6  rpcbind
|_  100000  3,4          111/udp6  rpcbind
8000/tcp open  ssl/http   Splunkd httpd
| http-robots.txt: 1 disallowed entry 
|_/
|_http-server-header: Splunkd
| http-title: Site doesn't have a title (text/html; charset=UTF-8).
|_Requested resource was https://192.168.38.105:8000/en-US/account/login?return_to=%2Fen-US%2F
| ssl-cert: Subject: commonName=logger/organizationName=SplunkUser
| Not valid before: 2021-07-28T06:04:33
|_Not valid after:  2024-07-27T06:04:33
8080/tcp open  http       Apache Tomcat
| http-methods: 
|_  Potentially risky methods: PUT DELETE
|_http-open-proxy: Proxy might be redirecting requests
|_http-title: Apache Tomcat
9999/tcp open  ssl/abyss?
| fingerprint-strings: 
|   FourOhFourRequest: 
|     HTTP/1.0 302 Found
|     Content-Type: text/html; charset=utf-8
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:46 GMT
|     Content-Length: 38
|     href="/app/index.html">Found.
|   GenericLines, Help, Kerberos, LDAPSearchReq, LPDString, RTSPRequest, SSLSessionReq, TLSSessionReq, TerminalServerCookie: 
|     HTTP/1.1 400 Bad Request
|     Content-Type: text/plain; charset=utf-8
|     Connection: close
|     Request
|   GetRequest: 
|     HTTP/1.0 302 Found
|     Content-Type: text/html; charset=utf-8
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:21 GMT
|     Content-Length: 38
|     href="/app/index.html">Found.
|   HTTPOptions: 
|     HTTP/1.0 302 Found
|     Location: /app/index.html
|     Date: Fri, 30 Jul 2021 03:08:21 GMT
|_    Content-Length: 0
| ssl-cert: Subject: commonName=VelociraptorServer/organizationName=Velociraptor
| Subject Alternative Name: DNS:VelociraptorServer
| Not valid before: 2021-04-14T03:25:30
|_Not valid after:  2022-04-14T03:25:30
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Post-scan script results:
| clock-skew: 
|   0s: 
|     192.168.38.103
|     192.168.38.102
|_    192.168.38.104
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 4 IP addresses (4 hosts up) scanned in 185.53 seconds

There is a bunch of stuff here to poke at with your penetration testing tools. Deploy Kali Linux on another host in the network, or simply start looking through the logs and data coming into Splunk.

Conclusion

There are many things here to test, play with and explore. This is one of the great things about this setup. Fire it up and get your hands on tools that are preconfigured and ready to go. In case you did not notice from the previous praise, DetectionLab comes highly recommended whether you are just getting started in Cyber Security or you have been breaking computers for years.

The post Build a Cyber Security Lab with DetectionLab appeared first on HackerTarget.com.

Ninja Level Monitoring and System Visibility

Osquery is a monitoring framework. It provides detailed visibility into the operating system, processes, and network connections of a computer system.

Osquery can be used in production environments on both workstations and servers. A powerful selling point being that it performs great (minimal overhead) on Linux, OSX (macOS), and Windows systems.

There are many advantages for both IT and Security Operations. We will focus on the Security Operations and DFIR (Digital Forensics and Incident Response) features as part of this tutorial.

Install osquery on Ubuntu Linux

Originally developed by Facebook, osquery is a well-supported and documented tool. It has straightforward installation steps for a variety of operating systems and Linux distributions. In this tutorial, we will focus on installation on Ubuntu from the official repository. If you are using Fedora or other Linux distros the initial steps are well documented.

These steps can be used on Debian or Ubuntu based systems. It will add the apt repository to the system and install the package. The regular system level apt upgrade will upgrade the package as required in the future.

~$ export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $OSQUERY_KEY
~$ sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
~$ sudo apt-get update
~$ sudo apt-get install osquery

Following this installation the /etc/osquery location will be created for configuration files but these will not be populated at this stage.

Interactive Shell for Immediate Testing (osqueryi)

Before doing any configuration, we can load the interactive shell to perform test queries.

Using SQL (sqlite is the basis for the SQL syntax) queries, we can query tables to gather information about the operating system. In the below query, we get a list of users (example has been snipped).

vagrant@ubuntu-focal:~$ osqueryi
Using a virtual database. Need help, type '.help'
osquery> select * from users;
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+
| uid   | gid   | uid_signed | gid_signed | username         | description                        | directory                | shell             | uuid |
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+
| 0     | 0     | 0          | 0          | root             | root                               | /root                    | /bin/bash         |      |
| 1     | 1     | 1          | 1          | daemon           | daemon                             | /usr/sbin                | /usr/sbin/nologin |      |
| 2     | 2     | 2          | 2          | bin              | bin                                | /bin                     | /usr/sbin/nologin |      |
| 33    | 33    | 33         | 33         | www-data         | www-data                           | /var/www                 | /usr/sbin/nologin |      |
| 1001  | 1001  | 1001       | 1001       | ubuntu           | Ubuntu                             | /home/ubuntu             | /bin/bash         |      |
| 998   | 100   | 998        | 100        | lxd              |                                    | /var/snap/lxd/common/lxd | /bin/false        |      |
+-------+-------+------------+------------+------------------+------------------------------------+--------------------------+-------------------+------+

Another example this time with fields selected and a LIMIT:

osquery> select uid, username, directory from users LIMIT 5;
+-------+------------------+--------------------------+
| uid   | username         | directory                |
+-------+------------------+--------------------------+
| 0     | root             | /root                    |
| 1     | daemon           | /usr/sbin                |
| 2     | bin              | /bin                     |
| 3     | sys              | /dev                     |
| 4     | sync             | /bin                     |
+-------+------------------+--------------------------+

Running osqueryi from the Command Line

Executing queries directly from the command line with osqueryi can be useful. See the following examples.

~$ osqueryi "SELECT * FROM users;"
~$ echo "SELECT * FROM users;" | osqueryi
~$ osqueryi --json "SELECT * FROM users;"

In the third example above we have used the --json parameter to change the output format. This is a great trick to get operating system telemetry into json for use in bash scripts and command line processing.

Quick osquery Linux Example Queries

Examples are the best way to showcase a framework with so much flexibility. As the examples highlight use cases for osquery are very broad.

Using Math to Calculate Disk Space

Getting slightly more complicated with a query to calculate the free space on a partition.

osquery> SELECT path, ROUND( (10e-10 * blocks_available * blocks_size), 1) AS gb_free, 100 - ROUND ((blocks_available * 1.0 / blocks * 1.0) * 100, 1) AS percent_used, device, type FROM mounts WHERE path = '/';
+------+---------+--------------+-----------+------+
| path | gb_free | percent_used | device    | type |
+------+---------+--------------+-----------+------+
| /    | 39.8    | 4.3          | /dev/sda1 | ext4 |
+------+---------+--------------+-----------+------+

JOIN Example Showing LISTENING services with Executable Path

An example that shows the value of a SQL JOIN statement combining data from two tables.

osquery> SELECT p.path, local_port FROM process_open_sockets s JOIN processes p ON s.pid = p.pid WHERE s.state = 'LISTEN';
+-----------------------------------+------------+
| path                              | local_port |
+-----------------------------------+------------+
| /usr/lib/systemd/systemd-resolved | 53         |
| /usr/sbin/sshd                    | 22         |
| /usr/bin/nc.openbsd               | 4000       |
| /usr/sbin/sshd                    | 22         |
+-----------------------------------+------------+

Hardware Monitoring

Rather than digging through log files and the Windows Registry osquery can also help with monitoring for hardware changes.

Particularly important for high security environments (classified networks) or for those IT departments who just want to know when someone plugs in a malware ridden USB device.

osquery> select driver,vendor,model from hardware_events;
+-------------+-------------------+---------------------------+
| driver      | vendor            | model                     |
+-------------+-------------------+---------------------------+
| usb         | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] |
| usb-storage | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] |
+-------------+-------------------+---------------------------+

Another table of interest is the usb_devices

osquery> select usb_port, vendor, model, serial from usb_devices;
+----------+-------------------+---------------------------+------------------+
| usb_port | vendor            | model                     | serial           |
+----------+-------------------+---------------------------+------------------+
| 1        | Linux Foundation  | 1.1 root hub              | 0000:00:06.0     |
| 2        | Lexar Media, Inc. | LJDTT16G [JumpDrive 16GB] | AAXNSQBA0WN23C34 |
+----------+-------------------+---------------------------+------------------+

Query these tables on a schedule and know when users plug in a USB drive either for immediate alerting to the SOC or for historical purposes when incident handling.

osquery Configuration

Getting osquery working optimally requires an understanding of the configuration options (/etc/osquery/osquery.conf) as well as the runtime flags (/etc/osquery/osquery.flags).

The flags file is a convenient way to control runtime parameters as there can be quite a few required.

During initial testing the flags that you will want to pay attention to are those that control the logging and events.

Events vs Scheduled SQL Statement

Most of the table data is generated when an SQL statement requests data. Events are used to populate real time audit data such as process execution, network auditing, and filesystem changes (file integrity monitoring). Without the event (audit) option, a network or process event that occurred between two scheduled SQL queries may be missed.

By default, the event (pubsub) framework is disabled. Depending on the host configuration, other process auditing (auditd) may also be in use. Note that auditd and the osquery auditing cannot be used at the same time (see eBPF as an alternative).

In this example we get an error when attempting to query an events table where the events are disabled.

osquery> select * from socket_events;
W0809 06:38:53.354483  5130 virtual_table.cpp:969] Table socket_events is event-based but events are disabled
W0809 06:38:53.354588  5130 virtual_table.cpp:976] Please see the table documentation: https://osquery.io/schema/#socket_events

For this query to work we need to either pass parameters to the command line of osqueryi as shown below, or we can set the parameters in the /etc/osquery/osquery.flags file.

~$ osqueryi --audit_allow_config=true --audit_allow_sockets=true --audit_persist=true --disable_audit=false --events_expiry=1 --events_max=50000 --disable_events=false

eBPF and osquery

eBPF is the new alternative way capture the auditing data (available since osquery 4.6.0) on Linux systems. It uses new kernel functionality (eBPF) to capture the process, socket, and other types of events.

There is a great Youtube presentation on that covers the technical details of how eBPF and osquery work.

To use eBPF the kernel will need to be 4.18 or newer.

With eBPF enabled we will have access to tables bpf_process_events and bpf_socket_events that are equivalent to the standard process_events and socket_events tables.

Enabling eBPF for osquery on Linux requires the following flags:

--disable_events=false --enable_bpf_events=true

Container Monitoring

A further advantage when using eBPF rather than the audit subsystem is greater visibility into containers and management systems including both Docker and Kubernetes.

Testing eBPF & osquery on Ubuntu 20.04

When first testing eBPF you will want to ensure it works on test system. Using osqueryi is a great way to try things out.

~$ sudo osqueryi --disable_events=false --enable_bpf_events=true --verbose

If running this osquery on a default Ubuntu 20.04 install you may hit the following error:

I0819 00:01:57.169797 86613 bpfeventpublisher.cpp:297] Failed to load the BPF probe for syscall __x64_sys_execve: The 'enter' program could not be loaded: Failed to open the Linux kernel version header: /usr/include/linux/version.h
I0819 00:01:57.169965 86613 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Failed to create the function tracer: The 'enter' program could not be loaded: Failed to open the Linux kernel version header: /usr/include/linux/version.h

This is shown when running with the --verbose flag. Otherwise the bpf_process_events table will simply be empty.

~$ sudo apt install linux-libc-dev

This will resolve the issue, as it includes the missing version.h file. The output from osqueryi will now show:

I0819 00:14:05.886425 88447 eventfactory.cpp:390] Starting event publisher run loop: BPFEventPublisher

After a period of time or execution of a command on the host.

osquery> select uid,cmdline,duration,ntime from bpf_process_events;
+-----+-----------------+----------+----------------+
| uid | cmdline         | duration | ntime          |
+-----+-----------------+----------+----------------+
| 0   | cat /etc/passwd | 1014624  | 84133214411451 |
+-----+-----------------+----------+----------------+

Another potential error is if the osqueryi process does not have permission to access the kernel memory space.

Event publisher not enabled: BPFEventPublisher: Failed to setup the memory lock limits. The BPF tables may not work correctly.

Ensure you have used sudo when executing osqueryi.

osquery daemon

Running osquery as a service allows ongoing recording of data points using scheduled queries and event collection (process execution / network sockets).

By default under Linux the daemon will load the default location for the flags file and configuration file. Typically the configuration file will then include the location of additional query packs.

Following are two example queries that could be included in the configuration file:

{
"scheduled_query": {
 "users_snapshot": {
 "query": "SELECT * FROM users;",
 "description": "Returns full list of users on the system.",
 "interval": 86400,
 "snapshot": true,
 }
 }
}

Notice the snapshot key. This tells the query to be logged as full results. The interval is equal to 86400 seconds (daily snapshot).

{
"scheduled_query": {
 "users_differential": {
 "query": "SELECT * FROM users;",
 "description": "List any new users or changes in the users table.",
"interval": 3600,
 }
 }

In this query the daily snapshot is compared and any changes are logged. The interval is 3600 seconds (hourly).

The osquery daemon will run the scheduled queries logging locally to /var/log/osquery/osqueryd.results.json or whatever logging plugins are configured.

Configuring osqueryd for a quick start

With a default (example) configuration we see a number of errors about the Event publisher not being enabled.

I0810 05:14:07.526832 278165 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Publisher disabled via configuration
I0810 05:14:07.527535 278165 eventfactory.cpp:156] Event publisher not enabled: auditeventpublisher: Publisher disabled via configuration
I0810 05:14:07.527607 278165 eventfactory.cpp:156] Event publisher not enabled: inotify: Publisher disabled via configuration
I0810 05:14:07.527662 278165 eventfactory.cpp:156] Event publisher not enabled: syslog: Publisher disabled via configuration

A better option for getting up and running is to use a working example configuration from Palantir. They have published a solid Linux Server configuration that includes an osquery.flags and osquery.conf file.

Palantir Github https://github.com/palantir/osquery-configuration/tree/master/Classic/Servers/Linux

Put these files in /etc/osquery/ and change the location of the ossec-rootkit pack in the osquery.conf file to the one at /usr/share/osquery/packs/.

Restart osqueryd and you will start getting logs. This configuration enables process monitoring, socket events and a number of other useful monitoring queries.

It is a great starting point. There are also configurations here for both Windows Endpoints and MacOS but we have primarily tested and deployed on Linux Servers.

osquery Logging

There are a number of Logging plugins for osquery. The default plugin for the daemon is filesystem logger. Logging for osquery is based on delivering a json log entry per query. Making the logs easily parsed, shipped or processed by any logging processor or platform.

It does not matter what logging platform you use, whether its Splunk or another commercial option or open source solutions such as Elastic Stack or Graylog. The fact that osquery outputs simple json makes upstream processing straightforward and flexible.

One logging pipeline example showing this flexibility.

Post processing / filtering of the logs could occur at the filebeat or logstash stages. Analysis using the Mitre Att&ck Framework or Sigma Rules for example could then occur at Graylog.

The example pipeline could certainly be simplified depending on the infrastructure and requirements.

Centralized Management & Logging

There are a number of solutions for management of an osquery "fleet". The open source fleetdm is a fork of the Kolide platform. There are also Zentral and Uptycs as commercial offerings.

These all have a TLS endpoint that the osquery client connects to; both configuration and logging can then be controlled from the centralized platform.

osquery packs

osquery packs are sets of grouped queries that can be used for different use cases. A number of default packs are included:

~$ ls /usr/share/osquery/packs/
hardware-monitoring.conf it-compliance.conf ossec-rootkit.conf
unwanted-chrome-extensions.conf windows-attacks.conf
incident-response.conf osquery-monitoring.conf osx-attacks.conf    vuln-management.conf windows-hardening.conf

Based on use cases and or operating system these are included by default in the install but not enabled in the default configuration file /usr/share/osquery/osquery.example.conf.

$ sudo cp /usr/share/osquery/osquery.example.conf /etc/osquery/osquery.conf

Third parties may release osquery packs allowing the sharing of queries within the community.

File Integrity Monitoring (FIM)

Another event based auditing option is File Integrity Monitoring. Using the configuration, you will have to specify the locations and files that are to be monitored.

Enabling the File Integrity Monitoring requires the following flags for the file_events and process_file_events tables.

--enable_file_events=true --disable_audit=false

Testing File Integrity Monitoring with osqueryi. During load with --verbose enabled we see the file paths being monitored.

~$ sudo osqueryi --disable_events=false --enable_bpf_events=true --verbose --enable_file_events
<>
I0819 05:27:30.656767  1829 file_events.cpp:87] Added file event listener to: /usr/sbin/**
I0819 05:27:30.656881  1829 file_events.cpp:87] Added file event listener to: /usr/local/bin/**
I0819 05:27:30.656985  1829 file_events.cpp:87] Added file event listener to: /usr/local/sbin/**
I0819 05:27:30.657066  1829 file_events.cpp:87] Added file event listener to: /etc/hosts
<>

These paths are set in the osquery.conf file. Now, in the following example you can the see the file event was captured in the file_events table and reported in the query.

osquery> select target_path, category, action, atime, mtime from file_events;
+-------------+---------------+---------------------+------------+------------+
| target_path | category      | action              | atime      | mtime      |
+-------------+---------------+---------------------+------------+------------+
| /etc/hosts  | configuration | ATTRIBUTES_MODIFIED | 1629350896 | 1629350896 |
+-------------+---------------+---------------------+------------+------------+

Augeas

Augeas is an interesting open-source project that is packaged with osquery. Enabled by default are a number of configuration file "lenses". These allow osquery to parse configuration files and show the status of parameters on the system. This is a very helpful tool for compliance monitoring across a fleet of systems.

The default lenses are located here and can be reviewed to see what is possible. /usr/share/osquery/lenses

osquery> SELECT label, value FROM augeas WHERE path = '/etc/ssh/sshd_config' and label = "PasswordAuthentication";
+------------------------+-------+
| label                  | value |
+------------------------+-------+
| PasswordAuthentication | yes   |
+------------------------+-------+

A key concept is that this information is being collected at the time of the query. For many use cases, the query will be run on a schedule with the results being compared to a previous result in order to identify changes in the system state (new user account, logins, new network connections).

Yara and osquery

YARA is a powerful malware and file scanning framework. It can be incorporated into an osquery configuration allowing:
- on demand scanning when a file system change occurs (from file_events)
- a yara table for on-demand YARA scanning.

Configuring YARA requires that the osquery.conf identifies the signatures to use and the file_paths to monitor.

Conclusion

This tutorial provided a quick start guide for getting a usable osquery up and running. At the same time, we have covered the building blocks needed for a more complicated deployment.

There are significant benefits to be found with osquery whether you are looking to manage a fleet of servers, tens of thousands of workstations, or a handful of endpoints. Get in contact if you find this tutorial useful or have any feedback.

Work across the teams in your organisation to find advantages for more than security operations. DevOPS & IT will love it. Increase productivity, security visibility, and inter team communication all with one deployment project.

The post osquery Linux Tutorial and Tips appeared first on HackerTarget.com.

ClamAV Antivirus is an open source malware detection tool. In this tutorial we cover getting started with ClamAV and common use cases.

Through various configuration profiles it is able to perform real time filesystem detection, ad hoc file scanning, mail gateway filtering and http proxy scanning.

These use cases only scratch the surface of what is possible using the core engine and various open source addon components.

ClamAV and the key development team were acquired by SourceFire in 2007. Now maintained by Cisco Talos, after the acquisition of SourceFire in 2013. You may have heard of Cisco, a small startup that makes networking equipment.

Installing ClamAV on Ubuntu (or Debian)

To get the very latest version, or if a custom build is required installation from source would be necessary. However, using an Antivirus package in production is something that you want to be stable, secure and easily patched when security updates are released. For this reason I recommend using the Stable version of ClamAV from the Ubuntu repositories.

While the ClamAV core version will be not be the very latest release you will still be receiving the latest updates to the Antivirus signature database through the freshclam updates.

root@8ca32fd2f888:/# apt update
root@8ca32fd2f888:/# apt install clamav
root@8ca32fd2f888:/# clamscan --version
ClamAV 0.103.2
root@8ca32fd2f888:/# freshclam

It is as easy as that. I quickly spun up an Ubuntu 20.04 Docker image and ran the installation. The clamscan version is 0.103.2.

Running freshclam will ensure the latest database is available, updates to the signature database are made once or twice daily, so put freshclam in a cron job to keep things fresh.

root@8ca32fd2f888:/# clamscan /usr/bin
<< SNIP >>
/usr/bin/ucf: OK
/usr/bin/openssl: OK
/usr/bin/ucfq: OK
/usr/bin/crontab: OK

----------- SCAN SUMMARY -----------
Known viruses: 8528798
Engine version: 0.103.2
Scanned directories: 1
Scanned files: 287
Infected files: 0
Data scanned: 20.16 MB
Data read: 22.85 MB (ratio 0.88:1)
Time: 23.066 sec (0 m 23 s)
Start Date: 2021:05:13 00:04:35
End Date:   2021:05:13 00:04:58

A simple directory scan is performed by running clamscan against the folder name. If you need a quick antivirus solution, installing ClamAV, updating and starting the scan can be done in a few minutes.

ClamAV Signatures and Coverage

At the time of testing the latest signature count is 8528798. Updates are regularly released with the following chart showing the number of new antivirus signatures daily over the 3 months to April 2021.

Testing ClamAV against a PDF with an embedded DOC

Didier Stevens created a handy test file that contains the well known EICAR test file, however the file is dropped from a DOC that is embedded within a PDF.

https://blog.didierstevens.com/2015/08/28/test-file-pdf-with-embedded-doc-dropping-eicar/

root@8ca32fd2f888:/# clamscan pdf-doc-vba-eicar-dropper.pdf
/pdf-doc-vba-eicar-dropper.pdf: Doc.Dropper.Agent-1540415 FOUND

----------- SCAN SUMMARY -----------
Known viruses: 8528798
Engine version: 0.103.2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.06 MB
Data read: 0.01 MB (ratio 7.50:1)
Time: 17.054 sec (0 m 17 s)
Start Date: 2021:05:13 00:14:55
End Date:   2021:05:13 00:15:12
root@8ca32fd2f888:/#

Understanding clamd, clamdscan and clamscan

When you run clamscan the libclamav engine and signatures are loaded at runtime. The other way to run the scanning engine is via clamd.

Clamd runs as a background process that has the engine and signatures in memory. A clamd client (clamdscan) then connects to the service in order to have the scanning performed. The clamd service accepts various commands in order to perform the scanning.

Configuration of the scanning is controlled via the clamd.conf configuration and cannot be specified at runtime. Whereas using clamscan it is possible to configure a large number of options at runtime from the command line.

Note that the clamd service is unauthenticated. Do not make it accessible from the Internet.

Integrate ClamAV in a HTTP Proxy

By integrating ClamAV into a HTTP proxy such as Squid it is possible to have transparent antivirus across all your web traffic.

Using Squid it is possible to configure the proxy to perform SSL/TLS bumping (decryption) enabling scanning of SSL/TLS encrypted traffic.

squidclamav icap module

Squid is a popular open source HTTP proxy that can work with modules using the ICAP protocol. ICAP is standard that allows HTTP proxies to outsource content inspection and manipulation to an external process or server.

SquidClamAV is an antivirus for the Squid proxy based on ICAP, it is highly performant and able to handle thousands of HTTP connections simultaneously.

Download squidclamav

Installation and Configuration of SquidClamAV goes beyond the scope of this guide.

Conclusion

Even if you already have a commercial antivirus solution within your environment, knowing that you can quickly spin up ClamAV in a few minutes to perform ad hoc testing is definitely something to keep in your grab bag of tools.

Like Vulnerability Scanning, using multiple solutions and signature databases will ensure you get the best coverage in detecting and understanding the threats to your environment.

Running ClamAV on gateway servers (SMTP / HTTP) is a popular solution for companies that lean into the open source world. With a team run out of Cisco Talos, it is no wonder that this software continues to kick goals for organisations of all sizes.

The post ClamAV Antivirus for Linux Tutorial appeared first on HackerTarget.com.

Additionally, penetration testers or red teams needing to exploit Joomla targets will also find practical hints in this guide.

Introduction to Joomla Security

Recent statistics show Joomla is a popular open-source Content Management System (CMS), with close to 6% of all websites.

It is open-source, free to download, and easy to use. These things make it a popular option. Similar to WordPress's plugins, Joomla allows functionality through "Extensions"

This popularity makes it a target for bad guys aiming to use a compromised web server for malicious purposes.

A lot of Joomla security holes arise from lack of maintenance, not taking passwords seriously, poorly coded extensions and even site backup's left in the web root.

Enumeration and Reconnaissance

Stage 1 is to discover as much technical information regarding the site configuration. This information is essential as it will aid us as we move onto the actual attacking or exploitation phase.

Now is the time to put yourself in the hacker's mindset. Enumeration or reconnaissance can be conducted stealthily with regular web requests used to gather technical information about the site. Or it can be conducted overtly by aggressively brute-forcing web paths to identify the presence of extensions.

Joomla Identification & Version

To determine if the site is running Joomla, and identify the Joomla Core version, three simple methods can be used to determine the version of Joomla in use.

Meta Generator

Check the HTML source of the page for a meta generator tag in the HEAD section of the HTML source. This is the simplest way to determine if Joomla is being used.

This example is taken from the source of a default Joomla install.

<meta name="generator" content="Joomla! - Open Source Content Management" />

joomla.xml

To identify the version we can check the joomla.xml file within the directory /administrator/manifests/files/

https://www.joomla.org/administrator/manifests/files/joomla.xml

Result

/language/en-GB/en-GB.xml

Another option to find the version is the language file.

https://example.site/language/en-GB/en-GB.xml
<version> 3.6.5 </version>

Version in README.txt

If the meta tag has been disabled, check for the presence of /README.txt from the web root of the install. Joomla has the major version at the top of the ReadMe file.

Security Vulnerabilities in Joomla Core

Let's say a site with an older Joomla Core version is discovered by an attacker. This site may be directly exploitable via a security vulnerability in the Joomla core. It also shows the site is not being well maintained.

In a poorly maintained site, other components, such as Extensions or Templates, may not be updated. The likelihood of a successful attack has dramatically improved.

Joomla Extension and version Enumeration

Similar to WordPress's plugins, Joomla allows functionality through "Extensions"
Extensions are broken down into a few types:

• Modules
• Components
• Templates
• Plugins
• Languages

All can be installed as required. Vulnerabilities can arise in any of these when poorly coded, an example could be non-logged in users having access to the same features as logged in users.

Enumeration is attempting to find as many installed extensions as we can, including disabled extensions. Knowing these extensions may allow us to identify the version, and research whether it is vulnerable to known exploits.

Unfortunately, unless you have the administrator account details, there is no easy way to find every single extension of a particular Joomla install.

It is worth noting Joomla has a live list called - Vulnerable Extensions List (VEL)
This list is of vulnerable extensions for which no patch is known to exist. It can be used as a source of information or a place to start when looking at a Joomla site.

Passive

Hints to the extensions and modules present in a site may be found in the HTML source of the page. Once, an add-on is identified additional information can be gathered from the manifest file.

Active

Some extensions do not leave traces in the HTML source. To find all the installed extensions you have to be more aggressive. Several tools can brute force known extension/component list. There is no one size fits all when it comes to Joomla. Using a combination of available tools will get the best results.

One example is the Metasploit Joomla Plugin Scanner. This metasploit auxiliary module uses a wordlist to locate valid paths scanning for extensions and vulnerabilities.

Extension Version Enumeration

You have compiled a list of extensions of the site, now for the version. The design of Joomla means this information isn't forthcoming or comprehensive, especially when attempting to find it quietly.

It is possible you may find the extension version in a manifest file or other resources such as the addon's stylesheets or javascript.

With a valid version you can compare what has been found against known exploits. This comparison will attest if the site is likely vulnerable, before throwing any exploits.

Joomla Template Enumeration

As with extensions, Joomla Templates can contain vulnerabilities that may expose the site to compromise. Templates are simply collections of PHP code with HTML and CSS resources. Complex templates have additional components and are more prone to security vulnerabilities.

Enumeration of the template is conducted similarly to detecting the extensions. Inspect the HTML and locate the template. Alternatively, run a passive scan on Hacker Target's Joomla Security Scan and scroll through results to find the Joomla Template.

One important factor when testing for vulnerable Joomla Templates and components is where it may be installed but not active; as the code is still accessible it may still be vulnerable. For this reason, brute force testing for template paths is an additional step when assessing an unknown Joomla installation.

Enumerate Users

A quick tip is first see if the Administrator login page is publicly available.

https://exampledomain.com/administrator

Gather a list of valid usernames and attempt a password guessing attack to brute force the login credentials. The aim is to gain access to the administrator account. Admin access gives the attacker complete access, and consequently a full compromise of the site, the database, and remote code execution on the server through PHP code execution.

There is no simple way to do this manually in Joomla as opposed to WordPress where it may be possible to iterate through the users using a simple bash one-liner.

With Joomla, it requires guesswork. All new installs have a 'Super Administrator account' called admin. As part of the install, Joomla requests a password for this account. Joomla also suggests changing the name of the account from 'admin' to something more difficult to guess.

This makes it complicated for a dictionary-based attack against the admin panel.

Enumerating users through Guessing

Start with the common one admin and go from there.

Joomla doesn't seem to allow direct listing all users and / or leak their information.

A default install of Joomla allows 3 privileged user groups which have access to the control panel:

• Managers: content creation and backend system info.
• Administrators: admin functions except global options.
• Super Users/Administrator: ultimate power. Access all areas.

Note: from version 3.2 two-factor authentication was implemented as a core feature. Admins can enable it from User Manager in the Control Panel.

Password Re-use and Breach Datasets

A common technique used in targeted attacks is mining breach datasets for passwords. If a user is breached on another site, there is a chance they will use the same password or a variation on the password on other sites. Working from a targeted domain passwords can quickly be found especially in larger organisations.

Directory Indexing

A misconfigured server can allow you to view the contents of a directory in a web-accessible path.

Viewing the contents of the directory allows an attacker to gather sensitive information not intended for public viewing about the existence and contents of the files. Such as hidden files, backup files, config files, plugins, and templates, without the need to brute force the paths.

Start by browsing to folder locations and see if you get a 200 OK HTTP response and see a list of files / folders in the browser.

Network Service Discovery

Here we are checking network services. The main technique used for identifying the servers attack surface is Port Scanning.

An Nmap port scan will identify the network services listening on the server. These could include FTP, SSH, Webmin or even the web server itself. Working from the results of the Port Scan an attacker would identify server applications, versions and look for exploitation opportunities.

Bypass Sucuri or CloudFlare Web Firewall

If the Joomla site is protected by Sucuri or CloudFlare, exploits that might otherwise succeeed could be blocked. Even various reconnaissance techniques can be blocked by these web based firewall (WAF).

By knowing the real IP address of the server it is likely we could bypass the server simply by putting an entry in the clients /etc/hosts file. This works because we bypass the sites DNS that would otherwise send us via the Web Firewall.

Historical DNS Records

A common method is using historical DNS records to identify the real IP address.

Historical DNS records may show the original IP address before the firewall service was implemented.
Mail Records (MX), if mail is hosted on the same server as the website then this will reveal the real host
TXT SPF, records might also reveal IP addresses of interest

TLS / SSL Certificate Searches

TLS / SSL searches against Certificate Transparency Datasets may also find real hostnames associated with the sites actual IP address if they can matched.

JoomlaVS & Other Tools

Passive Joomla Security Scan

Hacker Target hosts a free and simple to use passive Joomla scan. Discover vulnerabilities, web server details, configuration errors, identify template, and test for directory indexing and others.

The freely available tools perform analysis from a simple page grab. Through the examination of the HTML source code, javascript, and a few other open publicly accessible pages, it is possible to gain immediate insights into the state of security on the target site. This is applying only passive analysis methods, without sending any aggressive security scanning.

JoomScan

JoomScan is the OWASP Joomla! Vulnerability Scanner. An open source project written in Perl. Ties some of these enumeration techniques together such as the Joomla version, vulnerabilities and the admin login page.

Check out the the latest version from github https://github.com/rezasp/joomscan

Note this project has not been updated for a number of years

JoomlaVS

JoomlaVS is an Open source Ruby application. Scan for vulnerabilities in components, modules and templates and basic fingerprinting. More info available on the projects at https://github.com/rastating/joomlavs

----------------------------------------------------------------------

     ??? ???????  ??????? ????   ???????      ?????? ???   ???????????
     ?????????????????????????? ????????     ???????????   ???????????
     ??????   ??????   ?????????????????     ???????????   ???????????
??   ??????   ??????   ?????????????????     ???????????? ????????????
????????????????????????????? ??? ??????????????  ??? ??????? ????????
 ??????  ???????  ??????? ???     ??????????????  ???  ?????  ????????

----------------------------------------------------------------------

[+] URL: http://testexample.com/
[+] Started: Mon Jun 12 11:02:01 2020

[+] Found 1 interesting headers.
 |  Server: Apache

[+] Joomla version 2.5.30 identified from language file (en-GB.xml)
[!] Found 8 vulnerabilities affecting this version of Joomla!

[!] Title: Joomla Akeeba Kickstart Unserialize Remote Code Execution
 |  Reference: https://www.exploit-db.com/exploits/35033
 |  Reference: http://www.cvedetails.com/cve/CVE-2014-7228
[i] Fixed in: 3.3.5


[!] Title: Joomla Media Manager File Upload Vulnerability
 |  Reference: https://www.exploit-db.com/exploits/27610
 |  Reference: http://www.cvedetails.com/cve/CVE-2013-5576
[i] Fixed in: 3.1.5


[!] Title: Joomla 2.5.x Language Switcher ModuleMultiple Cross Site Scripting Vulnerabilities
 |  Reference: https://www.exploit-db.com/exploits/37473
[i] Fixed in: 3


[!] Title: Joomla 1.5 - 3.4.5 - Object Injection Remote Command Execution
 |  Reference: https://www.exploit-db.com/exploits/38977
 |  Reference: http://www.cvedetails.com/cve/CVE-2015-8562
[i] Fixed in: 3.4.6


[!] Title: Remote Code Execution in third-party PHPMailer library
 |  Reference: http://www.cvedetails.com/cve/CVE-2016-10033
 |  Reference: http://www.cvedetails.com/cve/CVE-2016-10045
[i] Fixed in: 3.6.5


[!] Title: Unauthorised Logins
 |  Reference: http://www.cvedetails.com/cve/CVE-2014-6632
[i] Fixed in: 3.3.3


[!] Title: Denial of Service
 |  Reference: http://www.cvedetails.com/cve/CVE-2014-7229
[i] Fixed in: 3.3.4


[!] Title: Joomla! < 3.6.4 Privilege Escalation
 |  Reference: http://www.cvedetails.com/cve/CVE-2016-9838
[i] Fixed in: 3.6.4


[+] Scanning for vulnerable components...
[!] Found 0 vulnerable components.

------------------------------------------------------------------

[+] Scanning for vulnerable modules...
[!] Found 0 vulnerable modules.

------------------------------------------------------------------

[+] Scanning for vulnerable templates...
[!] Found 0 vulnerable templates.

------------------------------------------------------------------

[+] Finished

CMSMap

As the name implies, CMSMap covers the most popular of the CMS's. An open source project written in Python, this tool has support for Joomla, WordPress and Drupal. Useful for automating a scan for low-hanging fruit.

Download the latest version of CMSMap github: 

git clone https://github.com/Dionach/CMSmap 

Attacking and Exploitation

Brute Force Joomla logins

2013 Joomla 3.2 stable release bought 2FA as part of the core install which adds another challenge to brute-forcing an account. But it isn't enabled by default.

Popping Weak Passwords

Detecting weak passwords for Joomla comes in a variety of ways. There are many ways to to brute force a login page, here are a few.

Nmap NSE Scripts for Joomla

NMAP is most known for network discovery, however, NSE scripts extend the functionality of the popular NMAP port scanner. An Nmap NSE script is particularly helpful for performing a brute-force password play against a Joomla install.

• http-joomla-brute.html
• http-form-brute

    $ nmap -p80 http-joomla-brute example-site

Burpsuite

If there is a login form on the site or you have found the administrator interface, then burp suite can be used to try to brute force the password.

There are other tools around such as JoomBrute, and others such as Hydra and Ncrack, though the latter two are most suited for other protocols.

Metasploit

Rapid7's Metasploit provides a few modules for brute forcing CMS and Joomla for various Joomla versions. One is the Joomla Bruteforce login utility

msf > use auxiliary/scanner/http/joomla_bruteforce_login

Exploit Joomla Extensions

One of the most common reasons for Joomla sites being compromised is vulnerable extensions, modules & plugins. These all contain a large amount of PHP code and come from developers of differing levels of skills, abilities, and focus when it comes to writing software that is secure.

Keeping the Joomla extensions, core and templates updated and/or patched needs to be a routine task for the Joomla administrator of the site.

1,437
published exploits for
Joomla and its components

Joomla Security announcements and Vulnerable extensions list

The Joomla Developer Network has a Security Announcements which provides a feed of recently resolved security issues in Joomla software releases.

As part of the Joomla extension directory, Joomla has a list of Vulnerable extensions .

Exploit Example

An issue was discovered in the Creative Contact Form extension (2019). A directory traversal vulnerability resides in the filename field for uploaded attachments. An attacker could exploit this vulnerability with the "Send me a copy" option to receive any files of the filesystem via email.

Exploit References:
https://packetstormsecurity.com/files/156655/Creative-Contact-Form-4.6.2-Directory-Traversal.html
https://nvd.nist.gov/vuln/detail/CVE-2020-9364#VulnChangeHistorySection

Exploit Joomla Template

While vulnerabilities in templates are not as common as extensions, it is still worth checking the template in use. Check the developers page for security related updates, and if its a custom environment running standard web application testing may discover unpublished vulnerabilities.

In this example of an XSS vulnerability, we see that even the Joomla Core Default template had a vulnerability as recently as 2019.

Exploit References:
https://developer.joomla.org/security-centre/791-20190901-core-xss-in-logo-parameter-of-default-templates.html

Exploiting Joomla Core

Vulnerabilities in Joomla Core are highly valued by an Attacker as it does not depend on a particular extension being installed.

In 2016, 2 critical vulnerabilities allowed privilege escalation by remote users. Attackers were first able to create accounts even if account registration is disabled, and 2nd, increase their privileges on any Joomla site using versions 3.4.4 to 3.6.3. Attackers could then upload a backdoor and ultimately control the site.

Solution was the upgrade to version 3.6.4. Joomla devs released limited information on the the vulnerabilities, however it was enough for groups to figure things out and develop exploits. Exploits in the wild were spotted, some included ones that were automatically uploading backdoors to vulnerable sites.

Exploit References:
CVE-2016-8870
CVE-2016-8869
Joomla! 3.4.4 < 3.6.4 - Account Creation / Privilege Escalation

Sniff and Capture Credentials over non-secure login

HTTP over TLS should be enabled on any public website in 2020. If only HTTP is used, passwords, logins and session cookies are all sent over the network in the clear. Clear text sessions could be monitored on your local network, or on your Internet providers network or anywhere between the client and server.

With the easy access to Free certificates there is really no reason to be not using HTTPS. Check your TLS configuration with tools such as sslyze.

Vulnerable Server Software

Exploitation of the Joomla site could come from other server components - it is not always the Joomla Web Application that will be the exploitation vector. A simple misconfiguration of a network service or a failure to apply server updates could lead to compromise of the server and all hosted applications.

With the results of an Nmap scan, an attacker will review open services for known vulnerabilities. A small sample of these network services includes FTP, SSH, MySQL & Redis - any of these could lead to server compromise if vulnerable or poorly configured.

Compromise Systems Administration Tools

Related to the previous section, here we are identifying server applications that may lead to compromise of the Joomla site.

An example of this type of tool is phpmyadmin running on the web server. A weak database password or vulnerable version of phpmyadmin would lead to compromise of the database and possibly even code execution.

Discovery of the phpmyadmin installation would usually involve a content discovery scan that would include common paths for phpmyadmin in the content discovery scanner list.

Content Discovery

Looking for sensitive information, database credentials, backups, are just a few few items that can be found with content discovery. Content discovery is attempting to find any interesting items contained within a web path of an application. There are a variety of tools our there catering for this purpose. Choose the one or ones that suit your needs. A few example tools are; DIRB, dirbustor, dirsearch or gobuster.

Common locations to check are:

    /robot.txt
    /backup
    /images/
    /bin/
    /uploads/
    /components/
    /administrator/
    /.htaccess.txt
    /index.html
    /index.php
    /templates/exampletemplate
    /administrator/templates/exampletemplate
    /phpmyadmin/
    /phpinfo.php

Found Backups

Using content discovery a commonly found vulnerability is a site backup. This results from a mistake during system administration where a backup of the folder was taken and left in the publicly accessible web root (eg. /backup.tar.gz).

With a backup an attacker has access to all files of the installation including all installed components as well as the configuration file containing the database location and password.

PHP info

Another commonly found item during content discovery is a file with the PHP function phpinfo(). Site administrators will often create a file in the root of the site such as /phpinfo.php, the function in this file allows the administrator to know what modules, PHP version and many other server configurations are available on the webserver. This information is also valuable to an attacker if the file is forgotten and left on the server.

The post Attacking and Enumerating Joomla appeared first on HackerTarget.com.

Knowing a few ssh tricks will benefit any system administrator, network engineer or security professional.

Even if you are an experienced *nix guru there are a couple of examples further down that are only available in later versions of OpenSSH. Take a look at Proxy Jump -J and reverse dynamic forwarding -R.

First The Basics

Breaking down the SSH Command Line

The following ssh example command uses common parameters often seen when connecting to a remote SSH server.

localhost:~$ ssh -v -p 22 -C neo@remoteserver

-v : Print debug information, particularly helpful when debugging an authentication problem. Can be used multiple times to print additional information.
-p 22 : Specify which port to connect to on the remote SSH server. 22 is not required as this is the default, but if any other port is listening connect to it using the -p parameter. The listening port is configured in the sshd_config file using the Port 2222 format.
-C : Compression is enabled on the connection using this parameter. If you are using the terminal over a slow link or viewing lots of text this can speed up the connection as it will compress the data transferred on the fly.
neo@ : The string before the @ symbol denotes the username to authenticate with against the remote server. Leaving out the user@ will default to using the username of the account you are currently logged in to (~$ whoami). User can also be specified with the -l parameter.
remoteserver : The hostname ssh is connecting to, this can be a fully qualified domain name, an IP address or any host in your local machines hosts file. To connect to a host that resolves to both IPv4 and IPv6 you can specify parameter -4 or -6 to the command line so it resolves correctly.

Apart from remoteserver, each of the above parameters is optional.

Using a Configuration File

While many users are familiar with the sshd_config file, there is also a client configuration file for the ssh command. This defaults to ~/.ssh/config but can also be specified as a parameter with the -F option.

Host remoteserver
     HostName remoteserver.thematrix.io
     User neo
     Port 2112
     IdentityFile /home/test/.ssh/remoteserver.private_key

Host *
     Port 2222

In the above example ssh configuration file you can see two Host entries. The first is a specific host entry with Port 2112 configured, as well as a custom IdentifyFile and username. The second is a wildcard value of * that will match all hosts. Note that the first configuration option found will be used, so the most specific should be at the top of the configuration. More information is found in the man page (man ssh_config).

The configuration file can save a lot of typing by including advanced configuration shortcuts any time a connection is made to particular hosts.

Copy Files over SSH with SCP

The ssh client comes with two other very handy tools for moving files around over an encrypted ssh connection. The commands are scp and sftp. See examples below for basic usage. Note that many parameters for the ssh can be applied to these commands also.

localhost:~$ scp mypic.png neo@remoteserver:/media/data/mypic_2.png

In this example, the file mypic.png was copied to the remoteserver to file system location /media/data and renamed to mypic_2.png.

Don't forget the difference in the port parameter. This is a gotcha that hits everyone using scp on the command line. The port parameter is -P not -p as it is in the ssh client.!. You will forget, but don't worry everyone does.

For those familiar with command line ftp, many of the commands are similar when using sftp. You can push, put and ls to your hearts desire.

sftp neo@remoteserver

Practical Examples

In many of these examples, we could achieve the result using several methods. As in all our tutorials and example command sheets, the focus is practical examples that get the job done.

1. Proxy Traffic over SSH using SOCKS

The SSH Proxy feature has been placed at number 1 for good reason. It is more powerful than many users realise giving you access to any system that the remote server can reach, using almost any application. The ssh client can tunnel traffic over the connection using a SOCKS proxy server with a quick one liner. A key thing to understand is that traffic to the remote systems will have a source of the remote server. For example in a web server log file.

localhost:~$ ssh -D 8888 user@remoteserver

localhost:~$ netstat -pan | grep 8888
tcp        0      0 127.0.0.1:8888       0.0.0.0:*               LISTEN      23880/ssh

Here we start the socks proxy server running on TCP port 8888, the second command checks that the port is now listening. The 127.0.0.1 indicates the service is running on localhost only. We can use a slightly different command to listen on all interfaces including ethernet or wifi, this will allow other applications (browsers or other) on our network to connect to the ssh socks proxy service.

localhost:~$ ssh -D 0.0.0.0:8888 user@remoteserver

Now we can configure our browser to connect to the socks proxy. In Firefox select preferences | general | network settings. Add the IP address and the port for the browser to connect to.

Note the option at the bottom of the form to force browser DNS requests to also go over the socks proxy. If you are using the proxy to encrypt your web traffic on the local network you will definitely want to select this option so the DNS requests are also tunnelled over the SSH connection.

Enable Socks Proxy on Chrome

Using a command line parameter when launching Chrome will use the socks proxy and also tunnel DNS requests from the browser over the socks5 proxy. Trust but verify, use tcpdump (tcpdump not port 22) to confirm the DNS requests are no longer visible.

localhost:~$ google-chrome --proxy-server="socks5://192.168.1.10:8888"

Using other applications with the Proxy

Keep in mind that there are many other applications that can utilise a socks proxy. A web browser is simply the most popular. Some applications will have configuration options for use of the proxy. Others may need some help by using a helper program that talks the socks protocol. An example of this is proxychains. Using this tool we can for example use Microsoft RDP over the socks proxy.

localhost:~$ proxychains rdesktop $RemoteWindowsServer

The configuration options for the socks proxy are set in the proxychains configuration file.

Use Case for the SSH Socks Proxy

You are in a cafe or hotel having to use the somewhat sketchy WIFI. From our Laptop we run the ssh proxy locally and establish an ssh tunnel into our home network using a our local Rasberry Pi. Using the browser or other applications configured for the SOCKS proxy we can access any network services on our home network or browse to the Internet via our Home Network Connection. Everything between our Laptop and the Home Server (across the WIFI and Internet to home) is encrypted in the SSH tunnel.

2. SSH Tunnel (port forward)

In its simplest form an SSH tunnel opens a port on your local system that connects through to another port at the other end of the tunnel.

localhost:~$ ssh  -L 9999:127.0.0.1:80 user@remoteserver

Lets break down the -L parameter. Think of -L as the Local listening side. So in our example above the port 9999 is listening on localhost and port forwards through to port 80 on remoteserver, note that the 127.0.0.1 refers to localhost on the remote server!

Lets take it up a notch. In this following example the port that is listening can be connected to from other hosts on the local network.

localhost:~$ ssh  -L 0.0.0.0:9999:127.0.0.1:80 user@remoteserver

In these examples the port we are connecting is a listening web server. It could also be a proxy server or any other TCP service.

3. SSH Tunnel Forward to Secondary Remote host

We can use the same options seen above to have the tunnel connect to another service running on a secondary system from the remote server.

localhost:~$ ssh  -L 0.0.0.0:9999:10.10.10.10:80 user@remoteserver

In this example we are forwarding the tunnel from remoteserver to the web server running on 10.10.10.10. The traffic from remoteserver -> 10.10.10.10 is no longer within the ssh tunnel. The web server on 10.10.10.10 will see remoteserver as the source of the web requests.

4. SSH Reverse Tunnel

In this scenario we want to setup a listening port on the remote server that will connect back to a local port on our localhost (or other system).

localhost:~$ ssh -v -R 0.0.0.0:1999:127.0.0.1:902 192.168.1.100 user@remoteserver

With this ssh session established a connection to the remoteserver port 1999 will be forwarded to port 902 on our local client.

5. SSH Reverse Proxy

In this case we are establishing a SOCKS proxy with our ssh connection, however the proxy is listening at the remote server end. With connections to that remote socks proxy now emerging from the tunnel as traffic originating from our localhost. Requires OpenSSH version 7.6+.

localhost:~$ ssh -v -R 0.0.0.0:1999 192.168.1.100 user@remoteserver

Troubleshooting Remote SSH Tunnels

If you are having trouble getting the remote SSH options to work, check with netstat which interface the listening port is attached too. Even though we have specified 0.0.0.0 in the above examples, if GatewayPorts is set to no in the sshd_config then the listener will only bind to localhost (127.0.0.1).

6. Establish a VPN over SSH

A common term amongst offensive security folks (pentesters / red teams / etc), is to pivot into a network. Once you have a connection established on one system that system becomes a gateway point for further access to the network. This is known as pivoting and enables lateral movement through the network.

We can use the SSH proxy for this and proxychains, however there are some limitations. For example we cannot use raw sockets, so Nmap SYN scans cannot be used to port scan the Internal network.

Using this more advanced VPN option we move the connectivity down to layer 3. We can then route traffic through the tunnel using standard network routing.

This technique uses ssh, iptables, tun interfaces and routing.

First we need these options set in the sshd_config. Since we are making interface changes on the remote system and the client system, we will need root privileges on both sides.

PermitRootLogin yes
PermitTunnel yes

Then we will establish our ssh connection using the parameter that requests tun devices be initialised.

localhost:~# ssh -v -w any root@remoteserver

Now you should have a tun device when you show interfaces (# ip a). Next step is to add IP addresses to the tunnel interfaces.

SSH Client Side:

localhost:~# ip addr add 10.10.10.2/32 peer 10.10.10.10 dev tun0
localhost:~# ip tun0 up

SSH Server Side:

remoteserver:~# ip addr add 10.10.10.10/32 peer 10.10.10.2 dev tun0
remoteserver:~# ip tun0 up

Now we should have a direct route to the other host (route -n and ping 10.10.10.10).

It is now possible to route any subnet through the other side host.

localhost:~# route add -net 10.10.10.0 netmask 255.255.255.0 dev tun0

On the remote side we need to enable ip_forward and iptables.

remoteserver:~# echo 1 > /proc/sys/net/ipv4/ip_forward
remoteserver:~# iptables -t nat -A POSTROUTING -s 10.10.10.2 -o enp7s0 -j MASQUERADE

Boom! Layer three VPN through an SSH tunnel. Now that's winning.

Any trouble, try tcpdump and ping to see where its broken. Since we are playing at layer 3 our icmp packets should be jumping through that tunnel.

7. Copy your SSH key (ssh-copy-id)

There are multiple ways to achieve this however this command is a shortcut that saves time. What does it actually do? This command replicates what you can also do manually. Copying the ~/.ssh/id_rsa.pub (or the default) key from your system and adds it to an ~/.ssh/authorized_keys file on the remote server.

localhost:~$ ssh-copy-id user@remoteserver

8. Run Command Remotely (non-interactive)

The ssh command can be chained to other commands for the usual piping fun. Add the command you want to run on the remote host as a final parameter in quotes.

localhost:~$ ssh remoteserver "cat /var/log/nginx/access.log" | grep badstuff.php

In this example the grep is being performed on the local system after the log file has been pushed across the ssh session. If the file is large it would be more efficient to run the grep on the remote side by enclosing the pipe and grep in the double quotes.

Another example performs the same function as the ssh-copy-id short cut in Tip 7.

localhost:~$ cat ~/.ssh/id_rsa.pub | ssh remoteserver 'cat >> .ssh/authorized_keys'

9. Remote Packet Capture & View in Wireshark

I grabbed this one from our tcpdump examples. Use it for a remote packet capture with the results feeding directly into your local Wireshark GUI.

:~$ ssh root@remoteserver 'tcpdump -c 1000 -nn -w - not port 22' | wireshark -k -i -

10. SSH Copy Folder from Local to Remote

A neat trick that compresses a folder using bzip2 (that's the -j in the tar command), then extracts the bzip2 stream on the other side creating a duplicate of the folder on the remote server.

localhost:~$ tar -cvj /datafolder | ssh remoteserver "tar -xj -C /datafolder"

Copy remote folder to local tar archive

To go the other way, copying a remote folder to a local archive. Handy for quick backups of remote resources.

localhost:~$ ssh user@remoteserver "tar -jcf - /path/to/backup" > dir.tar.bz2

11. Remote GUI Applications with SSH x11 Forwarding

If the client and remote server both have X installed. It is possible to run a GUI command remotely, with the Window appearing on your local desktop. This feature has been around since the beginning of time, but can still be very useful. Run a remote web browser or even the VMWawre Workstation console as I do in this example.

localhost:~$ ssh -X remoteserver vmware

Requires X11Forwarding yes in the sshd_config.

12. Copy files remotely with rsync and SSH

Using the rsync has many advantages over scp, if periodically need to backup a directory, large numbers of files or very large files it should be used. It has the ability to recover from failed transfers and only copy differences between two locations saving bandwidth and time.

The example here uses gzip compression (-z) and archive mode (-a) that includes recursive copy.

:~$ rsync -az /home/testuser/data remoteserver:backup/

13. SSH over Tor Network

The anonymised Tor Network can tunnel SSH traffic by using the torsocks command. The following command will proxy the ssh connection through the Tor network.

localhost:~$ torsocks ssh myuntracableuser@remoteserver

Torsocks will use the localhost port 9050 to proxy traffic. As always when using tor serious consideration must be taken to understand what traffic is being tunnelled and other operational security (opsec) concerns. Where are your DNS requests going?

14. SSH to EC2 instance

When using SSH to connect to your EC2 instance within Amazon you will need to use a private key. Download the key (extension .pem) from your Amazon EC2 control panel and change the permissions (chmod 400 my-ec2-ssh-key.pem. Keep this key somewhere safe or put it in your ~/.ssh/ folder.

localhost:~$ ssh -i ~/.ssh/my-ec2-key.pem ubuntu@my-ec2-public

The -i parameter tells the ssh client to use this key. This would be an ideal example of where to use the ~/.ssh/config to configure the use of the key automatically when connecting to the ec2 host.

Host my-ec2-public
   Hostname ec2???.compute-1.amazonaws.com
   User ubuntu
   IdentityFile ~/.ssh/my-ec2-key.pem

15. Edit text files with VIM over ssh/scp

For all those vim users out there, this one can save some time. Using vim we can edit files over scp with one command. Using this method creates a file in /tmp on the local system and then copies it back once we write the file in vim.

localhost:~$ vim scp://user@remoteserver//etc/hosts

Note the format is slightly different to regular scp. After the host we have a double //. This references the absolute path. A single slash will have a path that is relative to the users home directory.

**warning** (netrw) cannot determine method (format: protocol://[user@]hostname[:port]/[path])

If you see this error, double check the format of your command. It usually means there is a syntax error.

16. Mount remote SSH location as local folder with SSHFS

Using sshfs - an ssh filesystem client, we can mount a local directory to a remote location with all file interaction taking place over the encrypted ssh session.

localhost:~$ apt install sshfs

On Ubuntu and Debian based system we install the sshfs package and then mount the remote location.

localhost:~$ sshfs user@remoteserver:/media/data ~/data/

17. SSH Multiplex using ControlPath

By default when you have an existing connection to a remote server with ssh, a second connection using ssh or scp will establish a new session with the overhead of authentication. Using the ControlPath options we can have the existing session be used for all subsequent connections. This will speed things up significantly. It is noticeable even on a local network but even more so when connecting to remote resources.

Host remoteserver
        HostName remoteserver.example.org
        ControlMaster auto
        ControlPath ~/.ssh/control/%r@%h:%p
        ControlPersist 10m

ControlPath denotes a socket that is checked by new connections to see if there is an existing ssh session that can be used. The ControlPersist option above means even after you exit the terminal, the existing session will remain open for 10 minutes, so if you were to reconnect within that time you would use that existing socket. See the ssh_config man page for more information.

18. Stream Video over SSH using VLC + SFTP

Long time users of ssh and vlc (Video Lan Client) are not always of aware of this handy option for when you need to watch video over the network. Using the vlc option to File | Open Network Stream one can enter the location as a an sftp:// location. A prompt will appear for authentication details if password is required.

sftp://remoteserver//media/uploads/myvideo.mkv

19. Two Factor Authentication

Most readers will understand the value in using Two Factor Authentication, the same benefits that apply to your banking or Google Account can be applied to your SSH service.

Of course ssh comes with a form of Two Factor capability included, that being a passphrase and an SSH key. An advantage of using a hardware based token or the Google Authenticator App is the fact that they are generally coming from a second physical device.

See our 8 minute guide to getting started with Google Authenticator and SSH.

20. Bouncing through jump hosts with ssh and -J

When network segmentation means you are jumping through multiple ssh hosts to get to a final destination network or host, this jump host shortcut might be just what you need. Requires OpenSSH version 7.3+.

localhost:~$ ssh -J host1,host2,host3 user@host4.internal

A key thing to understand here is that this is not the same as ssh host1 then user@host1:~$ ssh host2, the -J jump parameter uses forwarding trickery so that the localhost is establishing the session with the next host in the chain. So our localhost is authenticating with host4 in the above example; meaning our localhost keys are used and the session from localhost to host4 is encrypted end to end.

To use this ability in the ssh_config use the ProxyJump configuration option. If you regularly have to jump through multiple hosts; use the config file and your alias to host4 will save you a lot of time.

21. Block SSH Brute Force Attempts with iptables

Anyone who has managed an SSH service on the Internet, and viewed the logs will be aware of the amount of SSH brute force attempts that take place every hour of every day. An immediate way to reduce the noise in your logs is to move SSH to a port other than 22. Make the change in the sshd_config file using the Port ## configuration option.

Using iptables we can also block attempts to connect to the port from sources that reach a certain threshold. A way to do this is to use OSSEC, as this not only blocks SSH but will also perform a bunch of other host based intrusion detection functions (HIDS).

22. Modify Port Forwarding within a session with ~C

And our final ssh example is for modifying port forwarding on the fly within an existing ssh session. Picture this example scenario. You are deep in a network; perhaps you have jumped through half a dozen jump hosts and need a local port on your workstation forwarded to Microsoft SMB on the old Windows 2003 system you spotted (ms08-67 anyone?).

After hitting enter try typing ~C in your terminal. This a control escape sequence within the session that allows to make changes to the existing connection.

localhost:~$ ~C
ssh> -h
Commands:
      -L[bind_address:]port:host:hostport    Request local forward
      -R[bind_address:]port:host:hostport    Request remote forward
      -D[bind_address:]port                  Request dynamic forward
      -KL[bind_address:]port                 Cancel local forward
      -KR[bind_address:]port                 Cancel remote forward
      -KD[bind_address:]port                 Cancel dynamic forward
ssh> -L 1445:remote-win2k3:445
Forwarding port.

You can see here we have forwarded our local port 1445 to the Windows 2003 host we found on the internal network. Now launch msfconsole and we are good to go (assuming you were planning on exploiting that host).

Wrapping Up

These ssh examples, tips and commands are intended to give you a starting point; additional detail on each of the commands and capabilities is available using the man pages (man ssh, man ssh_config, man sshd_config).

Being able to reach out and run commands on systems anywhere in the world has always fascinated me. By developing your skills with tools such as ssh you will become more productive and effective at whatever game you play.

Thanks for reading and if you have any comments or suggestions please drop me a note using the contact form. Have fun!

The post SSH Examples, Tips & Tunnels appeared first on HackerTarget.com.