Guide to Happy Examiners – External Testing Selection

4/14/22 - We all want happy examiners . . . and board members . . . and clients . . . and employees. This guide will prepare you for a room full of smiles at your next examination . . . at least from an information security standpoint on your external network.

The term “Penetration Test” has been thrown around a lot in the Information Security industry. Some vendors and institutions use the term Penetration Test interchangeably with “Vulnerability Scan” (or Assessment), when in fact, the two define very different scopes, methodologies, and deliverables. The most recent update of the FFIEC Information Security Booklet discusses these types of tests and offers definitions and expectations of what is required of financial institutions in these areas. The short story is that yes, both are different, and yes, both are needed as part of an effective audit program.

Without wandering too far into the weeds, this post will attempt to shed some light on these two testing types, as well as methodologies associated with each.

Section IV.A.2(b) of the booklet released in September 2016 defines a Penetration Test as a test that “subjects a system to real-world attacks” and should “demonstrate a potential for loss.” Penetration Tests should simulate the methods and goals of an attacker that is determined to gain access to your systems. It is designed to show how all of an organization’s layered controls worked together (or did not work) to defend against a hacker. It is performed with “shields up” and often internal staff will respond to any detected attack as if it were real. The scope will generally not only consist of technical attacks but can also include social engineering and even physical penetration attempts. The resulting deliverable reads like an after-action report, detailing from start to finish how the “attackers” found vulnerable systems, if they breached the network, and what they would have accomplished if it were a real-world scenario. The report should then list what defensive controls could have detected and stopped each attack. This helps the institution identify gaps and weaknesses that need to be addressed.

On the flip side, Section IV.A.2(c) describes a Vulnerability Assessment as a process that “identifies, and classifies the vulnerabilities in a computer, network, or communications infrastructure.” These assessments are limited in scope and methodology, and are done with full access and knowledge of an organization’s network. The goal with this test is not to simulate an attack, but to identify all vulnerabilities or weaknesses in a given system or environment. The subsequent report is an exhaustive list of systems, the vulnerabilities identified on each, risk classifications, and the recommended remediation steps. It does not take into account other mitigating controls or real world consequences of exploitation; it simply helps administrators and management identify vulnerabilities that need to be remediated.

Both of these are essential, useful tools that help an institution obtain a clear determination of their resilience against cyber-attack, and both paint a different part of an overall picture of the security posture of your network. If you think of your network as a medieval castle, the Vulnerability Assessment will identify all of the cracks and weaknesses in the wall, whether the drawbridge and gates were installed correctly, and whether the tower walls are tall enough to prevent climbing. A Penetration Test, on the other hand, will show how real attackers performed reconnaissance from the tree line, jumped the moat, found a hole in a wall, slipped past the guards undetected, and found the keys to the tower door (thereby eliminating the need for climbing). It will also demonstrate in a very real way how the attackers were able to successfully make off with crown jewels, showing the King and Queen just how important it is to fix the weaknesses that were exploited (and they get their jewels back!).

FFIEC guidance and examination procedures indicate that while institutions can determine the frequency and types of Penetration Tests and Vulnerability Assessments, it does not say you can pick one or the other. Both are needed to help assure that your Information Security Program is providing adequate protection from cyber-threats. Additionally, recent feedback 10-D has seen from examiners appears to confirm that all institutions, regardless of size, need to include both methodologies in their audit and testing program.

Another thing to keep in mind is that the terms are still confused by many vendors when offering proposals. Some scopes may say Penetration Test, but organizations must look closely at methodology and deliverables to determine what is actually going to be done. True Penetration Tests are conducted by skilled Security Professionals with experience tailoring available tools and techniques to attack each unique environment. They are more manual and time intensive, and thus will generally cost more than a Vulnerability Assessment. Some key components of a Penetration Test scope will be:

• A Penetration Test will list the number of hours the test will be performed. If hours are missing from the scope, the provider is typically only running tools, and the testing is over when the tool is done running. This leaves a lot un-tested that a cyber-attacker would attempt to exploit.
• Goals or “Flags” to capture. A Penetration Test is targeted, meaning that just like an attacker, the test has a goal to accomplish. This can be collecting customer information, compromising a particular server, or just gaining Domain Admin level access.
• Rules. Laying out what is in-scope and what is not in-scope is important. The rules should define what, if anything, is off limits.
• Methodology. Again, the methodology for a Penetration Test is completely different. Beware of scopes that contain “vulnerability scanning.” While a Pen Tester will look for vulnerabilities, they generally will not perform full-scale vulnerability scanning. If this is involved, the scope may be more of a Vulnerability Assessment than a true Penetration Test.

Vulnerability Assessments are generally a simpler scope, but methodology matters here as well. When evaluating Vulnerability Assessment scopes, a few things to look for:

• What is scanned? It is highly recommended that all network devices be scanned. Do not scan just servers and workstations, include entire subnets. This can find issues and even devices that have fallen through the cracks.
• Authentication. As a Vulnerability Assessment does not simulate an attacker, and is more of an administrative security test, these scans should be run with Domain Administrator level credentials when assessing Windows systems, and Root credentials when scanning Unix based boxes wherever possible. This will allow the scanning software to scan each system for installed software, determine exactly what versions are present, and will be able to find vulnerabilities that un-authenticated scans cannot. Remember, admin level credentials are needed because you are scanning all systems remotely from a central location. This does not mean that an attacker would need the same privilege level to exploit the issue. A good example of this would be out of date Java. Any user that can log into the system can open Java and see the version, but if you want to check the version from a remote system on the network, you need administrative level access to perform that remote query. An attacker exploiting this issue merely needs to entice a standard user to click on link in a phishing email to exploit it and compromise the system.

10-D leads the industry with a wide variety of penetration testing and vulnerability assessment services to financial institutions. The services include:

External Penetration Test – A real-world cyber-attack simulation. Engineer starts with institution name and URL for identification purposes and works for the time allotted to find a way into your network. This will thoroughly test multiple layers for defense from outside your network. Also called a 'Black-box' Penetration Test.

Limited External Penetration Test –Limited External Penetration Test – Commonly called the ‘scan and poke’, the methodology for this test begins with Nessus scanning of all External IP addresses by 10-D. Once scanning is complete, our engineer tests the found vulnerabilities to determine if they are exploitable. Due to the methodology of this test, 10-D only offers to smaller institutions.

External Vulnerability Assessment - Our External Vulnerability Assessment tests your network’s public access areas for vulnerabilities and security issues. This gives you the opportunity to correct problems before malicious attackers can exploit them to gain access to sensitive information.

Internal Penetration Test - Our Internal Penetration Test gauges the effectiveness of your internal security controls against an attacker with access to internal network resources. This attacker could be a knowledgeable malicious insider, an external attacker that has gained limited access, or a “beachhead” on the internal network. The purpose of this test is to simulate a real-world attack with specific goals, generally gaining root or administrative access to targeted systems, or access to data stores.

Internal Vulnerability Assessment - Our Internal Vulnerability Assessment deliverable includes a risk-rated listing of detected vulnerabilities and mitigation recommendations, which provides a clear roadmap for prioritizing and resolving detected deficiencies. We also provide repeat clients with trend reporting and graphs that illustrate the institution’s progress throughout 10-D engagements over time.

Wireless Penetration Test –Wireless Penetration Test – The scope of our Wireless Penetration Test will test access controls, authentication and encryption controls, wireless sniffing and scanning, session hijacking and scanning, as well as end-point coaxing. We’ll challenge your team’s monitoring, controls, and incident response. Our Red Team employs innovative techniques and strategies used by today’s cybercriminals to detect and evaluate your security controls.

Mobile Application Penetration Test –Mobile Application Penetration Test – Mobile devices are at ever-increasing risk of compromise and your institutions mobile apps can be at risk. Mobile Application Penetration testing uses current methods and strategies to attempt to circumvent controls. Exploitation will be attempted against any potential vulnerabilities to determine viability and impact.

Purple Team Threat Assessment -Purple Team Threat Assessment - The goal of this Purple Team “Ransomware Simulation” Test is to gauge the effectiveness of security controls against a specific threat, while collaborating with bank staff on control visibility and effectiveness. The purpose of this test is to simulate a real-world attacker utilizing specific tools and methods. This test differs from a standard “Red Team” Penetration test in that it is conducted in concert with bank security staff to simulate specific attack methods and determine what controls are effective at detecting and stopping the threat.

We’ve also developed a guide to help you choose the right methodology for your testing. Use the table below to help you identify the testing you need, and the scope examiners want to see.

	Meet Examiner Requirements	Test Layers of Defense	Primarily Manual or Automated	Limited Access & FI Knowledge	Price
External Penetration Test	✓	✓	Manual	✓	$$
Limited External Penetration Test	⁇	⭙	Majority Automated	⭙	$$
External Vulnerability Assessment	✓	⭙	Automated	⭙	$
Internal Penetration Test	✓	✓	Manual	✓	$$$
Internal Vulnerability Assessment	✓	⭙	Automated	⭙	$$
Wireless Penetration Test	✓	✓	Manual	✓	$$
Mobile Application Penetration Test	✓	✓	Manual	✓	$$$
Purple Team Threat Assessment	✓	✓	Manual	⭙	$$$

Cybersecurity and FFIEC regulations are fairly complex topics on their own. Couple them together and add in examiner expectations along with complexities of individual institutions, and the difficulties of selecting the appropriate methodology for your unique environment can be daunting. 10-D has 18 years of experience and expertise helping financial institution all over the country make this decision. We are more than happy to help your institution as well. Please visit our website, www.10dsecurity.com, or email bgoetsch@10dsecurity.com, and we’ll arrange an opportunity to chat with one of our expert resources.

Reference:

FFIEC Information Security Booklet IV.A.2, Types of Tests and Evaluations https://www.ffiec.gov/press/PDF/FFIEC_IT_Handbook_Information_Security_Booklet.pdf

Authored By: Jeremy Johnson, OSCP, CISSP

  Download Blog