For the second post in the cyber series for business owners, I’d like to focus upon penetration testing. Again, credit goes to infosec engineer Shaggie Scheferman for his technical input. I’d like to first establish what a penetration test is (and what it is not), look at some of the reasons why organizations invest in this type of testing, and ultimately lead the reader to ask “What does my organization want out of a penetration test and why?” I’ll also share some important considerations when it comes to selecting a partner to conduct a penetration test.
What is a Penetration Test?
There are a lot of different ways that penetration testing is described, conducted and marketed. Often confused with conducting a “vulnerability scan”, “compliance audit” or “security assessment”, penetration testing stands apart from these efforts in a few critical ways:
- A penetration test doesn’t stop at simply uncovering vulnerabilities: it goes the next step to actively exploit those vulnerabilities in order to prove (or disprove) real-world attack vectors against an organization’s IT assets, data, humans, and/or physical security.
- While a penetration test may involve use of automated tools and process frameworks, the focus is ultimately on the individual or team of testers, the experience they bring to the test, and the skills and wherewithal they leverage in the context of an active attack on your organization. This can’t be over-emphasized. Even highly automated, well-resourced, and advanced networks employing sophisticated counter-measure technologies are often vulnerable to the unique nature of the human mind, which can think laterally and outside of the box, can both analyze and synthesize, and is armed with motive and determination.
- A penetration test is designed to answer the question: “What is the real-worldeffectiveness of my existing security controls against an active, human, skilled attacker?” We can contrast this with security or compliance audits that check for the existence of required controls and their correct configurations, by establishing a simple scenario: Even a 100% compliant organization may still be vulnerable in the real world against a skilled human threat agent.
- A penetration test allows for multiple attack vectors to be explored against the same target. Often it is the combination of information or vulnerabilities across different systems that will lead to a successful compromise. While there are examples of penetration testing that limit their scope to only one target via one vector (example, a web application pen test conducted only from the point of view of the Internet browser), their results should always be taken with a grain of salt: while the test may have provided valuable results, its results are only useful within the same context the test was conducted. Put another way, limiting scope and vector yields limited real-worldunderstanding of security risk.
What is the Value of a Penetration Test?
Here are a few of the reasons organizations invest in penetration testing:
- Determining the feasibility of a particular set of attack vectors
- Identifying higher-risk vulnerabilities that result from a combination of lower-risk vulnerabilities exploited in a particular sequence
- Identifying vulnerabilities that may be difficult or impossible to detect with automated network or application vulnerability scanning software
- Assessing the magnitude of potential business and operational impacts of successful attacks
- Testing the ability of network defenders to successfully detect and respond to the attacks
- Providing evidence to support increased investments in security personnel and technology to C-level management, investors, and customers
- Meeting compliance (for example: the Payment Card Industry Data Security Standard (PCI DSS) requires both annual and ongoing penetration testing (after any system changes)
- Post security incident, an organization needs to determine the vectors that were used to gain access to a compromised system (or entire network). Combined with forensic analysis, a penetration test is often used to re-create the attack chain, or else to validate that new security controls put in place will thwart a similar attack in the future.
As is apparent, there are many reasons penetration testing is conducted. Defining the scope and nature of a penetration test is largely dependent on what the drivers are for an organization, which will determine the stated goals going into an engagement. Those drivers may also influence other aspects of the engagement such as target selection scope, assumptions, and even funding ceilings that limit the amount of time a test team has to explore and compromise the organization’s assets. For example, if the goal is merely to ‘check off the box’ that says an organization has conducted penetration testing in order to meet compliance, then the scope and allocated funding may be much more constrained. Contrast that with an organization that is genuinely worried about its intellectual property and cares about the real-world risk to that IP from a motivated, skilled attacker’s perspective, and you might want to allocate a budget amount that will allow for a more thorough test.
This brings us to the most important question of the day: “What is the right kind of penetration for your organization?” If penetration testing is something that is mandated in your industry, you may be tempted to find the lowest cost automated pen testing service available so that in a few days’ time you’ll have a branded report on your desk and be finished. But there are a few considerations to think about before going this route. 1) If you are already going to allocate funds to do a penetration test, and since it is already mandated that you have one done, then whey not leverage the situation to gain the benefits of a more comprehensive and personalized test? 2) If you go with a light weight online pen test service and then get compromised later down the road, will you be able to rationally defend your selection of pen testing service, and the associated narrow scope of coverage? Ultimately, if we care about the security of our people and our data, it is the real world threat that counts the most. While compliance requirements may be a necessary evil, they do not equate to a necessarily secure environment. We’ve seen this time and time again in the Department of Defense in the context of meeting FISMA requirements. So much effort is spent meeting compliance requirements that sometimes actual operational security isn’t assessed adequately. It is easy to forget the reasons for having security in the first place when we are running around just trying to validate compliance, instead of analyzing real world threats and risks that are the ones that lead to eventual compromise. This is called “doing the job right instead of doing the right job.”
What do I need to look for in a Penetration Testing Service Provider?
So, let’s assume we want as real-world a pen test as our budget allows. What are some of the things to look for from a pen test service company during the selection and agreement phases? Here are some suggestions gained from experience:
- Hire the right talent. Ultimately, you are hiring a team of people with experience, skills, and tools to do the job right. Pen testing is an inherently high-risk endeavor. Things break, stuff goes wrong, alarms go off (hopefully) and that is the whole point. Make sure the team you are hiring is experienced and ask them detailed questions about how they come up with a test plan, rules of engagement, and the final reporting content. If an un-experienced penetration tester is hired, you’ll have just as many alarms go off, but you may not have any positive test results to show for their efforts. The last thing you want after a penetration test is no actionable results that come out of it. This is not the time to feel good about your security after a pen test doesn’t uncover weaknesses on your network!
- Pay attention to scope. This is one of the trickiest parts of any penetration test, and the right team will be the one that helps you both determine what should be scoped into the target environment and what should be scoped out. Before the test begins, there should be clearly defined IP address ranges, external URLs and IP addresses, and applications both internal and external that are defined. Other scope considerations include the degree to which social engineering is acceptable and if there are any off-limits people that should not be targeted. Similarly, physical access to everything from buildings to dumpsters should be defined at the outset. By limiting scope, you effectively focus more effort on those areas of you organization you want to be tested. And you also prevent unacceptable actions from being taken against resources that are deemed off-limits. Often over-looked, scope should also be prioritized as much as possible so that the test team spends focused time on high value assets, etc. You want to strike a balance between too broad and too narrow a scope, based in part on your budget. If it is defined too broadly, efforts will not be focused properly in the allotted time. If it is too narrow, however, the testers may not be given enough lateral flexibility to explore alternate paths towards real-world exploitation.
- Blackbox vs. Whitebox.There are advantages and disadvantages to both. A Whitebox test (in which the attacker is pre-loaded with information or network access going into the engagement that would be difficult to obtain on their own) has two advantages: 1) less time and money is spent on the discovery, reconnaissance and enumeration portions of the test, leaving more time and money to be spent on breaking applications, network devices, people, etc. 2) The threat posed by insiders is often underestimated by organizations that entrust them to physical and logical access to IT resources. By its very nature, whitebox testing allows the attacker to be one step closer to the internal environment and may help uncover vulnerabilities in internal applications that a blackbox test might not. The advantages of a blackbox test (in which only a small amount of an organization’s information is provided, or only that which is readably uncovered via Internet searches and making phone calls into the organization) include: 1) it provides the best ‘real-world’ perspective of the organization from an external attacker’s perspective 2) it naturally forces the attacker to spend time uncovering information on the organization that is public or able to be social engineered out of employees or partners. By analyzing the results of this process, an organization will learn a tremendous amount about how an attacker can gain a foothold in the organization starting from scratch, and then be able to take steps to mitigate or remediate those vulnerabilities.
- Goals and Objectives.By establishing what the overall goals of the test are going in, you will allow the test team to produce a report that caters to those goals and addresses them. If there is a particular hot button you want to make sure is addressed, be sure to include it outright in the goals. Understand that not all of the goals may be met during the test, and in some cases this may be a good thing! (e.g. test the ability to access the development environment from the production network and attempt to access source code or other intellectual property)
- Before choosing a test team, be sure to discuss whether or not, and to what extent, recommendations will be made in the report. Don’t assume that a pen test report will include detailed recommendations about how to mitigate or remediate every finding. Ask for a sanitized example of a report and review the recommendations. Are they written in a way that is actionable by your staff after the engagement? Avoid recommendation examples that read like this: “We recommend that your firewall is configured using industry best practices using the concept of least privilege”. That’s simply too high-level to be of value and won’t help your firewall admin know what needs to be changed on the firewall from how it is already configured.
- Schedule the events properly. Work with the test team to determine when certain systems should be tested. You might not want your online payment system to be tested during peak purchase hours, for example. Conversely, you definitely DO want the test team to run a sniffer on the network during normal business hours. The test team should be able to guide the conversation to account for any scheduling considerations before the test begins. If this doesn’t happen, or if the question never even gets asked, it’s a sign you may be headed for a painful experience.
While not exhaustive by any means, those are some good things to keep in mind when selecting and coordinating with a testing partner. One last suggestion: trust your gut. You are about to hire a team of experts to thwart your security, access your systems via atypical means, and undercover and expose these vulnerabilities in a consolidated report. Make no mistake; you are inviting a third party to penetrate deep into your organization. That is both the primary value of a pen test, as well as a red herring to trust your gut instincts. If you find yourself not trusting either the integrity or the capability of a test team, walk away. One significant mitigation to look for is whether a company holds a security clearance with one of the U.S. government branches. If the company itself holds a Secret clearance (ideally Top Secret) or higher, that means that the U.S. government has already investigated and trusted the company with their secrets, so you have at least one additional data point on the validity and integrity of the company.
The Take Away
At the end of the day when it comes to your data, your networks, your business and your people, one thing matters most: real-world security. The value you gain from a penetration test is largely dependent on your choices in who you trust as a partner, what degree of freedom you entrust them to operate within, and how they cater their reporting to your organization’s needs. Getting a penetration test is a bit like going to get an MRI: It’s never something you want to do, and you hope the results come back negative, but you do it because you want peace of mind and you want to know what things look like in the real-world.