Enhancing security reviews with scripted penetration testing
Snapshot
The Artificial Intelligence and Research (AI&R) organization at the Microsoft corporation is a group of scientists and engineers who work together to help solve global challenges. The Compliance and Security (C&S) team within AI&R did not have an existing penetration
testing capability for applications or web services deemed in-scope when performing security reviews.
Therefore, the C&S team was tasked with designing and implementing a penetration testing capability to enhance the organization’s application and service security review function. In addition, it would be imperative that the process and tooling be identified and documented while aligning to the existing corporate policy.
Challenge
Kalles Group (KG) was brought in to partner with the C&S team and help determine the best approach to testing different applications and services. Together with KG, the team worked with key stakeholders via strategy sessions to help define what should be considered in-scope for penetration testing as well as the depth and breadth of the penetration testing as applicable to high-level project types.
Developing this capability as a standalone function and accomplishing what would be needed when testing applications and services within the AI&R organization proved to be challenging due to the extremely rigid guidelines that are part of Microsoft security that exists outside AI&R. Additionally, the team had difficulty determining how to succinctly identify when a penetration test would be required by an application or service.
All penetration testing of in-scope applications and services is completed internally within the AI&R team, resulting in less lead time, quicker turnaround, and reduced dependencies to complete testing.
Approach
The penetration testing function became embedded into the security review process as it would help itemize when the work is required as well as track its completion. The team leveraged a number of publicly available tools but also internal toolsets in order to minimize procurement as well as maintain or exceed the capability provided by other internal teams that perform this function within Microsoft.
Because it is imperative that internal Microsoft teams know ahead of time about testing and ongoing work that may occur, KG and the C&S team developed a methodology where they proactively reach out to internal Microsoft teams when deemed appropriate based on the testing that will be done.
Results
The AI&R organization now has a security sign-off for an application or service that indicates it has been penetration-tested. All penetration testing of in-scope applications and services is completed internally within the AI&R team, resulting in less lead time, quicker
turnaround, and reduced dependencies to complete testing.
Furthermore, owning penetration testing efforts allows the team to adjust accordingly for each project where applicable to focus on what needs to be tested instead of simply performing a basic checklist-style test. As this is an ongoing capability, the team continues to
refine the solution.