Rebuilding a patch and vulnerability program
Snapshot
As part of a massive effort around PCI compliance, a large retail company must meet and potentially exceed all PCI (payment card industry) requirements. In the past, parts of the company’s infrastructure would be managed by centralized teams. Because the company is currently moving to an agile type of environment, every team must now take full responsibility for their infrastructure. The engagement involved a particular team that was still ramping up on the skillsets involved with this type of infrastructure management.
Part of PCI compliance involves patch and vulnerability remediation management, a challenge the company was still trying to understand and solve. A team within the company was scanning all infrastructure with a well known security tool and sending results to teams via email. The results varied in format and only came once a month. The timing of this might or might not coincide with the times that systems were set to patch. This and other issues made managing and auditing the current state of patch and vulnerability management a long, manual and expensive process that often produced inaccuracies.
Finally, though the company was moving to agile methodology, access to centralized update services and vulnerability scanning was still held by one team and not shared. The Identity and Access Management (IAM) team needed a way to understand when and if something was patched and to manage and implement all remediation that could not be done via patching. Their systems were used by the entire company to manage access, so system uptime was of the highest priority. This team needed auditing, scripting, testing and deployment strategies for vulnerabilities as far back as 2015.
Challenge
To understand project scope, Kalles Group (KG) audited and combined all available data from multiple locations and formats which was key to understanding the current state of infrastructure vulnerabilities. Once the data was gathered, modeled, and analyzed, the team then moved to vulnerability remediation strategies and planning.
Vulnerability discovery and patch management was handled by centralized teams. However, the entire company was still trying to understand how to remediate vulnerabilities that were being reported and not part of patch management. These generally required configuration changes, software installation, and were at high risk for impact to services. KG needed to work with many teams to identify solutions and strategies. The findings were added to the project plan for current vulnerability remediation.
Approach
All data was imported, joined and put into a PowerBI dashboard that was secured with role-based access. All vulnerabilities that were not remediated were analyzed and put into a project plan on a sprint format. Each vulnerability was individually automated, tested, deployed and verified. Each vulnerability provided a unique challenge and had to be automation-tested in multiple environments. Downtime risk assessment, solution complexity, and the scope of applicable systems were all considered when planning sprints.
Automation and verification was done via PowerShell. Data was exported to PowerBI dashboard for the team to be able to track the progress. Once caught up, analysis of a potential solution for ongoing management was presented to leadership, including a detailed presentation and demo with features, risks and estimated costs of the proposed solution.
Results
Except for one vulnerability that was going to be taken care of companywide, all systems were brought to PCI vulnerability compliance within 60 days. PowerBI dashboard was created and presented with training on how to use it and how to load data into it. Automation with the configuration management database was explored but was pushed to a later date.
New ‘remediation reports’ were emailed to the team every two weeks for updates without the dashboard. Full evidence for PCI audit was exported from the dashboard and will be exportable for the future.
Three years of past vulnerabilities were analyzed, automated, tested, deployed, and audited within 60 days. A full analysis and presentation of Microsoft Operational Management Suite was given as a recommendation for future management processes.
Except for one vulnerability, all systems were brought to PCI vulnerability compliance within 60 days.