There is a direct link between the time to detection and response and amount of data exfiltrated. Simply put, any delay in incident response will mean MORE lost records, INCREASED loss in revenue, and LOSS of customer trust. Unfortunately, this is a battle against the clock, as the time to compromise is measured in minutes, with exfiltration happening in just days, according to the 2016 Verizon Data Breach Investigations Report. Discovery and remediation of those breaches, however, often takes much longer—often months.
So what is taking so long? A lot of time is wasted on manual processes, including such simple things as data entry. Copying and pasting alert data between systems takes valuable time and resources. Once you have the alert, you may need to go back to the IT system involved to get additional data before work can begin. Also, incident reports, especially those coming via e-mail or phone, may be missing relevant data, requiring someone's time to track that info down.
Getting everyone together for a meeting might require hours to align schedules, not to mention time spent ironing out issues with the conference bridge. Many times finding the right system owners or subject experts can require significant detective work as you get passed from person to person within the organization. The same can be said for finding the right procedures. Even more time is spent manually writing statuses, notifications, and reports for your executives and other leaders.
A good solution, therefore, needs to make things move at a much faster pace and reduce or even eliminate all of those manual tasks. The first step is to establish a single system of record for IR (note: e-mail is not a system of record). This means all parties are working from a single source of information that can track tasks from start to finish and determine if response SLAs are being met. Even better, it should automate some of those basic manual tasks like data entry so that your Tier 1 staff can do actual security work rather than copying and pasting data.
Next, the solution should allow you to retire your paper-based security runbook. Populating the solution with standardized workflows removes any doubt about next steps when an incident occurs. Run simulations through these workflows to refine them and introduce continuous improvements. Staff will learn these repeatable procedures to move from conscious competence to unconscious competence, aka muscle memory.
These standardized workflows also mean that Tier 1 staff can carry out meaningful security work with safeguards in place to help them learn and grow as security analysts. Develop your talent pool internally and provide them the potential for advancement. This also means they can lighten the load of your Tier 2 and 3 staff, who would rather focus on the more complex hunting. Keeping the top staff engaged and interested can reduce turnover, especially since Tier 3 analysts are in such high demand.
In addition to workflows, correlating data with the Configuration Management Database (CMDB) reduces manual investigation and saves time. Analysts can quickly access information about an affected system or resource, including where it resides in the network (e.g. is it in the DMZ?), what business services depend on it, and how critical it is to keeping the business up and running. They can also answer: Were any changes recently made to it?, Who is the owner or SME?, Where does it physically reside?, Does it cross-reference with other systems in the environment?, and Can those be exploited as well?
Now that we have established the process and roles and have vital context about the affected syst ems, it's time to collaborate. Here's where that single system of record becomes crucial—it ensures the entire response team is on the same page to eliminate confusion or duplication of effort. They can tell who owns each system or process without breaking out the org chart. It also makes people accountable, as tasks and decisions are tracked so you always know who has the ball.
This single system also needs to have controlled access. Security data is confidential by nature and often subject to audits and other scrutiny. This means a lot of the methods teams had available came with the challenging choice of either insecure or inefficient. Emails and text messages can be easily forwarded or even sent to the wrong people by accident (thanks autocomplete) #insecure. Documents and spreadsheets are hard to control, resulting in multiple versions or corrupted files when accessed by many #inefficient. None of these methods generate good data for use in an audit.
The best way to track historical data for an audit or process improvement is via a Post-Incident Review/Report (PIR). These typically take hours to produce, are manual, and someone has to track down all of the events and interview the people involved to get the full story. Executives often want the post-mortem to happen ASAP, and many times they ask for them before the dust has settled. Wouldn't it be nice if your incident response system could generate the PIR automatically? A solution that tracks every task will put together a time-stamped review with every action, every communication, and every participant. It could even send assessments to those involved to collect additional data for the review without requiring someone to manually interview each participant.
The other item management always wants in addition to a quick PIR is metrics. Are the right metrics being captured? Can you measure your effectiveness and improvement? The right incident response solution will automatically establish a baseline against which you can measure. This baseline allows you to track improvement and can aid in investigations. If 80% of problems come from 20% of the causes, metrics and a solid baseline can help you find that 20%.
A solid incident response solution reduces manual processes, standardizes response processes, provides vital information about affected systems, and promotes accountability for more efficient response. By enabling security teams to focus more of their time on discovery and remediation instead of searching for contacts or processes, we can start to close the gap between when a system is potentially compromised and when that breach is discovered and remediated, reducing and hopefully preventing data exfiltration and protecting company assets and reputation.
** This blog has been cross posted on LinkedIn https://www.linkedin.com/pulse/manual-tasks-dragging-down-your-incident-response-myke-lyons
