The security operations center (SOC) at the University of Texas A&M System serves 11 universities and seven state agencies. But with just seven full-time analysts and a risk-rich environment of 174,000 students and faculty, triaging security events was overwhelming.
Security analysts had to look at network flow traffic and logs from disparate systems to determine which security events posed threats that needed investigating. The division of labor was typical: Tier-1 analysts looked at alerts, Tier-2 analysts hunted down likely attacks, and a security engineer dreamed up better ways to make the infrastructure more secure. And even the most knowledgeable analysts took a long time to connect disparate data points to come up with a threat profile.
“[I]t takes your analyst a very long time to build a case and dig through that information," said Dan Basile, SOC executive director for the A&M System. "The mean time to resolution—from the point when you think there is an attacker in the system to remediation—used to be measured in hours, if not days, on average."
Then the university adopted a trainable, machine-learning-based security system —artificial intelligence—for incident detection and triage. It now takes 10-to-20 minutes to resolve an incident, on average, and Basile expects that to eventually drop to 5 minutes.
Automation, machine learning, and AI are changing the SOC, and security analysts need a different set of skills to succeed. Here's what you need to know to stay on top.
How AI is transforming operations
In SOCs with little automation, an entry-level analyst must triage security events and then decide which warrant the attention of a more knowledgeable worker, in mych the same way an emergency room technician routes incoming patients to a nurse or a specialist. The process is stultifying, said Peter Guerra, vice president of cyber forensics for Booz Allen Hamilton.
"It is a lot of the cyber drudgery. The entry-level folks have a set of rules, and they can see a lot of what the cyber operations can see, and they are crunching through alerts to apply context."
—Peter Guerra
To keep up with attackers, however, SOCs must intelligently automate. Guerra, for example, worked with one client to reduce the time to remediate spearphishing attacks from three hours per incident to less than two minutes per incident.
What AI means for analysts
Yet, while adopting AI-like technologies can deliver immediate benefits, they can be disruptive for your staff. Analysts' jobs change when security automation and orchestration platforms not only reduce the number of alerts, but automate much of the response.
"Can you even call them a Tier-1 analyst anymore?"
—Guerra
The dramatic changes for Tier-1 analysts in SOCs are not without precedent. The credit-card fraud detection industry used to rely on legions of analysts to look at transactions and detect patterns of fraud—usually after a transaction had been allowed. With automation, pattern recognition, and orchestration added to the mix, the need for analysts dramatically decreased.
It won't be the same model
But there is one major difference. You could easily determine whether a credit-card transaction is fraudulent or legitimate, so AI systems were quickly trained and didn't necessarily have to explain themselves because the proof was in the results. That's not true in the security space, said Oliver Tavakoli, chief technology officer for Vectra Networks.
"Credit-card fraud is a much more straightforward system, because you can measure the outcomes at the end."
—Oliver Tavakoli
For that reason, credit-card anti-fraud teams process many more transactions with fewer humans, he said. The result has been a smaller number of jobs in the credit-card fraud space, and a more skilled workforce that can deal with these systems.
That's not expected to happen in cybersecurity. While automation and machine intelligence can ease the workload for security analysts, there will not be a massive reduction in the SOC workforce, Tavakoli said.
Unlike credit-card fraud, there is no simple answer—accept or deny—for an information-security team. Responses must deal with a large variety of IT systems, and threat hunters have to decipher an even larger number of attack strategies.
The training difference
But automation will still have an impact on analysts' roles, and how companies need to train them. Because of the structure of the typical SOC, many companies view skills acquisition as a pyramid: Tier-1 analysts train up to be Tier-2 analysts, who then become Tier-3 analysts or security engineers.
With Tier-1 analysts morphing into a more holistic position, companies must refocus training programs to focus on skills that allow analysts to work better with AI systems. While automation and machine learning have removed the skill-based nature of many jobs, that won't necessarily be true for SOC analysts, said Anup Ghosh, chief security strategist of next-generation security for anti-malware firm Sophos.
Ghosh likens looking after alerts to a security guard watching monitors.
"I'm not convinced that looking at alerts is a meaningful human activity. That does not give you the right skills to be a forensic analyst. You need to be reversing malware and pulling it apart."
—Anup Ghosh
Delving deeper into problems
With an automated security operations center, analysts will need to focus more on figuring out the "why," rather than trying to identify the "what," A&M System's Basile added.
"Analysts are learning quicker with the new system. The system is saying something is bad, and then they have to go back and see why it is bad and justify that decision."
—Dan Basile
For that reason, Basile looks at the automated SOC as a training ground for new cybersecurity workers. Over the next three months he expects to grow his part-time student workforce to 20, up from five today.
"I work for an educational institution. We are worried about making sure that we are training students to do the jobs of the future."
—Basile
New skills needed
The required skills for analysts will change as well, said Booz Allen Hamilton's Guerra. The latest generation of graduates has typically taken data science classes and knows Python, so asking them to triage alerts is not a good match with their training.
"You want to see the folks that have a lot more skills. ... The big thing that we look for are things like curiosity and the mindset that you are never satisfied with the answer."
—Guerra
Finally, really good communications skills—and in particular being able to explain security issues to the business side of the company—are also very important, he said.
Current Tier-1 analysts should focus on developing data science, programming, and communication skills, as well as studying the attacker, said Vectra's Tavakoli.
"If you are on the defensive side, study the offensive side. It will give you the context to identify what the AI is seeing and understand more about the attacks."
—Tavakoli
Keep learning
Learn from your SecOps peers with TechBeacon's State of SecOps 2021 Guide. Plus: Download the CyberRes 2021 State of Security Operations.
Get a handle on SecOps tooling with TechBeacon's Guide, which includes the GigaOm Radar for SIEM.
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed on cyber resilience with TechBeacon's Guide. Plus: Take the Cyber Resilience Assessment.
Put it all into action with TechBeacon's Guide to a Modern Security Operations Center.