Before, During and After a Cyber Attack: What to Do When the Worst Happens

This blog post was written by Jim Ervin, Involta Senior Network Engineer, CCIE No. 5592.

In our previous post we outlined the various challenges to network security. The plan was to spur intrigue and cause you to review your own system. As companies increasingly rely on email, internet/cloud applications and remote access, the possibility of a cyber attack becomes more likely.

Unfortunately, the threat of an attack is not going away- if anything, the probability is becoming greater. McAfee projected that in 2014 the impact to the global economy was anywhere from $375 to $575 billion. Simply put, companies and governments can no longer ignore this critical threat to their bottom line.

With the threat of a cyber attack being real, what should you do if attacked? Here are the steps to take before, during and after:

Before a Cyber Incident

1. Have a plan. The key is to have a set of procedures in place before your IT infrastructure suffers an attack. The time for planning is now, not while the attack is occurring, and planning should not be an afterthought. The plan should be well documented, reviewed, tested, tweaked and perfected. If you have a good plan, then your chances of surviving an attack are far greater.
2. Identify critical assets. Which programs, servers and processes would prevent you from doing business if they were no longer present? It is important that you’ve taken steps to identify and adequately protect these critical systems. If you don’t concentrate on what really keeps your business running, then all the planning in the world isn’t going to help. Some companies may rely on databases and web portals, while others may have phone systems that are necessary for maintaining business continuity. Obviously, each company’s needs are unique. You may not have the monetary assets to protect everything to the same degree, so prioritizing your critical infrastructure is important.
3. Implement appropriate technology to protect critical assets. This is the software and hardware that is needed to respond to a cyber incident. This could include, but is not limited to, Intrusion Detection and Prevention (IDS/IPS), logging on critical servers (Syslogs), and perhaps most importantly, a good offsite backup system. When I got my first job after leaving the Navy in 1996 my manager offered these words of wisdom, “You need to have either a good backup plan or an updated resume.” I prefer the backup plan.
4. Get legal authorization to monitor internal user activity. We’ve all seen the annoying banners containing “legalese” that nobody ever reads, but they exist for a reason. Banners aren’t enough; the corporate legal counsel needs to be familiar with the company’s cyber incident posture. There are firms that specialize in this type of law, which is especially important when it is time to prosecute the offender(s).
5. Training and education. All of the IDS, firewalls and lawyers are meaningless if users aren’t aware of their responsibility to ensure cyber security. Human Resources should incorporate an “acceptable use” policy into company handbooks so that cyber security is clarified as a serious and critical part of each employee’s job description. Annual training is also recommended along with refresher courses if needed.

During a Cyber Incident

What if you have a solid plan in place, but you’ve still suffered an attack? Sony and Avid Life Media (Ashley Madison) probably had good plans in place, but it didn’t seem to matter. Now what?

1. Assess the situation. Is this a real attack or is one of your servers “acting up?” If you determine that this isn’t just a “glitch,” then it’s all hands on deck. Call everyone in the organization that can help identify the source and destination of the compromise. Is it a virus, a worm or is some “script kiddie” in Russia actually in your system? Look at logs. If logging isn’t enabled, then turn it on now. You’ll need those logs as you recover from an attack.
2. Minimize the damage. This could include implementing firewall rules to block the offending traffic (this is where your IDS system just paid for itself), notifying your ISP to block the offender further upstream, and deciding whether to block or monitor the intruder’s activity. Is the situation so dire as to necessitate taking the system offline in order to mitigate the possibility of the threat spreading deeper?
3. Get those backups ready to go. Depending on the type of compromise, it may be necessary to do a restore from a backup. Remember what my manager told me about the updated resume? This is the moment when that thought will be going through your head. And that backup had better contain relevant, recent data.
4. Gather forensic data from the affected system(s). This could include getting a “snapshot” of the server at the time of the attack (i.e. a backup) or, in some cases, notifying law enforcement so that they can do the same. Your documentation should include network topology drawings, any recent system additions, the identity of personnel working on the affected systems and relevant communication that may pertain to the affected systems under attack.
5. Notify authorities. Bring the incident to the attention of law enforcement, the Department of Homeland Security, company personnel and any potential victims that may be affected by the attack. These notifications may prevent others from propagating the damage done by the attack. It is not only good customer service, but also good cyber etiquette. Failure to notify these potential victims could lead to unwanted legal action.
6. Things NOT to do during an attack include using the affected systems to communicate about the incident. For example, if an email server has been compromised, don’t use that server to send emails about the breach. The attackers may be hoping you will do just that, essentially making a bad thing worse. Don’t attempt to hack into the attacking system, this is probably illegal and could result in civil or criminal penalties.

After a Cyber Incident

The worst case has occurred. How do you recover from this event?

1. Continue to monitor the affected system(s). The attacker may have installed a “back door” into the system that you missed. It is imperative that continued vigilance is at the top of the priority list.
2. Initiate measures to prevent future attacks. Your existing plan was not good enough, so now is the time to shore up these weaknesses. Conduct post-incident reviews with all relevant personnel and engage outside resources that can help ensure that this doesn’t happen again. This isn’t the time to point fingers; it is the time to fix the problem. There will be time later to review employee performance and take appropriate action if necessary.

In Conclusion

No matter how much planning goes into a cyber incident response, things will occasionally happen. New exploits are developed every day. The IT department needs to remain vigilant and ongoing training is an important part of this. Systems need to be constantly patched and monitored. IDS systems need to have their signatures updated DAILY. Utilize all tools at your disposal and learn to use them effectively. Apathy is your worst enemy. The time to learn about a potential exploit is not during or after the event, but before the event. Now is the time to act.