It’s all about the Logs

Looking into Your Past Will Secure Your Network’s Future

By Gary Brown, Principal Consultant/CISO, Mosaic451

We see it repeatedly. The newly installed CISO or CIO installs the latest blinky-box in “the quadrant.” As they discuss all the great features and how it’s going to help protect their network, it’s discovered that while the device will get MOST of the logs, but there are still areas that aren’t logging. Plus, nothing in the back end network getting aggregated and processed either.

A better investment of time and resources would be to get the entire network logging to a centralized log aggregator before organizations spend cash on the trendy blinky-box. Without a complete picture of your network, you get partial information, which can be misleading. Many CISOs understand this when it’s presented, but surprisingly many don’t understand the importance of proper logging.

Complete and accurate logs are the keystone of an effective information security program. Almost every aspect of infosec touches logs at some point. If you can’t query the activity of every device on your network, dedicate yourself to getting that fixed – quickly. Don’t waste money on a SIEM or a honeypot or anything else before you’ve addressed this Infosec 101 prerequisite first.

Many network admins don’t seem to care if their backend machines get broken into, they care about production.  Most companies with this approach do not have truly separate networks. They have VPN connections, bastion firewalls, or other protections that allow them to restrict access, but inevitably their production users live to eat and breathe on the backend as well. They usually get their emails, do time cards, file expenses, and so on, all on the same machine that their VPN connections are made from or their bastion host passwords are typed from. If malware is on that machine grabbing credentials and the security analysts are blind to that, then they very well might not notice anything amiss on the production side. There won’t be failed logins to alert on as the attacker will have the credentials. Don’t fall into that trap. Monitoring your backend is as critical as your production network. You are only as strong as your weakest link, and logging is how you identify your vulnerable spots.

If you are pretty sure you’ve got your devices all ready to log to an aggregator, make sure all your other ducks are in a row.

• Wherever possible, send all logs to a centralized log aggregator – make it the known sense of truth. If you can, try to have your logs on a dedicated resource. That is to say, don’t use your SIEM as a log aggregator, rather send all logs to the aggregator and have them forward to the SIEM (and whatever other resources you need logs from).
• Ensure everything is logging. Applications, operating systems, and network/appliance traffic. Log successes as well as failures. Be careful not to over log though. I’ve seen ambitious logging projects collapse miserably because they simply became overwhelmed with amounts of data they had not projected for. By throttling back from verbose logging to a more targeted approach they were able to get back on track.
• Ensure all sensitive logs are encrypted in transmission. If an attacker can sit on the wire and read your logs, they will be able to pick your bones clean.
• Be sure to spec your log storage devices (and budgets) properly – and plan for expansion. Logs can be large, even with little activity. Anyone who has ever sat watching a Wireshark capture on a Windows network will tell you that the operating system is extremely chatty. It’s surprising how much disk space you can chew up at a fast clip. Don’t size your log storage for current consumption, but also for projected growth.
• At a minimum grab authentication events, security-related events, device errors, database or file actions, and other critical pieces of information required to grab a minimal picture of what is happening on all machines.
• Pick a log format and stick with it. If you can, try to have all logs recorded in a common format. This makes parsing and searching much easier for all concerned.
• Make sure your logs are redundant, backed up, and recoverable. By this I mean:
• make sure your logs are able to fail over to another system if your main aggregator fails
• make sure you make frequent backups, with copies being stored off-site
• make sure you can recover your logs; finding out you can’t during a legal discovery process isn’t a place you want to find yourself in
• Pick a strategy that will best suit your needs. How frequently do you need to access logs, and how far back are your average searches going? Your budget and operational requirements will drive your decision-making process.
• Ensure all devices are syncing their time to the same common source. Timestamp disparities are an analyst’s nightmare, and render logs as potentially useless for investigative or legal use.
• IT/networking teams can very often be helpful in selling costs to the management, or even sharing a budget, as effective logs are also very effective troubleshooting tools. When something breaks, it usually shows up in the logs pretty quickly. Wherever possible, make logging a shared vision between teams.

A well designed and properly implemented logging strategy is the best way to equip your team to effectively detect and investigate suspicious activity, as well as to provide audit capabilities. In this increasingly hostile world, why wouldn’t you want to give them – and yourself – a head start?

About the Author

Gary Brown is Principal Consultant/CISO at Mosaic451.  Mosaic451 is a managed cybersecurity service provider (MSSP) and consultancy with specific expertise in building, operating and defending some of the most highly-secure networks in North America Gary can be reached online at [email protected] and at our company website http://www.mosaic451.com/