Passing the AWS Security Specialist Cert Part 2: Logging & Monitoring

This is part two of our series on the AWS Security Specialist certification! In part one, we looked at a broad overview of the cert, why you would pursue it, what it covers, and the two foundational AWS services covered under two of the five test domains: IAM and KMS.

We’ll continue the series today with the next domain: Logging and Monitoring. This is a critical domain. With no logging and monitoring, you have no idea what is happening in your environment. You don’t know when things are working as they should and when there are problems. You don’t know if someone is actively trying to hack your applications, or worse, has already gained access. 

Developers need application logs, flame graphs, and historical metrics to troubleshoot problems and improve performance. Security teams need access logs, AWS resource calls, and network logs to investigate incidents. Infrastructure engineers need to keep a tidy house in their AWS accounts and monitor items like rotating access keys, properly locked down security groups, and are any S3 buckets are unintentionally set to public. All this data must be available both in real-time and archived away for reviewing history and possible compliance reviews.

What good logs though without alerts? No one has the bandwidth to crawl through lines and lines of logs looking for problems. We need automation to catch anomalies, metrics hitting thresholds of poor performance, and security configurations falling out of compliance.

This is a tall order to keep your AWS accounts on lock and under close surveillance. Luckily AWS provides some core services in this realm to help you out. Let’s start by digging into the first one, probably the most essential service for keeping an eye on your AWS accounts.

Related: Passing the AWS Security Specialist Exam Part 3: Incident Response and Infrastructure Security.

Ready to Learn AWS Skills in 2023?

AWS is the undisputed leader in the cloud computing market. If you’ve been thinking about earning a certification or upskilling this year, learning AWS is as safe a bet as any. Find the online AWS training you need at CBT Nuggets. 

No matter if you are brand new to the cloud or a seasoned networking pro, our courses can get you up to speed on the latest AWS technologies and best practices. Our AWS cloud training maps to highly valued certifications — and covers skills that many organizations desperately need. 

Not a CBT Nuggets subscriber? Sign up for a 7-day free trial to get a feel of what it’s like to learn IT with us. Explore our AWS training and start learning skills that can help you level up your career today!

CloudTrail

CloudTrail, in AWS’s words, “monitors and records account activity across your AWS infrastructure, giving you control over storage, analysis, and remediation actions.” In more specific terms, it logs every single API call that happens on your account. These calls can be successfully authenticated or not. They can be on a service that the user has access to or when a user gets access denied. They can be from actions taken from the AWS console, from the AWS CLI, or from an SDK. CloudTrail logs any and all API calls.

These API calls can be any request made to an AWS resource, requests like uploading a file to a bucket, creating a new EC2 instance, updating a Lambda function, or even listing the available CloudTrail trails (trails are how CloudTrail logically groups its logs).

Take note of what CloudTrail isn’t. It is not application logs. It is not network traffic or HTTP requests. It is not access allowed or access denied in your applications. It’s not CPU or RAM usage on EC2 instances. It’s not when someone downloads a file from an S3 bucket or an application updates a row in RDS. It’s more about the management logs for your AWS resources.

So what’s in a typical CloudTrail log? Take a look at an example (details have been drastically truncated, see this page for the full log).

{
...
        "userIdentity": {
            "type": "IAMUser",
            "principalId": "111122223333",
            "arn": "arn:aws:iam::111122223333:user/myUserName",
            "accountId": "111122223333",
            "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
            "userName": "myUserName"
        },
        "eventTime": "2019-02-01T03:18:19Z",
        "eventSource": "s3.amazonaws.com",
        "eventName": "ListBuckets",
        "awsRegion": "us-west-2",
        "sourceIPAddress": "127.0.0.1",
        ...
    },
}

There’s a lot here but skimming through it we see the event source (S3), the action taken (listing all the buckets in the account), the user’s name, ARN, and access key, the region, and source IP address of the request. This request could have come from the AWS CLI by running a command like aws s3 ls. It also might have come from a user in the AWS console; when the page is loaded which shows all your buckets, under the hood the console makes an API call to S3, which CloudWatch logs.

One default best practice you must always remember (both for real life and for the exam) is to protect your CloudTrail logs at all costs! CloudTrail sends all its logs to an S3 bucket that you sent up when enabling it. The policies on this bucket must be strict: no read access for anyone except admins and auditors. To further protect the logs there are also digest files created in the same bucket. These digests contain hashes for the log files that can be used to verify that the logs haven’t been tampered with.

There is lots more that can be said about CloudTrail, must for now we must move on in our monitoring adventures to…

CloudWatch

Never let it be said that AWS services aren’t sometimes given annoying similar and sometimes non-descriptive names, but here we are with CloudWatch! It’s like CloudTrail, but… it’s Watch!

Questionable names aside, what is CloudWatch? Per AWS, CloudWatch “collects and visualizes real-time logs, metrics, and event data in automated dashboards to streamline your infrastructure and application maintenance.” Phew. Basically, it’s what we determined CloudWatch is not: application logs, performance metrics, resource events, and the like.

Hopefully, that’s clear enough because we’ll have to muddy the water a little bit further. CloudWatch is actually made up of three components: CloudWatch itself, CloudWatch Logs, and CloudWatch Events.

CloudWatch proper is basically where all your metrics are aggregated. You can build dashboards showing certain metrics on certain resources (like the CPU usages on your critical EC2 instances), then create alarms based on those metrics (like when an EC2 instance sits at over a certain CPU percentage used for a certain period of time). Those alarms can be sent to a notification service like SNS or SMS to text or email you.

CloudWatch Logs is like it sounds on the tin, the logs from your applications, operating systems, and certain AWS services. The classic example is EC2. To accomplish shipping logs, you must install the CloudWatch agent on your instance and configure it to gather the logs you are interested in. This can be a folder on disk where an app drops its logs for example. The agent then takes care of shipping the logs over to the CloudWatch API and within the console, you can view, filter, and search the logs.

Then finally we have CloudWatch Events. This is basically a system with rules that, when triggered, take action against a target. The events themselves that trigger the rules can be one of many types, like a resource state change (like an EC2 instance being created or stopped), specific types of API calls from CloudTrail, custom events from CloudWatch logs (like HTTP status codes from Apache logs), or scheduled events (like cron jobs).

That last item, scheduled events, is particularly interesting as you won’t find any other way in any AWS service to trigger actions on a schedule. Let’s say you have an EC2 instance that you only want to run during the day from 8 a.m. - 5 p.m. Manually doing this is annoying at best. To automate it you must create Lambda functions, one to start and one to stop the instance. Then create two CloudWatch events, one to execute the start script at 8 a.m., one to execute the stop script at 5 p.m. It’s not terribly intuitive and you would think that there would be an easier option for such automation, but this is the AWS way.

AWS Config

A lesser-used but still important service, AWS Config “continually assesses, audits, and evaluates the configurations and relationships of your resources.” Basically, how are things configured now, when they change, who did it, and when?

Imagine you have an EC2 instance with a security group. Best practice says that port 22, the SSH port, should never be opened up to the public internet, so that’s how you have your security group configured. Then one day you notice the group has changed and 22 is open to the entire internet! Perhaps a rogue developer got tired of using the VPN and wanted SSH access directly from home. But who was it?

AWS Config to the rescue! After hunting down the security group in question, you can look at a timeline of every change, including who made a change, what changed, and when. Then to make sure you aren’t caught unaware in the future, you can set up alerting so that any future change to your perfectly tuned security groups triggers an alert.

But AWS Config has more tricks up its sleeve. Triggered rules can also initiate action, like a Lambda script that edits the security group, closing SSH back and keeping that security group in compliance.

A lot of the common rules that you might want to implement (like monitoring for SSH being open) are already built in as AWS Managed Rules; all you need to do is enable them on your instances. Slick! And of course, you can write your own custom rules, both for alerting and compliance to remediate issues via Lambda.

Trusted Advisor

One final essential service to know for the security certification is Trusted Advisor. This is a set of compliance best practice checks that run against your entire account, advising you of any potential problems that need attention and their remediation. The checks are grouped, with security is our prime interest obviously. You’ll find checks for items like CloudTrail being enabled, security groups that might have undesired open ports, or access keys exposed on a site like GitHub.

There are other helpful categories of checks, including cost optimization, performance, fault tolerance, and service limits. You can see all the available checks for each category in the AWS docs.

Trusted Advisor should definitely be your trusted ally in keeping your account properly configured based on AWS’s own recommendations. It can also send data into CloudWatch, from which you can create alarms and notifications, way handier than trying to manually keep an eye on it.

With that, we have finished our overview of the Monitoring and Logging domain of the AWS Security Specialist cert! Next time we’ll finish up with the two remaining domains: Incident Response and Infrastructure Security!