AWS Cloud & Pentesting – 2

Chapter 2: AWS Services & Solution Design

This is the Second chapter in my series on AWS Cloud and Pentesting. Please feel free to approach me if you have any questions or recommendations for improvement.

AWS Services

AWS Services are the cloud platform offerings of Amazon. They play an essential role in the global cloud services market. In reality, Amazon provides more than 200 AWS services to fulfill the needs of a range of applications.

AWS Services grow from a variety of technological use cases to industry-specific use cases. Follow this link or this to discover more about the extensive AWS Services that Amazon provides.

In this chapter, we will examine a variety of AWS services in-depth, focusing on the most relevant ones.

AWS Launched with Three Services –

• Storage Buckets – To Store Data, Media, etc.
• Compute Instances – To use instances as servers for different applications.
• Messaging Queue – asynchronous service-to-service communication

Continuously develop diverse services to meet the demands of various user populations.

Elastic Compute Cloud (EC2)

Amazon EC2 is one of the AWS cloud computing services with the most rapid growth, offering virtual servers to manage any type of workload. It provides the optimal processors, networking facilities, and storage systems for the computer infrastructure. As a result, it facilitates exact task adaptation. Amazon EC2 delivers a highly secure, dependable, and efficient computing infrastructure that meets the needs of businesses. In addition, it allows you to swiftly access resources and flexibly increases capabilities based on demand.

Simply put, EC2 helps to create virtual computers in the cloud.

A typical usage scenario for EC2 Instances: Using an instance as a Web Application Server.

There are different types of EC2 Instances:

• General Purpose Instances
• Compute Optimized Instances
• Memory-Optimized Instances
• Accelerated Computing Instances
• Storage Optimized Instances

Details on the different types of EC2 Instances can be found here.

Elastic Load Balancing (ELB)

Load Balancing is essentially the automated allocation of traffic among many instances. ELB distributes incoming application traffic automatically among numerous targets and virtual appliances in one or more Availability Zones (AZs).

Simple Storage Service (S3) Buckets

Amazon S3 is another popular addition to the list of AWS services; it is a highly scalable object storage solution. It allows consumers to access any amount of data from any location. Here, data is kept in “storage classes” to minimize expenses without incurring additional expenditures and to facilitate its management. The data is well protected and facilitates the fulfillment of audit and compliance obligations. With Amazon S3’s sophisticated access controls, replication tools, and heightened visibility, you can manage any volume of data. Additionally, it offers data version management and accidental deletion prevention.

The graphic demonstrates how to transfer data to Amazon S3, maintain data stored in Amazon S3, and analyze data using other services.

You must first collect and transfer your data to an S3 Bucket. From this stored data, it is simple to implement access control, optimize costs, and duplicate these data to any area. Access them from on-premises or virtual private clouds (VPCs) and preserve and secure the data. Most significantly, the data may be analyzed and utilized in a variety of current intelligence settings. All of this is possible by utilizing various AWS Services and the data from the S3 Buckets we just uploaded.

Other Distinct Services related to Storage include –

Amazon S3 Glacier – Amazon S3 Glacier provides inexpensive cloud-based archival storage. It is constructed with three storage classes, including S3 Glacier instant retrieval, flexible retrieval, and deep archive. A disadvantage of this storage technology is its increased latency. These storage classes vary in the following ways:

• Use S3 Glacier Instant Retrieval to archive data that is infrequently accessed and requires retrieval within milliseconds.
• Use S3 Glacier Flexible Retrieval for archives when data must be recovered in minutes. Using Expedited retrieval, data stored in the S3 Glacier Flexible Retrieval storage class may be retrieved in as short as 1 to 5 minutes. Additionally, you may request free Bulk retrievals within 5 to 12 hours.
• Use S3 Glacier Deep Archive to store data that is infrequently accessed. The default retrieval time for data saved in the S3 Glacier Deep Archive storage class is 12 hours.

Amazon Elastic Block Store (EBS) – This storage system is quick and capable of handling high throughputs. This storage system is ideal for applications that require extensive data processing but also require more manual configuration by the developers.

EBS volumes operate like unformatted, uncompressed block devices. These volumes may be mounted as devices on your instances. Attached EBS volumes to an instance are presented as storage volumes that survive independently of the instance’s existence. On top of these volumes, you may establish a file system or utilize them as you would a block device (such as a hard drive). You can modify the settings of a volume associated with an instance in real time.

Amazon Elastic File System (EFS) – EFS expands and shrinks automatically as files are added and removed, without the need for administration or provisioning. It is an expensive, high-performing, and completely managed service.

Elastic Container Service (ECS)

Amazon ECS is a fully managed container orchestration service that simplifies application deployment, management, and scaling. Elastic Container Registry is Amazon’s own container registry. ECS can pull and execute containers. ECS API controls the Start, Stop, and Allocation of containers’ instances. It links to other products as well.

Other Distinct Services related to ECS include –

Elastic Container Registry – ECR is a fully managed container registry with high-performance hosting for deploying application images and artifacts everywhere in a reliable manner.

Elastic Kubernetes Service – Service that enables running Kubernetes. It is a managed container service for running and scaling Kubernetes applications on-premises or in the cloud.

AWS App Runner – If the application has already been created and containerized, it may be deployed quickly using the App Runner. App Runner develops and deploys the web application automatically, balances traffic with encryption, grows to meet traffic demands, and facilitates communication between services and other AWS services and applications that operate in a private Amazon VPC. With App Runner, you have more time to focus on your apps rather than servers or scalability.

AWS Fargate – This Service turns containers into serverless functions. It eliminates the requirement to assign EC2 instances to containers. It is a compute engine that allows you to focus on application development without having to manage servers. The following picture illustrates the facilities offered by Fargate.

AWS Lambda

Lambda is Amazon’s implementation of Serverless Computing. It offers Function-as-a-Service. Amazon Lambda is an AWS service for serverless and event-driven computing. It enables automated code execution without concern for servers and clusters. Simply, code may be uploaded and executed without concern about infrastructure provisioning or management. Basically, you can upload code. Choose an event to trigger code execution. Networking and traffic scaling happens in the background. Consequently, this service automatically accepts “code execution requests” regardless of their size. In addition, you only pay for the computed time, therefore AWS Lambda provides good cost management.

The above diagram illustrates the basic usage of Lambda. After a picture is uploaded to the S3 Bucket, our website must scale it for several distinct formats. In this example case, retrieving an instance and executing this application may be a bottleneck for memory, capacity, and storage. Lambda executes only the code, resizes the picture, and saves it to a separate S3 bucket.

Other Distinct Services related to Serverless include –

Serverless Application Repository – Using the Serverless Application Repository, you may deploy source code without cloning, building, packaging, or publishing it to AWS. Instead, serverless architectures can utilize pre-built apps from the Serverless Application Repository. This service offers pre-built functionality that can be activated with a single click.

AWS Lightsail – Amazon Lightsail is the AWS service for developing websites and applications. This service provides instances of Virtual Private Servers, containers, databases, and storage. AWS Lambda enables a serverless computing service. With Amazon Lightsail, you can quickly and affordably construct websites using pre-configured software such as WordPress, Magento, Prestashop, and Joomla. In addition, it is the greatest testing tool, allowing you to create, test, and remove sandboxes containing your fresh ideas.

Elastic Beanstalk

This AWS service facilitates the execution and management of web applications. Elastic Beanstalk enables simple application deployment, including capacity provisioning, load balancing, auto-scaling, and health monitoring. With its auto-scaling capabilities, this solution simplifies scaling requirements to meet corporate goals. It helps to manage workload and traffic surges with little expense. AWS Elastic Beanstalk is essentially a developer-friendly solution since it manages servers, load balancers, firewalls, and networks with ease. As a result, our service enables developers to concentrate significantly more on coding.

AWS Elastic Beanstalk is an easy-to-use tool for deploying and scaling web applications and services written in Java,.NET, PHP, Node.js, Python, Ruby, Go, and Docker on well-known servers including Apache, Nginx, Passenger, and IIS.

Elastic Beanstalk automatically manages deployment, including capacity provisioning, load balancing, auto-scaling, and application health monitoring, upon code upload. In addition, you have complete control over the AWS resources that run your application and can always access the underlying resources.

Its features include –

• Wide Selection of different Application Platforms
• Variety of Application Deployment Options
• Monitoring, Logging, and Tracing
• Scaling
• Customization
• Compliance

AWS Outpost – AWS Outposts is a fully managed solution that extends AWS infrastructure, APIs, services, and tools to customer premises. AWS Outposts enables users to design and execute applications on-premises using the same programming interfaces as AWS Regions, but utilizing local compute and storage resources for lower latency and local data processing requirements.

AWS Snow – This service is tailored to distant, harsh work conditions.

AWS CloudWatch – This AWS service meticulously checks cloud resources and applications. It is a single platform that facilitates the monitoring of all AWS resources and applications; it improves visibility so that issues can be resolved swiftly. Amazon Cloudwatch primarily delivers actionable data to enhance application monitoring, system-wide performance changes, and resource use. Additionally, you may obtain a comprehensive picture of the health of AWS resources, applications, and services operating on AWS and on-premises. In addition, Amazon Cloudwatch assists in detecting abnormalities in cloud environment behavior, setting alerts, visualizing logs and data, automating actions, troubleshooting problems, and discovering insights.

AWS Auto Scaling

This AWS service accurately adjusts compute capacity to suit demand. And this is accomplished by automatically adding or deleting EC2 instances.

Dynamic scaling and predictive scaling are the two forms of scaling. Here, dynamic scaling responds to fluctuating needs, whereas predictive scaling responds based on forecasts.

You may discover unhealthy EC2 instances, terminate them, and replace them with fresh ones using Amazon EC2 Auto-scaling. This is dynamic capacity for elastic load balancers. It ensures that the website is never unavailable due to load. Auto-scaling is able to receive CloudWatch data. There are multiple policies for creating instances based on infrastructure traffic and usage.

Amazon VPC

VPC, or Virtual Private Cloud, is a service that enables the use of portions of the AWS Cloud, enabling the creation of cloud-based networks and the operation of cloud-based servers.

It is a cloud resource that is segregated. It manages the virtual networking environment, including allocation of resources, connection, and security. And it allows you to construct and administer VPC networks that are interoperable with AWS cloud resources and on-premises resources. Here, security is enhanced by applying restrictions to incoming and outgoing connections. In addition, it analyzes the VPC flow logs sent to Amazon S3 and Amazon Cloudwatch in order to obtain visibility into network dependencies and traffic patterns. Amazon VPC also identifies pattern deviations, prevents data leakage, and resolves network connection and setup difficulties.

Amazon Relational Database Service (RDS)

Amazon RDS is the Relational Database Service (RDS) that is handled for MySQL, PostgreSQL, Oracle, SQL Server, MariaDB, etc. It expedites the deployment, operation, and scalability of relational databases on the cloud. Additionally, it achieves great performance by automating hardware provisioning, database setup, patching, and backups. You do not need to install and maintain database software while using Amazon RDS. By using this service, you may optimize expenses and achieve high resource availability, security, and compatibility.

Amazon RDS currently offers –

• Oracle
• SQL Server
• MySQL
• MariaDB
• PostgreSQL
• Aurora (by Amazon, built upon – MySQL, PostgreSQL)

Other Distinct Services related to Databases include –

Amazon Simple DB – Amazon SimpleDB is a general-purpose, highly available NoSQL data storage.

Amazon Dynamo DB – DynamoDB is an AWS service that provides a fully managed and serverless NoSQL database. And it is a quick and adaptable database system that affords developers low-cost options for innovation. It provides sub-millisecond speed with infinite throughput and storage capacity. DynamoDB has capabilities to provide actionable insights and relevant statistics, as well as monitor application traffic patterns.

Amazon Document DB – Amazon DocumentDB is a scalable, highly durable, and fully managed database service for running mission-critical MongoDB applications.

Amazon Aurora – It is a high-performance MySQL and PostgreSQL compatible relational database. It is five times quicker than conventional MySQL databases. In addition, it enables the automation of essential activities like hardware provisioning, database setup and backups, and patching. Amazon Aurora is an autonomously scalable, distributed, fault-tolerant, and self-healing storage system. In addition, you may drastically cut expenses and improve database security, availability, and dependability.

Amazon Athena – Amazon Athena is an interactive query tool that facilitates ordinary SQL analysis of Amazon S3 data.

Amazon Neptune – It is a fast, dependable, fully managed graph database service that simplifies the development and operation of applications. In heavily interconnected datasets, its performance is improved.

Amazon ElastiCache

Amazon ElastiCache is a fully-managed, customizable AWS caching solution that operates in memory. It facilitates enhancing the efficiency of your apps and database. In addition, this service reduces database load by caching data in memory. Amazon ElastiCache provides high-speed, microsecond-latency, and high-throughput access to in-memory data. With a self-managed cache service, you may decrease company expenses and remove operational overhead. Redis and Memcached are compatible with ElastiCache.

Amazon Timestream – Amazon Timestream is a fast, scalable, and serverless time series database solution for the Internet of Things (IoT) and operational applications that makes it simple to store and analyze billions of daily events.

Amazon Quantum Ledger DB – Amazon QLDB is a fully managed ledger database that offers a transaction log that is transparent, immutable, and cryptographically verifiable.

Other distinct services with a specific function include –

Amazon RedShift – This is similar to a warehouse in that it stores numerous business data sources where they can be analyzed together and queried using SQL.

Amazon Lake Formation – AWS Lake Formation is a service that enables rapid deployment of a secure data lake. A data lake is a consolidated, controlled, and secure repository that keeps all of your data, both in its raw form and in an analysis-ready format. A data lake enables the dismantling of data silos and the integration of several forms of analytics to gather insights and inform better business choices.

Amazon Kinesis

The AWS service is responsible for analyzing both video and data sources. All sorts of streaming data are collected, processed, and analyzed using Amazon Kinesis. In this context, the data may consist of audio, video, application logs, website clickstreams, and IoT telemetry. Once the data arrives, it delivers real-time insights within seconds. Using Amazon Kinesis, you can effortlessly stream and handle a massive amount of real-time data with minimal latency. The image below is an illustration of use (Clickstream Analytics).

Amazon Map Reduce – This service effectively manages big datasets using a parallel distributed approach.

Amazon Quicksight – Amazon QuickSight is a cloud-native, serverless business intelligence solution with native machine learning integrations and usage-based pricing, enabling all users to get insights.

Amazon Managed Streaming for Apache Kafka (MSK) – Amazon MSK simplifies the ingestion and real-time processing of streaming data with fully managed Apache Kafka.

Amazon Glue – AWS Glue is a serverless data integration service that enables simple discovery, preparation, and combination of data for analytics, machine learning, and application development.

Amazon Data Exchange – For purchase and gathering data from 3rd party.

Amazon Sagemaker – Amazon Sagemaker is an AWS service that enables the development, training, and deployment of Machine Learning (ML) models at scale. It is an analytical tool that uses Machine Learning to do more effective data analysis. With its unified set of tools, you can rapidly develop high-quality ML models. Amazon Sagemaker generates not just reports but also gives the means for creating forecasts.

Amazon Rekognition – Image Analysis tool

Amazon Lex – Conversational Bot

Amazon Robomaker – Simulate and test Robots

Amazon IoT Core – Collect data from IoT Products

Amazon Ground Station – Service used for Satelite Communication

Amazon Bracket – Interact with a quantum computer

Amazon Cognito – Logging in with various authentication methods. Manages user sessions.

Amazon Simple Notification Service (SNS) – It is a communications service between Application to Person and Application to Application (A2P) (A2Person). A2P facilitates communications between distributed systems, microservices, and event-driven serverless applications in this context. In addition, A2P enables programs to deliver messages to many recipients through email, SMS, etc. For example, you may send up to 10 messages for each API call. With efficient filtering mechanisms, subscribers will only get communications of interest to them. Additionally, Amazon SNS collaborates with Amazon SQS to send messages with precision and consistency.

Amazon Simple Email Service (SES) – Amazon Simple Email Service (SES) is an affordable, versatile, and scalable email service that allows developers to send emails from any application.

Amazon Simple Queue Service (SQS) – Amazon SQS is a fully managed message queuing service that allows you to decouple and grow microservices, distributed systems, and serverless applications. SQS reduces the complexity and expense associated with maintaining and running message-oriented middleware, allowing developers to concentrate on work that differentiates themselves.

AWS Amplify – AWS Amplify is a suite of purpose-built tools and capabilities that enables frontend web and mobile developers to create full-stack apps on AWS fast and efficiently, with the flexibility to harness the breadth of AWS services as your use cases change.

AWS Development & DevOps Related Services

AWS Cloud Formation – This is Infrastructure as a Code (IaaC). Using templates, this AWS service builds and maintains resources. It is a single platform capable of managing all AWS accounts worldwide. It automates resource management with AWS service integration and provides governance controls and turnkey application delivery. AWS Cloud Formation may also automate, test, and deploy infrastructure using continuous integration and delivery. Using this service, you may execute apps directly from AWS EC2, including complicated multi-region applications. It is a method for provisioning tools to construct templates based on YAML or JSON architecture. It allows access to hundreds of distinct services with a single click.

The following graphic illustrates how a DevOps Scenario with AWS Services happens.

The code commits may be accomplished in two ways:

In the first approach, where the emphasis is on complete automation, developers commit their code using a YAML or JSON IaaC template. The template operates in the cloud formation and rapidly installs the whole service.

In a second, more normal DevOps method, the code is committed and subsequently created using the AWS CodeBuild Service. After the code has been compiled, it is tested and deployed using the CodeDeploy AWS Service, which subsequently delivers the code across all instances. This is a fundamental CI/CD Pipeline. There are specific AWS Services designed to meet these requirements. Amazon CodePipeline employs CodeCommit, CodeBuild, and CodeDeploy to construct the whole pipeline. In addition, there is CodeStar, which integrates Project Management, Issue Tracking, and Continuous Delivery.

AWS Security Services

Identity & Access Management (IAM) – This is an authentication, authorization, and accounting mechanism. This allows us to declare who or what may access AWS services and resources, centrally manage fine-grained permissions, and evaluate access to optimize AWS restrictions.

Key Management Service (KMS) – AWS Key Management Service (AWS KMS) simplifies the creation and management of cryptographic keys, as well as the control of their usage across a broad variety of AWS services and applications.

Web Application Firewall (WAF) – AWS WAF is a web application firewall that assists in protecting web applications and APIs from typical web exploits and bots that may jeopardize availability, and security, or use excessive resources.

AWS Inspector – Amazon Inspector is an automated vulnerability management solution that checks AWS workloads continuously for software vulnerabilities and accidental network exposure. It is a machine-resident agent that checks for known vulnerabilities. It gives ongoing reporting on its results.

Solution Design (Conventional vs. AWS)

Consider a web application that is essentially social media. Let’s begin by implementing this using conventional/on-premise architecture methodologies.

The user may use a browser to navigate. When he inputs the URL, the DNS is first resolved. Given the server’s heavy load, the load balancer is utilized to distribute it. Numerous web servers and application servers connect with the database. Let’s imagine there are two types of databases in use: relational and non-relational (NoSQL). To reduce disk storage given that material is stored independently. Now that users may upload media, a content filter that screens and saves material in external storage should be in place. The user may use a desktop browser as well as a mobile/tablet device. Therefore, media should be accessed through CDNs, which will expedite content delivery. Given that mobile apps may support many video formats, the solution should include a video converter. Click Stream Data, on the other hand, is fed and kept in external storage before being delivered to the data warehouse for further analysis.

Now, let us implement this using AWS Services –

We will replace DNS with Amazon’s own DNS, Route 53. The traffic will then pass via ELB and be distributed to EC2 instances using EBS. The EC2 Instance is configured with Auto-Scaling, so it will increase on-demand. Relational DB Service RDS and NoSQL DB Service Dynamo DB are linked to the Instances. The user-uploaded Media files will be screened using Rekognition and saved in the S3 Bucket. The saved Data will then be delivered to Data Warehousing Redshift, where it can be queried using Athena. Quicksight may be used to get broad insights. This whole architecture will be contained inside a VPC, which will be accessible through Edge Locations. The Solution will also include SES, SNS, and SQS Services for various mail, notification delivery, and message querying needs. Cloudwatch is capable of monitoring the whole system.

Consequently, we have observed and comprehended how a solution might be migrated to a cloud-native design using AWS Services.

Reference: AWS Documentation and Various Learning Materials.

In this chapter, we have covered the various AWS Services, important AWS service details, and both traditional and AWS-style solution architecture. In Chapter 3: AWS Enumeration & Insecure S3 Buckets, we will investigate AWS Misconfigurations, How to Enumerate AWS, AWS-Specific Tools, and Insecure S3 Buckets.

Please reach out to me if you have any suggestions.