Commonly Confused Services on DEA-C01

Deciding signal

Kinesis Data Streams is AWS-native real-time streaming with configurable retention and multiple consumers. Your application reads records and controls processing. Kinesis Data Firehose automatically delivers data to a destination (S3, Redshift, OpenSearch) without you building a consumer — it optionally transforms via Lambda. Amazon MSK (Managed Streaming for Apache Kafka) is a fully managed Apache Kafka service. It is the right answer when the scenario specifies Kafka compatibility, existing Kafka producer code, or the need for Kafka-specific features (topics, consumer groups, Kafka Streams). When an existing Kafka ecosystem is in scope, MSK. When the scenario requires automatic delivery to S3 or Redshift without consumer code, Firehose. When it requires custom consumer logic and replay in AWS-native streaming, Data Streams.

Deciding signal

Athena is a serverless query engine — it runs SQL directly on S3 data without loading it into a database. No infrastructure to manage, and you pay per query. It is the right answer when the scenario involves interactive or scheduled SQL queries on S3 without data transformation. Glue is serverless ETL: you write PySpark or Python scripts, and Glue manages the compute. It catalogs data via the Glue Data Catalog and is optimal for transformation jobs that load data into S3 or a data warehouse. EMR provisions a managed cluster (EC2 or EKS) running Hadoop, Spark, HBase, or Presto. It is the right answer when the workload requires full cluster control, custom frameworks not supported in Glue, or cost optimization via Reserved Instances on large persistent clusters.

Deciding signal

DynamoDB is a key-value and document NoSQL store optimized for high-throughput transactional access — single-digit millisecond reads and writes at any scale. It is not a query engine for complex analytics. Athena runs SQL on S3 data without loading it into a database — suited for ad-hoc exploration, infrequent queries, and log analysis where S3 is already the data store. Redshift is a columnar data warehouse for complex analytical queries over large structured datasets. It loads data into columnar storage, supports joins across billions of rows, and integrates with BI tools. When the scenario describes transactional key-based access with sub-millisecond latency, DynamoDB. When it describes complex SQL analytics on structured data loaded into a warehouse, Redshift. When it describes SQL queries on existing S3 data without loading, Athena.

Deciding signal

The Glue Data Catalog is a central metadata store — it holds database and table definitions, schemas, and partitions discoverable by Athena, EMR, and Glue ETL jobs. It does not enforce access control beyond IAM. Lake Formation builds a governance layer on top of the Glue Data Catalog: it adds table-level, column-level, and row-level access control, data location registration, and an audit trail. Lake Formation permissions are evaluated in addition to IAM and are required when the scenario involves restricting which users can see specific columns or rows in the data lake. When the scenario involves cataloging and discovering metadata, Glue Data Catalog. When it involves fine-grained access control over who sees which data lake tables or columns, Lake Formation.

Deciding signal

Glue ETL Scripts are Python or PySpark jobs you write directly — full control, maximum flexibility, required when the transformation logic is complex or custom. Glue Studio is a visual drag-and-drop interface for building ETL jobs that generates PySpark code — it reduces scripting for straightforward ETL patterns while still running on Glue infrastructure. DataBrew is a visual data preparation tool for data analysts and scientists — it provides 250+ built-in transforms for cleaning, normalizing, and enriching data without writing any code, oriented toward data exploration rather than production ETL pipeline construction.

Deciding signal

DMS migrates relational databases — it performs a full load and then continuously captures changes (CDC) from the source database to the target with minimal downtime. It supports homogeneous (MySQL to MySQL) and heterogeneous (Oracle to Aurora) migrations. Glue is an ETL service for transforming and loading data between storage systems — it is not designed for live database replication with CDC. DataSync transfers large volumes of data between on-premises storage (NFS, SMB, HDFS) and AWS storage (S3, EFS, FSx) or between AWS storage services — it is optimized for bulk file and object transfers, not database rows. When the scenario involves migrating a live database with change capture, DMS. When it involves transforming and loading data in a batch ETL pattern, Glue. When it involves bulk transfer of files or objects between storage systems, DataSync.

Deciding signal

Athena is a standalone serverless query engine — it queries S3 directly with no Redshift dependency. It is the right answer when the scenario involves ad-hoc SQL queries on S3 data without any existing Redshift warehouse. Redshift Spectrum extends a Redshift cluster to query external S3 data alongside Redshift tables in the same SQL query. It requires an existing Redshift cluster. The key use case is joining warehouse data stored in Redshift with historical or raw data stored in S3 in a single query. When there is no Redshift and the goal is SQL on S3, Athena. When Redshift already exists and the goal is querying S3 data alongside warehouse data, Spectrum.

Deciding signal

Athena is a query engine — it executes SQL and returns results as data. It does not render charts or dashboards. QuickSight is a BI service for creating interactive visualizations, dashboards, and reports that can be shared with business users. It connects to Athena (and Redshift, S3, RDS, and others) as a data source. The relationship is not OR — they often work together: Athena queries the data; QuickSight visualizes it. When the scenario describes building dashboards or charts for business stakeholders, QuickSight. When it describes running SQL transformations or analysis on data, Athena.

Deciding signal

SQS is a durable pull-based queue — messages are retained until consumed, enabling decoupled, worker-paced processing with guaranteed delivery. It suits pipeline stages where the consumer controls processing speed. SNS pushes messages simultaneously to all subscribers (Lambda, SQS, HTTP) — it is for fanout to multiple systems when every subscriber must receive every event. EventBridge routes events based on content-matching rules — it filters events by source, detail type, or specific fields and delivers them to different targets based on those rules. In data pipelines, EventBridge is the right answer when different types of data ingestion events should trigger different processing stages. SQS is the right answer when a processing stage must buffer and pace consumption independently.

Deciding signal

Amazon Managed Service for Apache Flink (formerly Kinesis Data Analytics) runs Apache Flink applications for real-time stream processing — it supports complex event processing, windowed aggregations, and stateful streaming analytics in Flink or SQL. It is optimized for continuous real-time analytics with low latency. AWS Glue Streaming ETL runs Spark Structured Streaming jobs for continuous ETL — reading from Kinesis or Kafka and writing transformed data to S3 or other destinations. Glue Streaming is better suited for continuous ETL workloads where Spark ecosystem compatibility matters; Flink is better suited for real-time analytics, stateful computations, and complex event-driven patterns.