Amazon SQS Core Knowledge

Full Reference: See advanced.md for Java/Python/Go producers, Spring Cloud AWS consumers, Lambda integration, IAM policies, and CloudWatch monitoring.

Deep Knowledge: Use mcp__documentation__fetch_docs with technology: sqs for comprehensive documentation.

Quick Start (LocalStack)

# docker-compose.yml
services:
  localstack:
    image: localstack/localstack
    ports:
      - "4566:4566"
    environment:
      - SERVICES=sqs
      - DEFAULT_REGION=us-east-1
    volumes:
      - localstack_data:/var/lib/localstack

volumes:
  localstack_data:

# Create queue
aws --endpoint-url=http://localhost:4566 sqs create-queue \
  --queue-name orders-queue

# Create FIFO queue
aws --endpoint-url=http://localhost:4566 sqs create-queue \
  --queue-name orders-queue.fifo \
  --attributes FifoQueue=true,ContentBasedDeduplication=true

Core Concepts

| Concept | Description | |---------|-------------| | Standard Queue | At-least-once, best-effort ordering | | FIFO Queue | Exactly-once, strict ordering | | Visibility Timeout | Message lock period | | Dead Letter Queue | Failed message destination | | Long Polling | Efficient message retrieval | | Message Groups | FIFO ordering within group |

Queue Types Comparison

| Feature | Standard | FIFO | |---------|----------|------| | Throughput | Unlimited | 300 msg/s (3000 with batching) | | Ordering | Best-effort | Strict (per group) | | Delivery | At-least-once | Exactly-once | | Deduplication | None | 5-minute window |

Producer Pattern (Node.js)

import { SQSClient, SendMessageCommand, SendMessageBatchCommand } from '@aws-sdk/client-sqs';

const client = new SQSClient({
  region: 'us-east-1',
  // For LocalStack: endpoint: 'http://localhost:4566',
});

const queueUrl = 'https://sqs.us-east-1.amazonaws.com/123456789/orders-queue';

// Send single message
await client.send(new SendMessageCommand({
  QueueUrl: queueUrl,
  MessageBody: JSON.stringify(order),
  MessageAttributes: {
    'OrderType': {
      DataType: 'String',
      StringValue: order.type,
    },
    'CorrelationId': {
      DataType: 'String',
      StringValue: correlationId,
    },
  },
  DelaySeconds: 0,
}));

// Send batch (up to 10)
await client.send(new SendMessageBatchCommand({
  QueueUrl: queueUrl,
  Entries: orders.map((order, index) => ({
    Id: `msg-${index}`,
    MessageBody: JSON.stringify(order),
    MessageAttributes: {
      'OrderType': { DataType: 'String', StringValue: order.type },
    },
  })),
}));

// FIFO queue
const fifoQueueUrl = 'https://sqs.us-east-1.amazonaws.com/123456789/orders-queue.fifo';

await client.send(new SendMessageCommand({
  QueueUrl: fifoQueueUrl,
  MessageBody: JSON.stringify(order),
  MessageGroupId: order.customerId,      // Required for FIFO
  MessageDeduplicationId: order.orderId, // Or use ContentBasedDeduplication
}));

Consumer Pattern (Node.js)

import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';

const client = new SQSClient({ region: 'us-east-1' });

async function pollMessages() {
  while (true) {
    const response = await client.send(new ReceiveMessageCommand({
      QueueUrl: queueUrl,
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20,           // Long polling
      VisibilityTimeout: 30,
      MessageAttributeNames: ['All'],
      AttributeNames: ['All'],
    }));

    if (!response.Messages) continue;

    for (const message of response.Messages) {
      try {
        const order = JSON.parse(message.Body!);
        await processOrder(order);

        // Delete on success
        await client.send(new DeleteMessageCommand({
          QueueUrl: queueUrl,
          ReceiptHandle: message.ReceiptHandle!,
        }));
      } catch (error) {
        // Message will return to queue after visibility timeout
        console.error('Processing failed:', error);
      }
    }
  }
}

Dead Letter Queue

// Create DLQ
await client.send(new CreateQueueCommand({
  QueueName: 'orders-dlq',
}));

// Main queue with DLQ
await client.send(new CreateQueueCommand({
  QueueName: 'orders-queue',
  Attributes: {
    RedrivePolicy: JSON.stringify({
      deadLetterTargetArn: 'arn:aws:sqs:us-east-1:123456789:orders-dlq',
      maxReceiveCount: '3',
    }),
    VisibilityTimeout: '30',
    MessageRetentionPeriod: '1209600', // 14 days
  },
}));

# Terraform
resource "aws_sqs_queue" "orders_dlq" {
  name = "orders-dlq"
  message_retention_seconds = 1209600
}

resource "aws_sqs_queue" "orders" {
  name = "orders-queue"
  visibility_timeout_seconds = 30
  message_retention_seconds = 1209600

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.orders_dlq.arn
    maxReceiveCount     = 3
  })
}

Lambda Integration

// Lambda handler
export const handler = async (event: SQSEvent): Promise<SQSBatchResponse> => {
  const batchItemFailures: SQSBatchItemFailure[] = [];

  for (const record of event.Records) {
    try {
      const order = JSON.parse(record.body);
      await processOrder(order);
    } catch (error) {
      // Report partial batch failure
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }

  return { batchItemFailures };
};

# serverless.yml
functions:
  orderProcessor:
    handler: handler.handler
    events:
      - sqs:
          arn: !GetAtt OrdersQueue.Arn
          batchSize: 10
          functionResponseType: ReportBatchItemFailures

When NOT to Use This Skill

Use alternative messaging solutions when:

• Event streaming with replay - Use Kinesis or Kafka
• Cross-cloud or hybrid cloud - Use Kafka, RabbitMQ, or Pulsar
• Complex routing patterns - RabbitMQ provides better routing
• Message ordering across all messages - FIFO queues have throughput limits
• Real-time low latency (<10ms) - Use Redis or NATS
• On-premise deployment - Use RabbitMQ or ActiveMQ
• Message size >256KB - Use S3 with SQS Extended Client

Anti-Patterns

| Anti-Pattern | Why It's Bad | Solution | |--------------|--------------|----------| | Short polling (WaitTime=0) | Wasteful API calls, higher cost | Use long polling (WaitTimeSeconds=20) | | No DLQ configured | Failed messages lost | Always configure dead letter queue | | Visibility timeout too short | Duplicate processing | Set timeout > max processing time | | Visibility timeout too long | Slow failure recovery | Balance with processing time | | Processing before delete | Message reprocessed on crash | Delete only after successful processing | | FIFO for high throughput | Limited to 300 msg/s | Use Standard queue if ordering not critical | | No batching | Higher latency and cost | Batch up to 10 messages per request | | Polling in Lambda | Wasted invocations | Use Lambda event source mapping | | No IAM policies | Security risk | Apply least privilege IAM policies |

Quick Troubleshooting

| Issue | Likely Cause | Fix | |-------|--------------|-----| | Messages not appearing | Wrong queue URL or permissions | Verify URL and IAM permissions | | Duplicate messages | Standard queue behavior or visibility timeout | Implement idempotent processing | | Messages delayed | Delay seconds set or queue backlog | Check DelaySeconds, scale consumers | | Messages in DLQ | Max receives exceeded | Check processing logic, increase retries | | Visibility timeout errors | Message processing too slow | Extend visibility timeout | | Throughput limited | FIFO queue limit | Use Standard queue or batch messages | | High costs | Short polling or frequent sends | Use long polling, batch operations | | Access denied | Missing IAM permissions | Add sqs:SendMessage/ReceiveMessage | | Message size limit | Payload >256KB | Use SQS Extended Client with S3 |

Production Checklist

• [ ] IAM policies with least privilege
• [ ] Server-side encryption enabled
• [ ] Dead letter queue configured
• [ ] Visibility timeout set appropriately
• [ ] Long polling enabled (20s)
• [ ] Message retention configured
• [ ] CloudWatch alarms set up
• [ ] DLQ monitoring alerts
• [ ] VPC endpoints (if needed)
• [ ] Access logging enabled

Key Metrics to Monitor

| Metric | Alert Threshold | |--------|-----------------| | ApproximateNumberOfMessagesVisible | > 10000 | | ApproximateAgeOfOldestMessage | > 3600s | | NumberOfMessagesReceived | Anomaly | | NumberOfMessagesSent | Anomaly | | ApproximateNumberOfMessagesNotVisible | > expected |

Reference Documentation

Deep Knowledge: Use mcp__documentation__fetch_docs with technology: sqs for comprehensive documentation.

Available topics: basics, producers, consumers, dlq, production