Sr. Data Engineer (GenAI & AWS) Remote/Hybrid with Travel

The key is time zone coverage and so we can expand our search across the West Coast - remote/hybrid but necessarily located in the western time zones. You must be comfortable to travel to the client site for workshops/meetings.

1. Senior Data Engineer (GenAI / AWS)
   

Overall experience: 12-20+ years Cloud Platform: AWS Primary Language: Python

Key Responsibilities

• Design and develop data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend.
  
• Build automated document-processing workflows for extracting, transforming, and processing structured and unstructured data.
  
• Implement document chunking, embeddings, vector indexing, and semantic search.
  
• Develop and maintain vector databases and knowledge bases to support Retrieval-Augmented Generation (RAG).
  
• Integrate knowledge bases with AI agents for contextual information retrieval.
  
• Build AI agents using Claude Agent SDK and deploy them on Amazon Bedrock AgentCore Runtime.
  
• Develop CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments.
  
• Optimize data pipelines for scalability, reliability, security, and performance.
  

Must-have vs. good-to-have skills

Must-have

Good-to-have

Python, SQL

TypeScript

AWS Step Functions, Lambda, S3

AWS Glue

Amazon Textract and Comprehend

Document AI / OCR

Vector databases and embeddings

OpenSearch, Pinecone, pgvector

Document chunking and indexing

Advanced retrieval and reranking

RAG and knowledge-base integration

LangChain / LlamaIndex

Amazon Bedrock

Claude Agent SDK

GitHub Actions CI/CD

Terraform / AWS CDK

AWS IAM and API integrations

AgentCore Runtime

Important: If this individual must personally build and deploy the Claude agents, move Claude Agent SDK and AgentCore Runtime into the must-have category.