Course 2 · Coming Soon
Advanced AI Engineering Concepts
Fine-Tuning, Managed AI Services, Applied NLP & Agentic System Design
An advanced, hands-on program focused on building, customizing, evaluating, and deploying production-oriented AI systems. Students learn parameter-efficient LLM fine-tuning, AWS managed AI services, applied NLP pipelines, and reliable agentic architectures.
- Duration
- 8 Weeks
- Level
- Intermediate to Advanced
- Format
- Hands-on Labs & Capstone
The 8-Week Curriculum
Week 1 — LLM Fine-Tuning Fundamentals
Topics
- Prompt Engineering vs. RAG vs. Fine-Tuning
- Pre-training, Supervised Fine-Tuning (SFT) & Instruction Tuning
- Dataset Preparation, Tokenization & Data Formatting
- Training, Validation & Test Dataset Splits
- Overfitting, Catastrophic Forgetting & Data Leakage
Hands-on Lab: Prepare and validate an instruction dataset for a domain-specific AI task.
Deliverable: A cleaned training dataset and baseline model evaluation.
Week 2 — LoRA, QLoRA & Parameter-Efficient Fine-Tuning
Topics
- Transformer Architecture & Attention Mechanisms
- Full Fine-Tuning vs. Parameter-Efficient Fine-Tuning (PEFT)
- LoRA: Rank, Alpha, Target Modules & Adapter Weights
- QLoRA: Quantization, 4-bit Loading & Memory Optimization
- Hugging Face Transformers, PEFT, TRL & bitsandbytes
- Training Configuration, Checkpoints & Adapter Management
Hands-on Lab: Fine-tune a small open-weight language model using LoRA or QLoRA in a hosted GPU environment.
Deliverable: A trained adapter, inference notebook and training logs.
Week 3 — Fine-Tuning Evaluation & Model Optimization
Topics
- Base Model vs. Fine-Tuned Model Comparison
- Evaluation Metrics: Precision, Recall, F1 & Exact Match
- LLM-as-a-Judge and Human Evaluation
- Hyperparameter Tuning & Experiment Tracking
- Quantization & Inference Optimization Fundamentals
- Model Cards, Reproducibility & Dataset Documentation
Hands-on Lab: Evaluate the baseline and fine-tuned models on the same test dataset.
Deliverable: A model evaluation report comparing quality, latency and cost.
Week 4 — Amazon Bedrock & Managed Foundation Models
Topics
- Amazon Bedrock Architecture & Foundation Model Access
- Model Invocation, Streaming & Structured Outputs
- Bedrock Knowledge Bases & Managed RAG
- Guardrails, Prompt Management & Model Evaluation
- IAM Permissions, API Security & Secrets Management
- Managed APIs vs. Self-Hosted Models
Hands-on Lab: Build a domain-specific question-answering API using Amazon Bedrock with grounded responses and input validation.
Deliverable: A working API with evaluation results and a cost estimate.
Week 5 — Amazon SageMaker AI & Model Lifecycle
Topics
- Amazon Bedrock vs. SageMaker AI: Use Cases & Trade-Offs
- Managed Training and Processing Jobs
- Amazon S3 for Training Data and Model Artifacts
- SageMaker JumpStart & Supported Model Customization
- Model Deployment, Endpoints & Inference
- Autoscaling, Compute Selection & Cost Optimization
- Model Lifecycle Management
Hands-on Lab: Execute a guided training or model-customization workflow and explore model deployment.
Deliverable: A reproducible training/deployment workflow and architecture diagram.
Week 6 — Applied NLP Engineering
Topics
- Text Preprocessing, Tokenization & Normalization
- Named Entity Recognition (NER) & Entity Extraction
- Text Classification, Sentiment Analysis & Intent Detection
- Semantic Similarity, Embeddings & Reranking
- Information Extraction from Unstructured Text
- Traditional NLP vs. Transformer-Based Approaches
- NLP Evaluation Metrics and Error Analysis
Hands-on Lab: Build an NLP pipeline to classify support tickets, extract key entities and identify urgent requests.
Deliverable: A reusable NLP service with evaluation metrics and a FastAPI endpoint.
Week 7 — Agentic System Design & Amazon Bedrock AgentCore
Topics
- Agent Architecture, Tool Calling, State & Orchestration
- LangGraph Workflows, Conditional Routing & Checkpointing
- Single-Agent vs. Multi-Agent Architectures
- Model Context Protocol (MCP) & Tool Integration
- Short-Term and Long-Term Memory
- Human-in-the-Loop Approval & Controlled Tool Execution
- Retries, Timeouts, Idempotency & Failure Recovery
- Amazon Bedrock AgentCore: Runtime, Gateway, Memory, Identity & Observability
- Security Boundaries, Tracing & Operational Monitoring
Hands-on Lab: Build an agent workflow that retrieves information, invokes approved tools, requests human approval for sensitive actions and records execution traces.
Deliverable: An agent workflow with reliability tests and an architecture decision record.
Week 8 — Production AI Engineering & Capstone
Topics
- AI System Design: Functional and Non-Functional Requirements
- Evaluation Datasets, Regression Testing & Quality Gates
- Observability, Latency, Throughput & Token Cost Management
- API Security, Authentication & Least-Privilege Access
- Rate Limiting, Fallbacks & Graceful Degradation
- Deployment Architecture, CI/CD & Operational Monitoring
Hands-on Lab: Integrate the selected components into a deployable AI application and execute a documented test suite.
Deliverable: A working capstone, source repository, architecture diagram, evaluation report and project demonstration.
Capstone Project
AI-Powered Support Resolution & Intelligence Platform
Build an AI-powered support platform that classifies customer tickets, extracts key information, retrieves relevant knowledge, generates response drafts and routes sensitive actions for human approval.
Core Features
- NLP-based ticket classification, entity extraction and urgency detection
- LoRA/QLoRA fine-tuned model for a focused classification or extraction task
- Amazon Bedrock-powered response generation
- LangGraph-based agent orchestration and knowledge retrieval
- Human approval and controlled tool execution
- FastAPI backend with evaluation, logging and error handling
- Quality, latency and cost comparison against a baseline
Optional Advanced Extensions
- Deploy the agent using Amazon Bedrock AgentCore
- Add automated evaluation and regression gates
- Implement production monitoring and cost dashboards
| Area | Technologies |
| Programming | Python, SQL |
| Fine-Tuning | PyTorch, Hugging Face Transformers, PEFT, LoRA, QLoRA, TRL |
| Managed AI Services | Amazon Bedrock, Amazon SageMaker AI, Bedrock AgentCore, Amazon S3, IAM, CloudWatch |
| Agent Engineering | LangGraph, MCP, Tool Calling |
| Backend | FastAPI, Pydantic |
| Testing & Evaluation | pytest, Precision, Recall, F1, LLM-as-a-Judge |
Assessment Structure
| Component | Weight |
| Weekly Hands-on Labs | 25% |
| Fine-Tuning Experiment & Evaluation Report | 20% |
| AWS Managed-Service Lab | 15% |
| Agent Architecture & Reliability Tests | 15% |
| Final Capstone & Demonstration | 25% |
Learning Outcomes
By completing this course, students will be able to:
- Fine-tune open-weight LLMs using LoRA, QLoRA and PEFT.
- Evaluate the trade-offs between prompting, RAG, fine-tuning and managed foundation models.
- Build applications using Amazon Bedrock and understand SageMaker AI model workflows.
- Develop applied NLP pipelines for classification, extraction and semantic similarity.
- Design agentic systems with tool integration, state management, human approval and failure recovery.
- Apply evaluation, observability, security and cost optimization practices to production-oriented AI systems.
Prerequisites: Python programming, basic LLM/API concepts, introductory RAG knowledge and familiarity with REST APIs. Capstone Project
