Public path
Agentic SDET
6 steps
Sign in to star or follow this path.
Follow along for free. Sign in to star, follow, and tick steps off as you go.
• Learn: LLM API calls, structured outputs, prompt design, golden datasets, LLM-as-judge, core metrics (correctness, relevance, groundedness, hallucination rate). • Tools: Python, promptfoo or DeepEval. • Ship: an eval harness for a small chat/RAG app, with a golden dataset and a pass/fail report.
• Learn: tracing, retrieval quality metrics, RAG basics (embeddings, chunking, pgvector), just enough to test a RAG system. • Tools: Langfuse, RAGAS. • Ship: traced RAG app with retrieval and answer-quality scores on a dashboard
• Learn: Playwright MCP, Claude Code / Copilot agent mode for test generation, self-healing locators, human-in-the-loop approval. • Explore: Stagehand. • Ship: a test-generation agent that reads a requirement, writes a Playwright test, runs it, fixes failures, and asks for approval. Measure and record time saved and self-heal rate.
• Learn: quality gates in GitHub Actions, guardrail and red-team tests, testing tool calls and agent trajectories. • Ship: CI pipeline that runs the eval suite on every change and blocks regressions.
• Learn: SQL refresher, FastAPI wrapper for the project, Docker packaging, one cloud AI platform (Azure/Databricks or AWS Bedrock, chosen by the roles I'm targeting). • Ship: containerized, deployable version of the project.
• README with architecture diagram and measured results, short demo video and applications. Timeline note: at 5-8 hours per week, expect 4-6 months instead.