An agentic AI project is a system where an LLM plans, calls tools and loops until a goal is met. Good 2026 projects use a tool layer such as MCP, an orchestration framework, a 30-case evaluation set, step traces and cost limits. Evaluation and tool design, not the framework, make a project credible.
Agent demos are easy to build and hard to trust.
The 7 projects
MCP server for an analytics database
The question: Can any MCP client safely answer questions about a database through a small set of well-described tools?
- Start with
- Olist Brazilian E-Commerce: about 100,000 orders across nine linked tables.
- Steps
- Load the CSVs into DuckDB or Postgres.
- Expose tools:
list_tables,describe_table,run_readonly_query,get_metric(revenue, AOV). - Enforce read-only access, a row limit and a query timeout.
- Test with an MCP client using 20 prompts and log which tool each prompt used.
- Measure
- Tool-selection accuracy on 20 prompts and the rate of blocked unsafe queries.
- Ship
- Repo, tool descriptions, a transcript and the test table.
- Why it counts
- MCP is the common way agents reach tools and data, so a clean server is directly relevant.
Built a read-only MCP server exposing 4 database tools with query limits; achieved [x]% correct tool selection on 20 test prompts.
Text-to-SQL analytics copilot
The question: Can a business user ask a question in English and get a correct, checkable answer?
- Start with
- Same Olist tables. Write 40 questions with reference SQL and answers.
- Steps
- Retrieve only the relevant schema for each question.
- Generate SQL, run it read-only, and check that the result shape is sensible.
- Retry once on an error, then say 'I am not sure' rather than guess.
- Return a table, a chart and the SQL shown to the user.
- Measure
- Execution accuracy (result matches gold), refusal rate on unanswerable questions and cost per question.
- Ship
- Live demo, the 40-question benchmark and a failure analysis.
- Why it counts
- Directly mirrors analytics-engineering use cases companies are building.
Built a text-to-SQL copilot over a 9-table store dataset; [x]% execution accuracy on a 40-question benchmark at $[y] per question.
Agentic RAG over a document set
The question: Can the agent decide when to retrieve, rewrite a weak query and still cite its sources?
- Start with
- A public document set (policy FAQs, regulations, product manuals). Create 30 questions, including 5 that the documents cannot answer.
- Steps
- Build baseline RAG first and score it.
- Add a router (answer directly, retrieve, or ask a clarifying question).
- Add query rewriting and a reranker; add a final 'is this supported by the sources?' check.
- Compare baseline and agentic versions on the same 30 questions.
- Measure
- Ragas faithfulness, answer relevancy, context precision and context recall, before and after.
- Ship
- Side-by-side results table and annotated failures.
- Why it counts
- Shows you add complexity only when measurements justify it.
Extended baseline RAG with routing, query rewriting and reranking; raised faithfulness from [x] to [y] on a 30-question set while holding cost per answer under $[z].
Support-ticket triage agent with human escalation
The question: Which tickets can be answered automatically, and which must go to a person?
- Start with
- CFPB complaints or 200 synthetic tickets you generate and label by hand.
- Steps
- Classify intent and urgency with structured output.
- Retrieve policy snippets and draft a reply with citations.
- Escalate below a confidence threshold or on sensitive topics.
- Log every decision and the reason.
- Measure
- Routing accuracy, escalation precision and recall, and human ratings on 50 drafted replies.
- Ship
- Demo, confusion matrix and escalation policy.
- Why it counts
- Human-in-the-loop design is what real deployments need.
Built a triage agent that routes and drafts replies with citations; escalated [x]% of tickets with [y]% precision against hand labels.
Research and competitor-monitor multi-agent
The question: Can a planner, a researcher and a writer produce a sourced brief better than a single prompt?
- Start with
- Live web sources on a topic you choose (for example, five competitors' pricing pages). Save the pages so runs are repeatable.
- Steps
- Planner splits the task; researcher fetches and extracts facts with source URLs.
- Writer drafts only from extracted facts.
- A checker verifies each claim against its source.
- Compare with a single-agent baseline.
- Measure
- Percentage of claims supported by the saved sources, and cost per brief for each design.
- Ship
- Two briefs side by side and a claim-check table.
- Why it counts
- Proves whether multi-agent design adds value for the cost.
Compared multi-agent vs single-agent research on [n] briefs; claim support [x]% vs [y]% at [z]x the cost.
Narrow coding agent with a pass-rate benchmark
The question: Can an agent fix one category of bug reliably, for example failing unit tests in a small repo?
- Start with
- A small open-source repository plus 20 bugs you seed (off-by-one, wrong operator, missing null check).
- Steps
- Give the agent tools: read file, edit file, run tests (in a sandbox).
- Stop after a fixed number of steps or a token budget.
- Record every action and whether tests pass.
- Report which bug types fail and why.
- Measure
- Pass rate on the 20 seeded bugs, steps per fix and cost per fix.
- Ship
- Benchmark repo, results table and traces.
- Why it counts
- Scoped agents with benchmarks are far more convincing than 'build a developer'.
Built a sandboxed coding agent that fixed [x] of 20 seeded bugs; analysed failure categories and step budgets.
Metric anomaly explainer for product teams
The question: When a metric drops, can an agent segment the data, test hypotheses and write a cautious explanation?
- Start with
- Retailrocket e-commerce events with anomalies you plant on purpose (for example a broken checkout step for one device).
- Steps
- Detect the anomaly with a simple control chart.
- Agent segments by device, channel and step; runs tests for each slice.
- Ranks candidate causes with the evidence for each.
- Writes a short, hedged report.
- Measure
- How often the planted cause appears in the top 3 explanations, and the false-cause rate.
- Ship
- Report examples and a planted-anomaly benchmark.
- Why it counts
- Combines analytics, product sense and agents in one project.
Built an anomaly-explainer agent that ranked the planted root cause in the top 3 for [x] of [y] scenarios.
12 more project ideas
Shorter briefs for when you want a different angle. Apply the same rules: a real source, a baseline, a number and a README.
Meeting-notes to action-items agent
BeginnerLLM API, structured output, a task tool
DataTranscripts you generate or record with consent
Stand-outPrecision of extracted actions against hand labels
Email triage assistant with human approval
IntermediateGmail API or a mock inbox, LangGraph
DataA synthetic inbox of 200 emails
Stand-outRouting accuracy and a log of every escalation
Data-quality monitor agent
IntermediateSQL tools, scheduled runs, alerts
DataAny warehouse-style dataset
Stand-outPlanted data bugs and how many it caught
Interview practice agent
BeginnerLLM API, speech-to-text optional
DataQuestion banks you write
Stand-outRubric-based scoring calibrated against two humans
Agentic web-form filler in a sandbox
AdvancedPlaywright, an LLM, a local test site
DataA site you build for testing
Stand-outSuccess rate across 20 form variants and failure analysis
Policy document Q&A with citations and refusal
IntermediateRAG, reranker, LangGraph
DataPublic government FAQs
Stand-outCorrect refusals on unanswerable questions
Slide-deck generator from a data table
IntermediateLLM API, python-pptx, charts
DataAny analytics output
Stand-outHuman rating of 10 decks for accuracy, not looks
Travel or event planner with constraints
BeginnerTool calling, a constraint checker
DataPublic event and venue data
Stand-outConstraint-violation rate on 30 scenarios
Code-review agent for pull requests
AdvancedGitHub API, LLM, diff parsing
DataYour own repositories
Stand-outAgreement with human review comments on 20 PRs
Customer-feedback clustering agent
Intermediateembeddings, clustering, LLM labels
DataApp-store reviews
Stand-outCluster labels validated by a human on a sample
Agent evaluation harness
IntermediatePython, pytest, tracing
DataAny agent above
Stand-outA reusable 30-case eval runner with cost tracking
Multi-tool research agent with budget caps
AdvancedMCP tools, LangGraph, Langfuse
DataSaved web pages
Stand-outCost per task and p95 latency reported honestly
Build the evaluation set first
Thirty cases is enough to start. Aim for 20 normal tasks, 5 edge cases and 5 adversarial or unanswerable ones. Write the expected outcome for each before you run the agent.
- Pick a metric that matches the job: execution accuracy, claim support, pass rate or routing accuracy.
- Score with code where you can; use an LLM judge only for fuzzy criteria, and check it against 20 to 50 human ratings.
- Version the set and rerun it after every change.
- Report pass rate, cost per task and p95 latency in one table.
Safety checklist mapped to the OWASP Top 10 for LLM Applications (2025)
Agents act on instructions found in data, so security is part of the project. The OWASP GenAI project lists the risks below among its ten.
| OWASP risk | What to implement and show |
|---|---|
| LLM01 Prompt Injection | Treat retrieved text and tool output as untrusted; test with injected instructions in documents. |
| LLM02 Sensitive Information Disclosure | Strip secrets from prompts and logs; test that the agent will not reveal them. |
| LLM05 Improper Output Handling | Validate structured output before it touches SQL, code or APIs. |
| LLM06 Excessive Agency | Give each tool the minimum permission; use read-only credentials; ask a human before irreversible actions. |
| LLM10 Unbounded Consumption | Set step, token and cost limits per run. |
The agent stack used in industry
| Layer | Options (official docs) |
|---|---|
| Tool protocol | Model Context Protocol, FastMCP |
| Orchestration | LangGraph, OpenAI Agents SDK, Google ADK, CrewAI, smolagents |
| Retrieval | Qdrant, Chroma, FAISS, Elasticsearch BM25 |
| Evaluation | Ragas, custom pass or fail sets, LLM-as-judge calibrated to humans |
| Observability | Framework trace exporters or an OpenTelemetry-based tracer |
Pick one framework and finish a project. Switching frameworks adds nothing to a portfolio; measured results do.
Frequently asked questions
What is an agentic AI project?
A system where an LLM plans steps, calls tools such as databases, search or code execution, observes the results and repeats until it meets a goal or a budget limit.
Which agent framework should I learn first?
Pick one: LangGraph for stateful workflows or the OpenAI Agents SDK for a lighter start. The evaluation set and tool design matter more than the framework.
What is MCP and why build with it?
The Model Context Protocol is an open standard for connecting AI applications to tools and data. Building an MCP server shows you can make tools that any compatible agent can use.
How do I evaluate an AI agent?
Write 30 test tasks with expected outcomes, score runs automatically, and report pass rate, cost per task and latency. Calibrate any LLM judge against human ratings.
Do I need to fine-tune models for agentic projects?
No. Most portfolio value comes from tool design, retrieval, evaluation and safety, using hosted or open models as they are.
Learn this with mentors
AI and machine learning programmes with admissions open: live classes, mentors and project work.
