From Prompt to Product: Your First Steps in Generative AI

Table of Contents

1. The project
2. Step 1: Write a prompt worth reusing
3. Step 2: What's actually happening underneath
4. Step 3: Your first API call
5. Step 4: The naive version that works
6. Step 5: When the document doesn't fit — RAG
7. Step 6: Agents — and why to skip them for now
8. Step 7: What turns a project into a product
9. Common mistakes
10. What to do this week

Most "getting started with Generative AI" guides hand you a vocabulary list—LLMs, prompt engineering, RAG, embeddings, vector databases, agents. You finish knowing what to Google and not much else.

This one is different. We'll build one thing—a document assistant that answers questions about a PDF you upload—and each concept appears only when the project needs it. RAG, for instance, arrives the moment the document gets too big to paste.

By the end, you won't have memorized the ecosystem. You'll have a small thing that runs and a clear sense of what to break next.

The project

A user uploads a document, asks a question, and gets an answer grounded in that document.

It sounds trivial. It isn't — it quietly contains prompting, API integration, chunking, embeddings, retrieval, cost control, and failure handling. Which is why almost every real enterprise AI product is a more expensive version of it.

Step 1: Write a prompt worth reusing

A prompt is just an instruction. What beginners miss is that in a product it isn't something you type—it's a template your code fills in, over and over, for inputs you've never seen. So the bar isn't "Does this work once?" but "Does this behave predictably across a thousand documents?"

Compare these. First attempt:

Answer the question about this document.

Second attempt:

You are a careful document analyst. Answer the user's question using only the excerpts provided below. If the excerpts don't contain the answer, say "I couldn't find that in this document" — do not use outside knowledge. Quote the specific line you based your answer on.

Excerpts: {context}

Question: {question}

The second is longer, but look at what each clause buys you. "Using only the excerpts stops the model answering from general knowledge and pretending it read your file. "If the excerpts don't contain the answer, say..." gives it a legal way to fail—without that escape hatch, a model under pressure invents something. "Quote the specific line" lets your user check the work.

That's the whole discipline. This isn't clever phrasing; it's anticipating the failure and writing a clause that closes it.

Two techniques worth knowing early. Few-shot prompting — showing two or three examples of the pattern you want — is the fastest fix when the format drifts. Structured output means asking for JSON so your code can parse the result instead of scraping prose.

Step 2: What's actually happening underneath

You don't need the mathematics. You need four concepts, because each will eventually cause a bug you can't diagnose without it.

Tokens. The model sees tokens, not words—chunks of roughly four characters. This matters because you're billed per token and limited per token.

Context window. The total tokens a model considers at once: prompt, document, history, and its own answer combined. Modern models hold a few hundred thousand. That's a lot. It is not infinite, and the moment your document exceeds it is the moment you need Step 5.

Hallucination. The model generates plausible continuations, not verified facts. It has no internal signal for "I don't know" unless you give it one—exactly what that clause in Step 1 was doing.

Statelessness. Each API call starts from nothing. Every "conversational" AI product you've used re-sends the whole conversation on every turn.

And the one that trips people up most: the model knows nothing about your company, your database, or the PDF on your laptop. It only knows what you put in the prompt. That single fact is the reason RAG exists.

Step 3: Your first API call

The jump from using AI to building with it happens here, and it's smaller than you'd expect.

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from your environment

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system="You are a careful document analyst.",
    messages=[
        {"role": "user", "content": "Explain what a context window is, in two sentences."}
    ],
)

print(message.content[0].text)

Run it. That's the entire foundation.

Three details the code doesn't announce. system holds persistent instructions—role, tone, rules—kept separate from the user's turn so the two never blur. max_tokens caps the response, not the input; set it too low, and answers truncate mid-sentence. And message. Content is a list of blocks, not a string.

For Python, be comfortable with functions, dictionaries, JSON, file handling, and try/except. That's it for this project. You don't need design patterns to ship your first AI app, and anyone telling you otherwise is selling a course.

Step 4: The naive version that works

Here's the part most tutorials skip, and they shouldn't.

If your document fits in the context window — and under about fifty pages, it does—you don't need retrieval, embeddings, or a vector database. You paste the document into the prompt.

from pypdf import PdfReader

def read_pdf(path):
    return "\n".join(page.extract_text() for page in PdfReader(path).pages)

PROMPT = """Answer the user's question using only the document below.

If the document doesn't contain the answer, say "I couldn't find that in this document."

Quote the line you based your answer on.

Document:

{document}

Question: {question}"""

def ask(path, question):
    message = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": PROMPT.format(document=read_pdf(path), question=question)
        }],
    )
    return message.content[0].text

print(ask("report.pdf", "What are the main risks discussed?"))

That is a working document assistant. Thirty lines. Ship it and find out where it breaks.

Build this before anything fancier — the naive version teaches you what the fancy version is actually for. Plenty of production "RAG systems" solve a problem their authors never confirmed they had, at ten times the complexity and worse accuracy than stuffing the document in the prompt would have given them.

It breaks in two places, and you should go find them yourself. Feed it a 400-page manual and watch it error out. Then check what one call on a large document costs, and multiply by a thousand users.

Step 5: When the document doesn't fit — RAG

Now the problem is real, so the solution makes sense.

Retrieval-Augmented Generation means: don't send the whole document, send the paragraphs most likely to contain the answer. Keyword search won't find them, because a user asking about "staff turnover" needs to match a document that says "employee attrition."

Embeddings solve this. An embedding turns text into a list of numbers, positioned so that similar meaning lands in a similar place. "Employee attrition" and "staff turnover" become neighbours despite sharing no words. Compare two embeddings with cosine similarity and you get a score for how related they are.

The pipeline:

Document → split into chunks → embed each chunk → store → embed the question → find the closest chunks → send those to the model

from sentence_transformers import SentenceTransformer
import numpy as np

encoder = SentenceTransformer("all-MiniLM-L6-v2")  # runs locally, free

def chunk(text, size=1000, overlap=200):
    return [text[i:i+size] for i in range(0, len(text), size - overlap)]

chunks = chunk(read_pdf("report.pdf"))
vectors = encoder.encode(chunks)

def retrieve(question, k=4):
    q = encoder.encode([question])[0]
    scores = vectors @ q / (np.linalg.norm(vectors, axis=1) * np.linalg.norm(q))
    return [chunks[i] for i in np.argsort(scores)[-k:]]

Feed retrieve(question) into the {document} slot from Step 4 and you have RAG.

Notice what's not here: a vector database. Under roughly ten thousand chunks, a NumPy array and a dot product are fast enough, and you've removed a whole piece of infrastructure from your first project. Reach for Chroma, pgvector, or Pinecone when you need persistence or outgrow memory — not because a tutorial said RAG requires one.

What actually determines whether your app works is size and overlap in that chunking function. Chunk too small and you slice sentences from the context that explains them. Too large and you burn tokens on irrelevant text. The overlap exists so an answer straddling a boundary survives intact in at least one chunk. There's no correct value — change 1000 to 300, then to 3000, and watch the answers get worse in two different ways. That experiment will teach you more about RAG than any diagram.

Step 6: Agents — and why to skip them for now

An agent is a model that calls tools in a loop: decide what it needs, call a function, look at the result, decide again, until done. Instead of answering "find information about three competitors," it searches, compares, and writes the report.

This is where the field is heading, and genuinely not where you should start.

Agents fail in ways that are hard to debug. A single prompt either works or it doesn't. An agent taking twelve steps can go wrong at step four in a way that only surfaces at step eleven, and your logs show a plausible chain of reasoning that quietly went off a cliff. Costs compound too — every loop is another billed call.

The honest sequencing: build the assistant, run it for real users, learn what breaks. Then add one tool call — say, looking up a definition — and feel how the debugging changes. Starting with a multi-agent framework is how people end up with a system they can't fix.

Step 7: What turns a project into a product

The gap between "runs on my laptop" and "people depend on it" is mostly unglamorous:

• Evaluation. Write twenty questions with known answers and re-run them after every change. Otherwise you're guessing, and "it seemed better" is not a signal.
• Cost control. Know your per-request cost before launch. Cache repeated work, use a smaller model for easy calls, cap max_tokens.
• Failure handling. APIs rate-limit and time out. Retry with backoff, and decide what users see when the model is unavailable.
• Grounding. Show which chunk an answer came from. The cheapest trust mechanism there is.
• Prompt injection. If your app reads user-supplied documents, assume one contains "ignore your instructions." Treat document text as data, never as commands.

One thing you almost certainly don't need: fine-tuning. It's expensive, needs a real dataset, and is the wrong tool for teaching a model facts — that's retrieval's job. It shapes behaviour and format; exhaust prompting and retrieval first. Most teams reaching for it early are solving a prompting problem the hard way.

Common mistakes

Learning breadth-first. The ecosystem is enormous and most of it is irrelevant to your first three projects. Learn a tool when a problem demands it.

Stopping at prompting. Prompt engineering is real and useful, and roughly fifteen percent of building an AI product. The rest is ordinary software engineering.

Copying tutorials without breaking them. Change a parameter. Feed it a document it wasn't built for. Delete a line and see what dies. Understanding lives in the debugging, not the following.

Trusting the output. The model will produce a confident, well-formatted, entirely wrong answer, and nothing in its tone will warn you. Build the check into the product, not into your hopes.

What to do this week

Get an API key. Run the twelve lines in Step 3. Then take the thirty-line assistant in Step 4, point it at a PDF you actually need to understand — a contract, a manual, a paper — and ask five questions you already know the answers to.

You'll find at least one wrong answer. Fixing that specific answer is your real first lesson in Generative AI, and no tutorial can give it to you.

Then make it worse on purpose. Feed it something enormous and watch the failure from Step 5 happen to your own code. That's when RAG stops being a term and starts being a thing you needed.

The journey from prompt to product isn't a curriculum. It's one project you keep refusing to abandon.