How to give ChatGPT and Claude access to a live website

“Give the model access to the web” sounds like one thing and is actually four, with different costs, different failure modes and different amounts of control. Choosing wrong is why a lot of grounding work produces confident answers about pages nobody successfully read.

TLDR

• Pasting a URL into a chat delegates fetching and extraction to something you cannot inspect.
• Fetching yourself and pasting the text is the simplest thing that actually works, and it is usually enough.
• A tool call in your own code is right when the model should decide what to fetch.
• MCP is right when you want the model to reach the web from any client, without you writing the plumbing.
• The common failure is not hallucination. It is that the page never arrived and nothing said so.

Why pasting a URL is not the same as access

When you paste a link into a chat, something behind the scenes fetches it. You do not control which fetcher, whether JavaScript ran, how the content was extracted, or what happened if the site refused.

That last one is the problem. A refused fetch frequently returns a normal-looking page containing a cookie notice or a verification prompt. The model receives it, and it is well-formed prose, so it summarises it. You get a confident answer about a consent banner, phrased exactly like a correct answer would be.

For a casual question this is fine. For anything you are building on, you want to see the text before the model does.

Option one: fetch it yourself and put it in the prompt

The simplest approach that actually works. Fetch the page, convert it to clean markdown, put it in the prompt with the question.

page = fetch_as_markdown("https://example.com/article")

answer = model.complete(f"""
Answer using only the article below. If it does not say, say so.

<article>
{page}
</article>

Question: {question}
""")

Unglamorous and it covers most real cases. You can log exactly what was sent, check the page arrived, and diff it when an answer looks wrong. That auditability is the entire value.

Two things worth doing here:

• Check the content is really there before you spend a model call. A page that came back as three words of navigation should fail loudly, not get summarised.
• Tell the model what to do when the answer is absent. Without that instruction, a model asked a question about a page that does not address it will usually find something adjacent and answer anyway.

Option two: let the model decide what to fetch

Sometimes you do not know the URL in advance. The user asks something, and which page answers it is part of the problem.

That is a tool call: you describe a fetch tool, the model decides when to use it and on what, you execute and hand back the result. Every major SDK supports this and it is the right shape when discovery is genuinely part of the task.

The cost is a round trip and some non-determinism, since the model now chooses what to retrieve and can choose badly. Worth it when the alternative is guessing on the user’s behalf; not worth it when you already know the URL.

Option three: connect over MCP

The Model Context Protocol standardises how tools are exposed to models, so any MCP-capable client can use them without bespoke integration.

The practical difference: instead of writing fetch plumbing into your application, you connect an MCP server once and the model can reach the web from Claude Desktop, from an IDE, from an agent framework, from anything that speaks the protocol. Tools arrive described, so the model can see what is available rather than being told in a prompt.

This is the right answer when you want web access available across tools rather than embedded in one application, and when you would rather not maintain the integration yourself.

Option four: hand over a whole corpus

Sometimes the question is not about one page but about a site: all the documentation, every changelog entry, an entire knowledge base.

That is a crawl into a retrieval index, not a fetch. It is a genuinely different piece of work with its own decisions about chunking and refresh cadence, and reaching for it when you needed a single page is a common and expensive mistake.

Choosing between them

you know the URL	model picks the URL	across many clients	whole site
fetch and paste	tool call	MCP	crawl and index

Most people need the first one and reach for the fourth.

The failure that actually bites

Grounding rarely fails because a model invented something from nothing. It fails because the content never arrived and nothing in the pipeline said so.

The page was walled, or JavaScript-rendered and captured before it rendered, or the extractor took the navigation and dropped the article. What reached the model was empty or wrong, and a model given no useful content will still produce a fluent answer.

So the check that matters is not on the model, it is on the input. Before the prompt is built, ask: did real content come back, is there enough of it, and does anything indicate a wall. If your pipeline cannot answer those, it cannot tell a good answer from a confident one.

The short version

Fetch it yourself, look at what you got, then give it to the model. Everything else is an optimisation on top of that.

If you want the fetching part handled, scrape returns clean markdown with a quality signal on every response, so your pipeline can tell an empty page from a real one before it spends a model call. There is also an MCP server if you would rather the model reach the web itself. One credit a page, failures never billed, and 500 free credits on a work email.