# Robots meta tag: how noindex and nofollow work

> Apply noindex to a page, distinguish page-level nofollow and inspect HTTP headers for resources such as PDFs without blocking the directives from being read.


**A robots meta tag communicates indexing and link-following instructions for an HTML page.** `noindex` asks for the page to be excluded from the index. `nofollow` asks for its links not to be followed; Google treats it as a hint rather than a guarantee that destinations will remain undiscovered. Google needs access to the response to read these instructions: blocking crawling can prevent it from finding the tag.

## Noindex for an HTML page

Suppose a public confirmation page contains no private information, but the business wants it excluded from search. This tag in the `head` expresses that choice:

```html
<meta name="robots" content="noindex" />
```

There is no `nofollow` in the example: excluding the page does not require applying that instruction to all its links. If a CMS emits `noindex, follow`, the `follow` value states the default following behavior; it does not make the page indexable.

Google explains the scope of these rules in its [robots meta tag and X-Robots-Tag documentation](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag). Check every applicable instruction when they conflict. A permissive declaration does not automatically cancel a more restrictive one.

## What nofollow changes

This version applies a link-following instruction across the page:

```html
<meta name="robots" content="nofollow" />
```

It is distinct from `noindex`. To describe the relationship of a particular link, such as a paid placement, review [nofollow, sponsored and ugc attributes](/en/guides/nofollow-sponsored-ugc/). Using a page-wide value to address one link may affect more content than intended.

Nofollow also cannot hide destinations. Google may discover those URLs through other links or sources. Private documents require appropriate access controls.

## PDFs and other resources: the HTTP header

A PDF has no HTML `head` for a meta tag. The server can return this header to communicate exclusion for that resource:

```http
X-Robots-Tag: noindex
```

Inspect the exact file's response. An overly broad server rule might also affect PDFs that should appear in search. With a CMS, identify the responsible setting before changing a whole folder without examining its contents.

## The common robots.txt conflict

Blocking a URL in [robots.txt](/en/guides/robots-txt-explained/) may prevent Google from crawling it and reading `noindex`. A blocked address can still appear in results based on outside references, even without its content being shown. Exclusion requires an instruction the search engine can process.

Before changing a block, establish why it exists and whether the page contains anything that must remain restricted. Opening a private area solely to expose a tag to a crawler would be the wrong fix. Noindex governs search inclusion; it does not grant or restrict access.

## Verify the actual response

Check the final URL, published HTML and headers. Look for directives from the template, SEO plugin and server. For a property you manage, [URL Inspection](https://support.google.com/webmasters/answer/9012289) helps compare indexed information with a live test.

After correcting an accidental directive, verify the new response and record the change date. That check does not confirm an index update or establish a removal or inclusion deadline. For a public class page that should not be excluded, the task should identify the exact rule to remove, where it is generated and how the corrected response will be verified.

## Plan your next post in HeyMark.

Keep the idea, review the draft with your team, and see how it performed in the accounts you connected.

[Start free](https://app.heymark.ai) · [See how it works](https://heymark.ai/en/#product)

## Developer and agent resources

- [HeyMark MCP documentation](https://heymark.ai/en/mcp/)
- [llms.txt](https://heymark.ai/llms.txt)
- [Full site content for language models](https://heymark.ai/llms-full.txt)
- MCP protocol endpoint: `POST https://mcp.heymark.ai`
- OAuth protected-resource metadata: [/.well-known/oauth-protected-resource](https://mcp.heymark.ai/.well-known/oauth-protected-resource)
- MCP server card: [/server-card](https://mcp.heymark.ai/server-card)
