> ## Documentation Index
> Fetch the complete documentation index at: https://docs.novita.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tavily Crawl

Tavily is a web data retrieval API designed for developers and AI applications. This document describes the Tavily passthrough endpoints exposed by the platform gateway. It is based on Tavily's Crawl API with platform route prefixes applied.

Base URL example: `https://api.novita.ai`

Crawl a site starting from a root URL, following links to retrieve content from multiple pages.

## Request Headers

All endpoints require platform API authentication.

**Content-Type** `string` *required*

Use `application/json`.

**Authorization** `string` *required*

Platform API key, formatted as `Bearer <api_key>`.

## Request Body

**url** `string` *required*

Crawl root URL.

**instructions** `string`

Crawler instructions in natural language.

**chunks\_per\_source** `integer`

Max number of content snippets per source to return. Default is `3`; range is `1` to `5`. Available only when `instructions` are set.

**max\_depth** `integer`

Crawler depth limit. Default is `1`; range is `1` to `5`.

**max\_breadth** `integer`

Max links per page. Default is `20`; range is `1` to `500`.

**limit** `integer`

Link processing limit for crawler. Default is `50`; minimum is `1`.

**select\_paths** `string[]`

Restrict URL paths matching these regex patterns.

**select\_domains** `string[]`

Restrict domains matching these regex patterns.

**exclude\_paths** `string[]`

Exclude URL paths matching these regex patterns.

**exclude\_domains** `string[]`

Exclude domains matching these regex patterns.

**allow\_external** `boolean`

Include external domain links.

**include\_images** `boolean`

Returns a list of images along with the response.

**extract\_depth** `string`

Extraction mode. Known values include `basic` and `advanced`. `basic` is set as the default value.

**format** `string`

Output format of extracted content. Common values include `markdown` and `text`. `markdown` is set as the default value.

**include\_favicon** `boolean`

Include favicon URL from result.

**timeout** `float`

Crawl operation time limit in seconds. Default is `150`; range is `10` to `150`.

## Request Example

```bash theme={"system"}
curl -s -X POST 'https://api.novita.ai/v3/tavily/crawl' \
  -H 'Authorization: Bearer sk-123' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://docs.tavily.com",
    "chunks_per_source": 3,
    "extract_depth": "basic",
    "format": "markdown",
    "max_depth": 1,
    "max_breadth": 5,
    "limit": 5,
    "allow_external": false,
    "include_images": false,
    "timeout": 120
  }'
```

## Response

**base\_url** `string`

Crawl base URL.

**results** `object[]`

Extracted content from crawled URLs. See [Result Object](#result-object).

**response\_time** `float`

Request duration in seconds.

**request\_id** `string`

Unique request identifier.

### Result Object

**url** `string`

Crawled URL.

**raw\_content** `string`

Full page content.

**favicon** `string`

Site favicon URL. Returned if `include_favicon` is `true`.

## Errors

The platform may return standard HTTP errors before forwarding the request, and Tavily may return upstream errors after forwarding.

**400** `status`

Invalid request body or unsupported parameter value.

**401** `status`

Missing or invalid API key.

**403** `status`

Access denied by platform or upstream provider, or the URL is not supported.

**404** `status`

Route or requested resource not found.

**429** `status`

Rate limit exceeded.

**500** `status`

Internal server error.

**502** `status`

Upstream provider error.

**503** `status`

Service unavailable.

## Notes

* All request bodies are JSON.
* Extra Tavily parameters not listed here may be passed through.
* Response shapes can vary depending on request options.
* This document intentionally omits billing-related fields.

## References

For more details, see the [Tavily Crawl API reference](https://docs.tavily.com/documentation/api-reference/endpoint/crawl).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.