# Answer API
Source: https://docs.llmlayer.ai/answer
Generate web-grounded answers with /answer and real-time SSE with /answer_stream.
## Overview
The Answer API combines live web search with LLM generation in one request.
Use it when you need:
* Current information from the web
* Source-backed responses
* Structured output (JSON) for downstream processing
* Streaming UX for chat and copilots
## Endpoints
| Endpoint | Method | Best for | Supports JSON output |
| ----------------------- | ------------ | ------------------------------- | -------------------- |
| `/api/v2/answer` | `POST` | Standard request/response flows | Yes |
| `/api/v2/answer_stream` | `POST` (SSE) | Real-time streaming UX | No |
`provider_key` is accepted for backward compatibility but currently ignored; provider usage is not billed to the user's provider account.
## Authentication
All requests require:
```bash theme={null}
Authorization: Bearer YOUR_LLMLAYER_API_KEY
```
Keep API keys server-side. Do not expose them in browser code.
## Model Selection
| Model | Pricing model | Best for |
| -------------------- | ---------------------------- | ------------------------- |
| `llmlayer-web` | Flat `$0.007 × max_queries` | Default recommendation |
| `llmlayer-fast` | Flat `$0.009 × max_queries` | Faster responses |
| `openai/gpt-4o-mini` | Token pricing + LLMLayer fee | Budget + quality |
| `openai/gpt-5.1` | Token pricing + LLMLayer fee | Highest reasoning quality |
If unsure, start with `llmlayer-web` and `max_queries=1`.
## Quickstart
### Non-streaming (`/answer`)
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.answer({
query: 'What are the latest developments in quantum computing?',
model: 'llmlayer-web',
returnSources: true,
});
console.log(response.answer);
console.log('Sources:', response.sources?.length || 0);
console.log('LLMLayer cost:', response.llmlayer_cost);
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key='YOUR_LLMLAYER_API_KEY')
response = client.answer(
query='What are the latest developments in quantum computing?',
model='llmlayer-web',
return_sources=True,
)
print(response.answer)
print('Sources:', len(response.sources or []))
print('LLMLayer cost:', response.llmlayer_cost)
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/answer \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "What are the latest developments in quantum computing?",
"model": "llmlayer-web",
"return_sources": true
}'
```
Example response (simplified):
```json theme={null}
{
"answer": "...",
"sources": [
{
"title": "...",
"link": "https://...",
"snippet": "..."
}
],
"response_time": "2.14",
"input_tokens": 1432,
"output_tokens": 288,
"model_cost": null,
"llmlayer_cost": 0.007
}
```
### Streaming (`/answer_stream`)
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const stream = client.streamAnswer({
query: 'Explain retrieval-augmented generation in simple terms',
model: 'openai/gpt-4o-mini',
returnSources: true,
});
for await (const event of stream) {
if (event.type === 'answer') {
process.stdout.write(event.content || '');
} else if (event.type === 'sources') {
console.log('\n\nSources:', event.data?.length || 0);
} else if (event.type === 'usage') {
console.log('\nCost:', (event.model_cost || 0) + (event.llmlayer_cost || 0));
} else if (event.type === 'done') {
console.log('\nDone in', event.response_time, 'seconds');
}
}
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key='YOUR_LLMLAYER_API_KEY')
stream = client.stream_answer(
query='Explain retrieval-augmented generation in simple terms',
model='openai/gpt-4o-mini',
return_sources=True,
)
for event in stream:
t = event.get('type')
if t == 'answer':
print(event.get('content', ''), end='', flush=True)
elif t == 'sources':
print('\n\nSources:', len(event.get('data', [])))
elif t == 'usage':
total = (event.get('model_cost') or 0) + (event.get('llmlayer_cost') or 0)
print('\nCost:', total)
elif t == 'done':
print('\nDone in', event.get('response_time'), 'seconds')
```
```bash cURL (SSE) theme={null}
curl -N -X POST https://api.llmlayer.dev/api/v2/answer_stream \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"query": "Explain retrieval-augmented generation in simple terms",
"model": "openai/gpt-4o-mini",
"return_sources": true
}'
```
## Request Parameters
This is the canonical request-body table for both endpoints.
| Parameter | Type | Required | Default | Applies to | Cost impact | Details |
| --------------------- | ---------- | ----------- | ---------- | ---------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `query` | `string` | Yes | - | Both | - | User question/instruction |
| `model` | `string` | Yes | - | Both | Depends on model | Example: `llmlayer-web`, `openai/gpt-4o-mini` |
| `search_type` | `string` | No | `general` | Both | - | `general` or `news` |
| `date_filter` | `string` | No | `anytime` | Both | - | `anytime`, `hour`, `day`, `week`, `month`, `year` |
| `location` | `string` | No | `us` | Both | - | Country code for localized search |
| `domain_filter` | `string[]` | No | `null` | Both | - | Include domains, or exclude with `-` prefix |
| `search_context_size` | `string` | No | `medium` | Both | Indirect | `low`, `medium`, `high` |
| `max_queries` | `integer` | No | `1` | Both | Increases LLMLayer fee | Range `1-4` |
| `max_tokens` | `integer` | No | `1500` | Both | Increases model usage | Response length cap |
| `temperature` | `number` | No | `0.7` | Both | - | Range `0.0-2.0` |
| `response_language` | `string` | No | `auto` | Both | - | Example: `en`, `fr`, `es` |
| `citations` | `boolean` | No | `false` | Both | Indirect | Adds inline citation markers |
| `return_sources` | `boolean` | No | `false` | Both | - | Includes `sources` array |
| `return_images` | `boolean` | No | `false` | Both | `+ $0.001` | Includes `images` array |
| `answer_type` | `string` | No | `markdown` | `/answer` | - | `markdown`, `html`, `json` |
| `json_schema` | `string` | Conditional | `null` | `/answer` | - | Required when `answer_type="json"`. Raw HTTP expects a JSON-encoded string; SDKs accept objects and serialize them. |
| `system_prompt` | `string` | No | `null` | Both | - | Custom behavior instructions |
| `provider_key` | `string` | No | `null` | Both | - | Accepted for backward compatibility, ignored |
HTTP requests use `snake_case` field names. JavaScript SDK examples use `camelCase`.
For `/answer` JSON mode, raw REST requests should send `json_schema` as a string. The Python and TypeScript SDKs accept schema objects for convenience and serialize them before sending the request.
### Parameter Rules
1. `max_queries` must be between `1` and `4`.
2. `answer_type="json"` requires `json_schema`.
3. `/api/v2/answer_stream` does not support structured JSON output.
4. `provider_key` does not change routing or billing, and provider usage is not billed to the user's provider account.
5. Use `-domain.com` in `domain_filter` to exclude domains.
## Non-streaming Response Contract (`/answer`)
| Field | Type | When present | Description |
| --------------- | ------------------ | ------------------------ | ------------------------------------------ |
| `answer` | `string \| object` | Always | Generated answer (`object` when JSON mode) |
| `sources` | `array` | If `return_sources=true` | Source documents used |
| `images` | `array` | If `return_images=true` | Image search results |
| `response_time` | `string` | Always | Total processing time in seconds |
| `input_tokens` | `integer` | Always | Input token usage |
| `output_tokens` | `integer` | Always | Output token usage |
| `model_cost` | `number \| null` | Usually | Model usage cost |
| `llmlayer_cost` | `number` | Always | LLMLayer infrastructure cost |
Source objects typically include `title`, `link`, `snippet` (plus provider-specific extras).
Image objects typically include `title`, `imageUrl`, `thumbnailUrl`, `source`, `link`.
## Streaming Event Contract (`/answer_stream`)
The stream is Server-Sent Events (`text/event-stream`). Each frame contains JSON under `data:`.
| Event type | Payload | Notes |
| ---------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------- |
| `sources` | `{ "type": "sources", "data": Source[] }` | Emitted when `return_sources=true` |
| `images` | `{ "type": "images", "data": Image[] }` | Emitted when `return_images=true` |
| `answer` | `{ "type": "answer", "content": "..." }` | Main text chunks |
| `usage` | `{ "type": "usage", "input_tokens": ..., "output_tokens": ..., "model_cost": ..., "llmlayer_cost": ... }` | Billing and token usage |
| `done` | `{ "type": "done", "response_time": "..." }` | Final event |
| `error` | `{ "type": "error", "error": "..." }` | Runtime stream error |
Example stream sequence:
```json theme={null}
{"type":"sources","data":[...]}
{"type":"answer","content":"The "}
{"type":"answer","content":"main idea ..."}
{"type":"usage","input_tokens":1234,"output_tokens":210,"model_cost":0.0002,"llmlayer_cost":0.004}
{"type":"done","response_time":"2.41"}
```
On early setup failures, some clients may receive an immediate frame like `{ "error": "missing_query" }`.
## Practical Examples
### 1) News summary with citations
```json theme={null}
{
"query": "Latest developments in renewable energy",
"model": "openai/gpt-5.1",
"search_type": "news",
"date_filter": "week",
"citations": true,
"return_sources": true,
"max_queries": 2
}
```
### 2) Structured JSON extraction (`/answer` only)
```json theme={null}
{
"query": "Summarize top AI model launches this month",
"model": "openai/gpt-5.1",
"answer_type": "json",
"json_schema": "{\"type\":\"object\",\"properties\":{\"summary\":{\"type\":\"string\"},\"items\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}}},\"required\":[\"summary\",\"items\"]}"
}
```
### 3) Domain-constrained answer
```json theme={null}
{
"query": "What are current type 2 diabetes treatment guidelines?",
"model": "openai/gpt-5.1",
"domain_filter": ["pubmed.gov", "nih.gov", "-reddit.com"],
"search_context_size": "high",
"temperature": 0.3,
"return_sources": true
}
```
## Error Handling
### `/answer` error format
```json theme={null}
{
"detail": {
"error_type": "validation_error",
"error_code": "missing_query",
"message": "Query parameter cannot be empty",
"details": null
}
}
```
### Common status codes
| Status | Category | Typical reason |
| ------ | ------------------- | ----------------------------------- |
| `400` | Validation | Missing/invalid request parameters |
| `401` | Authentication | Missing or invalid LLMLayer API key |
| `429` | Rate limit/provider | Provider or account rate limiting |
| `500` | Internal/provider | Unexpected backend/provider failure |
| `502` | Provider | Upstream provider-specific failure |
### `/answer_stream` errors
* Runtime failures are emitted as stream events (`type: error`).
* Early validation failures can appear as an immediate single `error` frame.
## Pricing
### Standard token-priced models
```text theme={null}
Total = (0.004 × max_queries)
+ model_input_cost
+ model_output_cost
+ (0.001 if return_images=true)
```
### LLMLayer fixed-price models
```text theme={null}
llmlayer-web = 0.007 × max_queries
llmlayer-fast = 0.009 × max_queries
```
Use `llmlayer_cost` and `model_cost` from responses as the billing source of truth.
## Implementation Checklist
1. Start with `/answer` unless you need progressive rendering.
2. Set `max_queries=1` first; increase only for research-style queries.
3. Enable `return_sources=true` for trust-sensitive use cases.
4. Use `answer_type="json"` + `json_schema` for structured pipelines.
5. Add retry/backoff logic for transient `429/500/502` paths.
6. Keep keys server-side and log `llmlayer_cost` + token usage.
## FAQ
Use `/answer_stream` for chat UIs and live typing effects. Use `/answer` for batch jobs, strict request/response flows, and JSON structured output.
No. Streaming returns incremental text chunks and does not support JSON schema-constrained output.
Use `domain_filter`, `search_type`, `date_filter`, and `search_context_size` together. For factual tasks, use lower `temperature`.
It is accepted for backward compatibility but currently ignored; provider usage is not billed to the user's provider account.
The backend currently falls back to `llmlayer-web` instead of failing the request.
Start with `llmlayer-web`, keep `max_queries=1`, only request images/sources when needed, and tune `max_tokens` to expected output length.
## Next Steps
Raw search results without LLM generation
Extract full page content from URLs
OpenAPI reference for SSE endpoint
Python package and usage examples
JS/TS package and usage examples
## Need Help?
Ask implementation questions
[support@llmlayer.ai](mailto:support@llmlayer.ai)
# Answer API - Web-enhanced AI responses
Source: https://docs.llmlayer.ai/api-reference/endpoint/answer
post /api/v2/answer
Search the web and generate AI-powered answers using your chosen LLM model. Supports multiple search queries, custom prompts, and structured JSON output.
# Crawl API - Stream markdown pages from a website
Source: https://docs.llmlayer.ai/api-reference/endpoint/crawl
post /api/v2/crawl_stream
Crawl multiple pages from a website with depth control. Streams Server-Sent Events as pages are scraped. The public endpoint currently returns markdown content per page. Billed per successfully scraped page ($0.001/page).
# Extract API - Multi-mode page extraction
Source: https://docs.llmlayer.ai/api-reference/endpoint/extract
post /api/v2/extract
Extract from a single web page in one call. Combine any modes: structured JSON ('json', $0.005), a markdown summary ('summary', $0.005), an answer to a question ('qa', $0.005), all page links ('links', $0.001), and a brand profile with logos/colors/socials ('brand', $0.002). Pricing is summed per selected mode; advanced proxy adds $0.004 once when a scrape runs. If the request fails before any AI cost is incurred (page fetch failure, empty content, brand fetch failure), you are fully refunded.
# Map API - Discover URLs on a website
Source: https://docs.llmlayer.ai/api-reference/endpoint/map
post /api/v2/map
Firecrawl-style URL discovery. Maps all URLs on a website with lightweight titles (no content extraction). Useful for site exploration before crawling.
# Scraper API - Multi-format content extraction
Source: https://docs.llmlayer.ai/api-reference/endpoint/scrape
post /api/v2/scrape
Extract content from a non-PDF URL as markdown, HTML, or a screenshot (base64). Direct PDF URLs are rejected; use /api/v2/get_pdf_content.
# PDF Content API - Extract text from PDFs
Source: https://docs.llmlayer.ai/api-reference/endpoint/scrape-pdf
post /api/v2/get_pdf_content
Extract text content from PDF documents. 15-second timeout with proper 504 handling.
# Answer API - Stream responses in real-time
Source: https://docs.llmlayer.ai/api-reference/endpoint/stream-answer
post /api/v2/answer_stream
Stream search results and AI responses in real-time using Server-Sent Events (SSE). Does not support JSON structured output.
# Web Search API - Raw search results
Source: https://docs.llmlayer.ai/api-reference/endpoint/web-search
post /api/v2/web_search
Search the web across multiple content types (general, news, shopping, videos, images, scholar) without AI processing. Returns raw search results.
# YouTube Transcript API - Extract video transcripts
Source: https://docs.llmlayer.ai/api-reference/endpoint/youtube-transcript
post /api/v2/youtube_transcript
Extract transcripts from YouTube videos with optional language selection. Returns transcript plus video metadata (title, author, views, likes, date).
# API Reference
Source: https://docs.llmlayer.ai/api-reference/introduction
HTTP endpoints, authentication, request bodies, and generated OpenAPI schemas.
## Overview
LLMLayer exposes REST endpoints under `https://api.llmlayer.dev/api/v2`. Authenticate every request with a Bearer token.
```bash theme={null}
Authorization: Bearer YOUR_LLMLAYER_API_KEY
```
Keep API keys server-side. Do not expose them in browser code.
## Endpoints
`/api/v2/answer` - web-grounded LLM responses.
`/api/v2/answer_stream` - Server-Sent Events for streaming answers.
`/api/v2/web_search` - raw search results.
`/api/v2/scrape` - single-page markdown, HTML, or screenshot extraction.
`/api/v2/extract` - structured data, summaries, Q\&A, links, and brand profiles.
`/api/v2/map` - website URL discovery.
`/api/v2/crawl_stream` - streamed markdown pages from a website.
## First Request
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
response = client.answer(
query="What happened in AI today?",
model="llmlayer-web",
search_type="news",
return_sources=True,
)
print(response.answer)
```
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.answer({
query: 'What happened in AI today?',
model: 'llmlayer-web',
searchType: 'news',
returnSources: true,
});
console.log(response.answer);
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/answer \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "What happened in AI today?",
"model": "llmlayer-web",
"search_type": "news",
"return_sources": true
}'
```
## Field Naming
Raw HTTP request bodies use the field names shown in the OpenAPI schemas. The Python SDK follows Python naming conventions for method arguments, and the TypeScript SDK uses `camelCase` for many convenience options.
Examples:
| Raw HTTP | Python SDK | TypeScript SDK |
| ------------------- | ------------------- | ----------------- |
| `return_sources` | `return_sources` | `returnSources` |
| `json_schema` | `json_schema` | `jsonSchema` |
| `main_content_only` | `main_content_only` | `mainContentOnly` |
| `statusCode` | `statusCode` | `statusCode` |
| `structured_data` | `structured_data` | `structured_data` |
## Related Guides
Shared error format and refund behavior.
Supported model IDs and pricing formulas.
# Changelog
Source: https://docs.llmlayer.ai/changelog
New features, improvements, and fixes across the LLMLayer API, SDKs, and dashboard.
## New: Extract API — `POST /api/v2/extract`
Turn any web page into exactly the data you need, in a single call. Combine any of five modes — one page fetch is shared across all of them, and AI modes run in parallel:
| Mode | What you get | Price |
| --------- | -------------------------------------------------------------------------------------- | ------- |
| `json` | Structured data matching your schema (formal schema, example object, or plain English) | \$0.005 |
| `summary` | Faithful markdown summary | \$0.005 |
| `qa` | Answer to your question, grounded only in the page | \$0.005 |
| `links` | Every link on the page, deduplicated, internal/external classified | \$0.001 |
| `brand` | Brand kit: logos, colors, socials, industry classification | \$0.002 |
**Highlights:**
* **List extraction with real URLs** — when your schema asks for links, the API recovers per-item hyperlinks that page-to-text conversion normally loses
* **Fair billing** — failures before any AI cost (page fetch failure, empty content, brand failure) are **fully refunded**, automatically
* **Page metadata included** by default on every response
* All five result fields are always present in the response; unselected modes are `null`
[Read the Extract API docs →](/extract)
## SDKs v0.3.0
* **Python** (`pip install llmlayer`): new `client.extract()` and `client.extract_async()`. The `json` mode result is exposed as `response.structured_data`
* **TypeScript** (`npm install llmlayer`): new `client.extract()` with both `extract(args)` and `extract(url, opts)` call styles
* Both SDKs now map HTTP 422 responses to `InvalidRequest` for cleaner error handling
## Dashboard
* New **Extract playground** with a visual JSON schema builder: typed fields, nested objects, list-of-objects extraction, and a live schema preview — no JSON writing required
## Docs
* New [Errors & Refunds](/errors) reference covering every error format, status code, and the automatic refund policy
* This changelog
# Crawl API
Source: https://docs.llmlayer.ai/crawl
Stream markdown content from multiple pages on a website.
## Overview
Use Crawl when you need content from multiple pages under one site. The endpoint streams Server-Sent Events as pages finish, so your application can process pages without waiting for the full crawl to complete.
The public crawl endpoint currently returns markdown page content only.
## Endpoint
`POST /api/v2/crawl_stream`
## Quickstart
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
for await (const event of client.crawlStream({
url: 'https://www.ycombinator.com',
maxPages: 10,
maxDepth: 2,
mainContentOnly: true,
})) {
if (event.type === 'page') {
console.log(event.page.final_url, event.page.markdown?.slice(0, 120));
}
if (event.type === 'usage') {
console.log('Cost:', event.cost);
}
}
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
for event in client.crawl_stream(
"https://www.ycombinator.com",
max_pages=10,
max_depth=2,
main_content_only=True,
):
if event.get("type") == "page":
page = event["page"]
print(page.get("final_url"), (page.get("markdown") or "")[:120])
if event.get("type") == "usage":
print("Cost:", event.get("cost"))
```
```bash cURL theme={null}
curl -N -X POST https://api.llmlayer.dev/api/v2/crawl_stream \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"url": "https://www.ycombinator.com",
"max_pages": 10,
"max_depth": 2,
"main_content_only": true
}'
```
## Request Parameters
| Parameter | Type | Required | Default | Description |
| -------------------- | ---------------- | -------- | -------------- | ---------------------------------------------------- |
| `url` | `string` | Yes | - | Seed URL |
| `max_pages` | `integer` | No | `25` | Maximum pages to return, hard limit `100` |
| `max_depth` | `integer` | No | `2` | Link depth from the seed URL |
| `timeout` | `number \| null` | No | `60` | Total crawl time budget in seconds |
| `include_subdomains` | `boolean` | No | `false` | Include subdomains |
| `include_links` | `boolean` | No | `true` | Keep links in markdown content |
| `include_images` | `boolean` | No | `true` | Keep image references in markdown content |
| `advanced_proxy` | `boolean` | No | `false` | Use for protected sites |
| `main_content_only` | `boolean` | No | `false` | Reduce navigation and boilerplate |
| `formats` | `["markdown"]` | No | `["markdown"]` | Accepted for compatibility; only markdown is honored |
HTTP requests use `snake_case`. The TypeScript SDK uses `camelCase`, for example `maxPages`, `maxDepth`, and `mainContentOnly`.
## Stream Events
Each SSE frame contains a JSON object under `data:`.
### Page
```json theme={null}
{
"type": "page",
"page": {
"requested_url": "https://www.ycombinator.com",
"final_url": "https://www.ycombinator.com",
"title": "Y Combinator",
"hash_sha256": "abc123...",
"markdown": "# Docs\n\n...",
"success": true,
"error": null
}
}
```
### Usage
```json theme={null}
{
"type": "usage",
"billed_count": 10,
"unit_cost": 0.001,
"cost": 0.01
}
```
### Done
```json theme={null}
{
"type": "done",
"response_time": "21.44"
}
```
### Error
```json theme={null}
{
"type": "error",
"error": "Upstream crawl failed"
}
```
## Pricing
Crawl reports usage as `$0.001` per successfully scraped page in the `usage` event. Advanced proxy can improve success rates on protected sites.
Use the emitted `usage.cost` and your dashboard ledger as the billing source of truth.
## When to Use Map First
Use [Map](/map) before Crawl when you want to inspect or filter URLs before fetching page content.
```python theme={null}
site = client.map("https://www.ycombinator.com", limit=100)
print([str(link.url) for link in site.links[:10]])
```
## Errors
| Status / event | Meaning |
| -------------- | --------------------------------------------- |
| `400` | Invalid URL, invalid `max_pages`, or PDF URL |
| `401` | Missing or invalid LLMLayer API key |
| `500` event | Upstream crawl failure after streaming starts |
See [Errors & Refunds](/errors) for the shared error format.
## More Examples
Persist pages, handle usage events, and retry failures.
Discover URLs before crawling content.
# Errors & Refunds
Source: https://docs.llmlayer.ai/errors
Every error format, status code, and error type the LLMLayer API returns - plus when you are automatically refunded.
## Error Formats
LLMLayer returns errors in two shapes, depending on where the request fails.
### 1. Structured Errors (most endpoint errors)
Endpoint-level failures return a structured envelope inside FastAPI's `detail` field:
```json theme={null}
{
"detail": {
"error_type": "validation_error",
"error_code": "missing_json_schema",
"message": "json_schema is required when 'json' is in modes. ...",
"details": { "url": "https://www.ycombinator.com" }
}
}
```
| Field | Description |
| ------------ | ---------------------------------------------------- |
| `error_type` | Error category — see the [table below](#error-types) |
| `error_code` | Machine-readable code, stable per failure case |
| `message` | Human-readable explanation |
| `details` | Optional context (URL, provider, status code, ...) |
### 2. Plain-String Errors (account-level guards)
Account-level checks (API key, balance, rate limit) return a plain string in `detail`:
```json theme={null}
{ "detail": "insufficient funds on LLMLAYER, please top up your account" }
```
These occur **before** your request is processed, so they never incur any charge.
***
## HTTP Status Codes
| Status | Meaning | Charged? | Retry? |
| ------ | --------------------------------------------------------------------------------- | ------------------------------------------- | --------------------------- |
| `400` | Invalid request — missing or malformed parameter | No | After fixing the request |
| `401` | Missing or invalid API key | No | After fixing the key |
| `402` | Insufficient credits | No | After topping up |
| `403` | Blocked target (private IP, blocked domain or port) | No | No — target not allowed |
| `422` | Unprocessable — e.g. empty page content (refunded) or extraction output truncated | Depends — see [Refunds](#automatic-refunds) | After adjusting the request |
| `423` | Billing reconciliation in progress for this key | No | Yes, after a short wait |
| `429` | Rate limit exceeded for your [tier](/rate-limits) | No | Yes, with backoff |
| `500` | Internal error or upstream fetch failure | Depends — see [Refunds](#automatic-refunds) | Yes |
| `502` | Upstream service failure (e.g. brand profile, model output invalid) | Depends — see [Refunds](#automatic-refunds) | Yes |
| `504` | Operation timeout (e.g. PDF scrape, DNS resolution) | — | Yes |
***
## Error Types
The `error_type` field groups errors into categories:
| `error_type` | Used for | Typical statuses |
| ---------------------- | -------------------------------------------------------------- | -------------------------------- |
| `validation_error` | Bad or missing request parameters | 400 |
| `authentication_error` | Missing API key header | 401 |
| `provider_error` | An upstream AI provider failed (rate limit, outage, auth) | 401, 429, 500 |
| `scraping_error` | Scrape/PDF/URL-validation failures (Scraper, PDF, URL checks) | 400, 403, 500, 504 |
| `search_error` | Web search failures (Answer API) | 500 |
| `extraction_error` | Extract API failures (fetch, empty content, truncation, brand) | 422, 500, 502 |
| `map_error` | Map endpoint failures | 4xx/5xx (passes upstream status) |
| `internal_error` | Unexpected server errors | 500 |
Streaming endpoints (`/answer_stream`, `/crawl_stream`) deliver mid-stream failures as SSE frames of the form `{"type": "error", "error": "..."}` instead of HTTP error responses, because the HTTP status is already committed once streaming starts.
***
## Automatic Refunds
The **Extract API** debits your selected modes up front, then **automatically refunds the full amount** if the request fails before any AI cost is incurred:
| Failure | Error code | Status | Refunded |
| ------------------------------------ | -------------------- | ---------------------------- | ------------- |
| Target page could not be fetched | `page_fetch_failed` | 500 (or the target's status) | ✅ Full refund |
| Page returned no extractable content | `empty_page_content` | 422 | ✅ Full refund |
| Brand profile could not be built | `brand_fetch_failed` | 502 | ✅ Full refund |
Refunded errors say so explicitly: *"You have not been charged for this request."*
Failures that happen **after** AI processing has started (e.g. an upstream model outage, or `output_truncated` when extracted JSON exceeds the size limit) are not refunded, because the model cost was already incurred.
Account-level rejections (401, 402, 423, 429) always happen **before** any charge.
***
## SDK Exception Mapping
Both official SDKs raise typed exceptions so you can branch cleanly:
| Condition | Python (`llmlayer.exceptions`) | TypeScript (`llmlayer`) |
| -------------------------------------------------------------------------------- | ------------------------------ | ---------------------------- |
| `validation_error`, or status 400 / 422 | `InvalidRequest` | `InvalidRequest` |
| `authentication_error`, or status 401 / 403 | `AuthenticationError` | `AuthenticationError` |
| `provider_error` | `ProviderError` | `ProviderError` |
| Status 429 | `RateLimitError` | `RateLimitError` |
| `internal_error`, `scraping_error`, `search_error`, `map_error`, or status ≥ 500 | `InternalServerError` | `InternalServerError` |
| Anything else (e.g. 402 insufficient credits, 423) | `LLMLayerError` (base class) | `LLMLayerError` (base class) |
All exceptions inherit from `LLMLayerError`, so `except LLMLayerError` / `catch` on the base class catches everything.
### Recommended Retry Strategy
```typescript TypeScript theme={null}
import {
LLMLayerClient,
InvalidRequest,
AuthenticationError,
RateLimitError,
InternalServerError,
} from 'llmlayer';
async function withRetries(fn: () => Promise, maxRetries = 3): Promise {
for (let attempt = 0; ; attempt++) {
try {
return await fn();
} catch (error) {
// Never retry: fix the request or the key instead
if (error instanceof InvalidRequest) throw error;
if (error instanceof AuthenticationError) throw error;
// Retry with backoff: rate limits and server-side failures
const retryable = error instanceof RateLimitError || error instanceof InternalServerError;
if (!retryable || attempt >= maxRetries - 1) throw error;
await new Promise(r => setTimeout(r, 2 ** attempt * 1000));
}
}
}
```
```python Python theme={null}
import time
from llmlayer.exceptions import (
InvalidRequest,
AuthenticationError,
RateLimitError,
InternalServerError,
)
def with_retries(fn, max_retries=3):
for attempt in range(max_retries):
try:
return fn()
except (InvalidRequest, AuthenticationError):
raise # never retry: fix the request or the key instead
except (RateLimitError, InternalServerError):
if attempt == max_retries - 1:
raise
time.sleep(2 ** attempt) # backoff, then retry
```
***
## Common Error Codes by Endpoint
| Code | Status | Meaning |
| ---------------------------------------------------- | ------ | ---------------------------------------------------------- |
| `missing_llmlayer_api_key` | 401 | No `Authorization: Bearer ` header |
| *(plain string)* `LLMLAYER API KEY NOT VALID` | 401 | Key is malformed, revoked, or unknown |
| *(plain string)* `insufficient funds on LLMLAYER...` | 402 | Credit balance too low — [top up](https://app.llmlayer.ai) |
| *(plain string)* `rate N/min exceeded on LLMLAYER` | 429 | Over your tier's requests-per-minute |
| Code | Status | Refunded |
| ------------------------------------------------------------------------------- | ------------------- | ----------------------------------------------- |
| `missing_modes` / `missing_json_schema` / `missing_query` / `pdf_not_supported` | 400 | Nothing charged |
| `page_fetch_failed` | 500 / target status | ✅ Yes |
| `empty_page_content` | 422 | ✅ Yes |
| `brand_fetch_failed` | 502 | ✅ Yes |
| `output_truncated` | 422 | No (AI ran) — reduce schema fields or page size |
| `model_json_invalid` | 502 | No (AI ran) — safe to retry |
| Code | Status | Meaning |
| ----------------------------------------------------------- | ------ | ----------------------------------------------------------- |
| `invalid_url` / `unsupported_scheme` / `credentials_in_url` | 400 | URL malformed or not http(s) |
| `blocked_domain` / `blocked_ip` / `blocked_port` | 403 | Target not allowed (security policy) |
| `dns_resolution_failed` | 400 | Host did not resolve |
| `url_scrape_failed` / `pdf_scrape_failed` | 500 | Upstream fetch failed — retry or use `advanced_proxy` |
| `pdf_scrape_timeout` | 504 | PDF took too long |
| `pdf_not_supported` | 400 | PDF URL sent to a non-PDF endpoint — use `/get_pdf_content` |
| Code | Status | Meaning |
| ---------------------------------------------------------------------- | --------------- | ----------------------------------------- |
| `missing_query` / `missing_model` / `missing_json_schema` | 400 | Required parameter absent |
| `invalid_search_parameters` | 400 | Bad search\_type/date\_filter combination |
| `query_generation_error` / `search_context_error` | 500 | Search pipeline failure — retry |
| `_rate_limit` / `_auth_error` / `_error` | 429 / 401 / 500 | Upstream AI provider issue |
***
## Need Help?
Chat with other developers
[support@llmlayer.ai](mailto:support@llmlayer.ai)
# Streaming Crawl Examples
Source: https://docs.llmlayer.ai/examples/crawl
Handle page, usage, done, and error events from the Crawl API.
## TypeScript: Save Successful Pages
```typescript theme={null}
import { LLMLayerClient } from 'llmlayer';
import { mkdir, writeFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
await mkdir('crawl-output', { recursive: true });
for await (const event of client.crawlStream({
url: 'https://www.ycombinator.com',
maxPages: 25,
maxDepth: 2,
mainContentOnly: true,
})) {
if (event.type === 'page') {
const page = event.page;
if (!page.success || !page.markdown) {
console.warn('Skipped:', page.final_url, page.error);
continue;
}
const id = createHash('sha1').update(page.final_url || page.requested_url || '').digest('hex');
await writeFile(`crawl-output/${id}.md`, page.markdown);
}
if (event.type === 'usage') {
console.log(`Billed pages: ${event.billed_count}, cost: $${event.cost}`);
}
if (event.type === 'error') {
console.error(event.error);
}
}
```
## Python: Stream Into Memory
```python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
pages = []
for event in client.crawl_stream(
"https://www.ycombinator.com",
max_pages=25,
max_depth=2,
main_content_only=True,
):
event_type = event.get("type")
if event_type == "page":
page = event["page"]
if page.get("success") and page.get("markdown"):
pages.append(page)
else:
print("Skipped:", page.get("final_url"), page.get("error"))
elif event_type == "usage":
print("Billed pages:", event.get("billed_count"))
print("Cost:", event.get("cost"))
elif event_type == "error":
print("Crawl error:", event.get("error"))
print("Saved pages:", len(pages))
```
## Python: Async Crawl
```python theme={null}
import asyncio
from llmlayer import LLMLayerClient
async def main():
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
async for event in client.crawl_stream_async(
"https://www.ycombinator.com",
max_pages=10,
max_depth=1,
main_content_only=True,
):
if event.get("type") == "page":
page = event["page"]
print(page.get("final_url"), bool(page.get("markdown")))
asyncio.run(main())
```
## Notes
* Crawl currently returns markdown page content only.
* Use [Map](/map) first if you need to inspect or filter URLs before fetching content.
* Use the `usage` event as the request-level cost summary.
# Extract Examples
Source: https://docs.llmlayer.ai/examples/extract
Ready-to-adapt Extract API recipes for structured data, lists, Q&A, summaries, and brand enrichment.
## Program Data
```typescript TypeScript theme={null}
const program = await client.extract('https://www.ycombinator.com/about', {
modes: ['json'],
jsonSchema: {
program: 'string',
duration: 'string',
funding: 'string',
benefits: ['string'],
},
instructions: 'Return concise values. Use null when a field is missing.',
});
console.log(program.structured_data);
```
```python Python theme={null}
program = client.extract(
"https://www.ycombinator.com/about",
modes=["json"],
json_schema={
"program": "string",
"duration": "string",
"funding": "string",
"benefits": ["string"],
},
instructions="Return concise values. Use null when a field is missing.",
)
print(program.structured_data)
```
## List Extraction
Use an array in your schema when the page contains repeated items.
```python theme={null}
posts = client.extract(
"https://www.ycombinator.com/blog",
modes=["json"],
json_schema={
"posts": [
{
"title": "string",
"url": "string",
"date": "string",
}
]
},
instructions="Preserve each article URL when available.",
)
for post in posts.structured_data.get("posts", []):
print(post["title"], post.get("date"), post.get("url"))
```
## Page Summary + Q\&A
```typescript theme={null}
const result = await client.extract('https://www.ycombinator.com/blog', {
modes: ['summary', 'qa'],
query: 'What are the three most actionable recommendations?',
responseLanguage: 'en',
});
console.log(result.summary);
console.log(result.answer);
```
## Brand Enrichment
```python theme={null}
brand = client.extract(
"https://www.ycombinator.com",
modes=["brand", "links"],
)
print(brand.brand)
print(brand.links[:10] if brand.links else [])
```
## Combined Workflow
```typescript theme={null}
const site = await client.extract('https://www.ycombinator.com', {
modes: ['summary', 'brand', 'links'],
});
const posts = await client.extract('https://www.ycombinator.com/blog', {
modes: ['json'],
jsonSchema: {
posts: [
{
title: 'string',
url: 'string',
date: 'string',
},
],
},
});
console.log(site.summary);
console.log(site.brand);
console.log(posts.structured_data);
```
## Reliability Tips
* Keep `json_schema` focused on fields visible on the page.
* Use `instructions` for normalization rules, not for unrelated business logic.
* Split very large pages into scrape/search workflows when extraction output gets too large.
* Treat `structured_data` as the only structured extraction result field.
## Related
Main guide and response contract.
Error codes and refund behavior.
# Search + Scrape Pipeline
Source: https://docs.llmlayer.ai/examples/search-scrape
Find relevant pages, scrape full content, and generate a grounded answer.
## TypeScript
```typescript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const search = await client.searchWeb({
query: 'latest research on LLM evaluation',
searchType: 'general',
domainFilter: ['arxiv.org', 'openai.com', '-reddit.com'],
});
const pages = [];
for (const result of search.results.slice(0, 3)) {
const url = result.link;
if (typeof url !== 'string') continue;
const page = await client.scrape(url, {
formats: ['markdown'],
mainContentOnly: true,
});
pages.push({
title: page.title,
url: page.url,
markdown: page.markdown,
});
}
const context = pages
.map((page) => `Source: ${page.title}\nURL: ${page.url}\n${page.markdown}`)
.join('\n\n---\n\n');
const answer = await client.answer({
query: `Use this context to summarize the main LLM evaluation trends:\n\n${context}`,
model: 'llmlayer-web',
returnSources: true,
});
console.log(answer.answer);
```
## Python
```python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
search = client.search_web(
query="latest research on LLM evaluation",
search_type="general",
domain_filter=["arxiv.org", "openai.com", "-reddit.com"],
)
pages = []
for result in search.results[:3]:
url = result.get("link")
if not isinstance(url, str):
continue
page = client.scrape(
url,
formats=["markdown"],
main_content_only=True,
)
pages.append(
{
"title": page.title,
"url": page.url,
"markdown": page.markdown,
}
)
context = "\n\n---\n\n".join(
f"Source: {page['title']}\nURL: {page['url']}\n{page['markdown']}"
for page in pages
)
answer = client.answer(
query=f"Use this context to summarize the main LLM evaluation trends:\n\n{context}",
model="llmlayer-web",
return_sources=True,
)
print(answer.answer)
```
## When to Use This Pattern
* You need to inspect or rank search results before scraping.
* You want tighter control over which pages become LLM context.
* You need to persist scraped markdown before answering.
For simple web-grounded answers, call [Answer](/answer) directly.
# Extract API
Source: https://docs.llmlayer.ai/extract
Extract structured data, summaries, answers, links, and brand profiles from a single web page.
## Overview
Use Extract when you need one page transformed into application-ready data. A single request can combine modes, sharing the same page fetch:
| Mode | Response field | Requires LLM | Cost |
| --------- | ----------------- | -----------: | -------: |
| `json` | `structured_data` | Yes | `$0.005` |
| `summary` | `summary` | Yes | `$0.005` |
| `qa` | `answer` | Yes | `$0.005` |
| `links` | `links` | No | `$0.001` |
| `brand` | `brand` | No | `$0.002` |
All result fields are present on every response. Modes you did not request are `null`.
## Endpoint
`POST /api/v2/extract`
## Quickstart: Structured Data
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const program = await client.extract('https://www.ycombinator.com/about', {
modes: ['json'],
jsonSchema: {
program: 'string',
duration: 'string',
funding: 'string',
benefits: ['string'],
},
instructions: 'Return concise values and use null when a field is missing.',
});
console.log(program.structured_data);
console.log(program.summary); // null
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
program = client.extract(
"https://www.ycombinator.com/about",
modes=["json"],
json_schema={
"program": "string",
"duration": "string",
"funding": "string",
"benefits": ["string"],
},
instructions="Return concise values and use null when a field is missing.",
)
print(program.structured_data)
print(program.summary) # None
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/extract \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.ycombinator.com/about",
"modes": ["json"],
"json_schema": {
"program": "string",
"duration": "string",
"funding": "string",
"benefits": ["string"]
},
"instructions": "Return concise values and use null when a field is missing."
}'
```
## Request Parameters
| Parameter | Type | Required | Default | Description |
| ------------------- | ------------------ | ----------- | ------------ | --------------------------------------------------------- |
| `url` | `string` | Yes | - | Public `http` or `https` page URL |
| `modes` | `string[]` | No | `["json"]` | Any of `json`, `summary`, `qa`, `links`, `brand` |
| `json_schema` | `object \| string` | Conditional | `null` | Required when `modes` includes `json` |
| `query` | `string` | Conditional | `null` | Required when `modes` includes `qa` |
| `instructions` | `string` | No | `null` | Extra guidance for LLM modes: `json`, `summary`, `qa` |
| `response_language` | `string` | No | `auto` | Best for summary and Q\&A text |
| `advanced_proxy` | `boolean` | No | `false` | Use for heavily protected pages |
| `main_content_only` | `boolean \| null` | No | API-selected | Omit to let the API choose the best mode-specific default |
HTTP requests use `snake_case`. The TypeScript SDK uses `camelCase`, for example `jsonSchema`, `responseLanguage`, and `advancedProxy`.
## Response
```json theme={null}
{
"url": "https://www.ycombinator.com/about",
"title": "What Happens at YC | Y Combinator",
"metadata": {
"description": "..."
},
"structured_data": {
"program": "Y Combinator startup program",
"duration": "3 months",
"funding": "$500k per company",
"benefits": ["office hours", "founder community", "Demo Day"]
},
"summary": null,
"answer": null,
"links": null,
"brand": null,
"cost": 0.005,
"response_time": "3.42",
"statusCode": 200
}
```
| Field | Type | Description |
| ----------------- | ---------------- | ---------------------------------- |
| `url` | `string` | Final URL after redirects |
| `title` | `string \| null` | Page title |
| `metadata` | `object \| null` | Page metadata found by the scraper |
| `structured_data` | `object \| null` | Result of `json` mode |
| `summary` | `string \| null` | Result of `summary` mode |
| `answer` | `string \| null` | Result of `qa` mode |
| `links` | `array \| null` | Result of `links` mode |
| `brand` | `object \| null` | Result of `brand` mode |
| `cost` | `number \| null` | Total cost for selected modes |
| `response_time` | `string` | Total processing time in seconds |
| `statusCode` | `integer` | `200` on success |
The structured result field is `structured_data` in the API, Python SDK, and TypeScript SDK.
## Combining Modes
```python theme={null}
profile = client.extract(
"https://www.ycombinator.com",
modes=["summary", "links", "brand"],
)
print(profile.summary)
print(profile.links)
print(profile.brand)
```
```typescript theme={null}
const profile = await client.extract('https://www.ycombinator.com', {
modes: ['summary', 'links', 'brand'],
});
console.log(profile.summary);
console.log(profile.links);
console.log(profile.brand);
```
## JSON Schema Guidance
`json_schema` can be:
* A formal JSON schema
* An example object
* A plain object where values describe expected types
* A plain-text description
For reliable extraction:
* Keep schemas focused.
* Use arrays only when the page clearly contains lists.
* Put normalization rules in `instructions`.
* Use nullable expectations when fields may be missing.
## Errors and Refunds
| Status | Common reason | Charged? |
| ------ | ------------------------------------------------------------- | -------------------------------------------- |
| `400` | Missing `json_schema`, missing `query`, invalid mode, PDF URL | No |
| `422` | Empty extractable content, or JSON output truncated | Depends on whether AI work ran |
| `500` | Page fetch failed | Refunded when failure happens before AI work |
| `502` | Brand fetch or model JSON failure | Depends on failure stage |
See [Errors & Refunds](/errors) for exact error codes and refund behavior.
## More Examples
Products, lists, Q\&A, and brand enrichment.
Full request and response schema.
# Introduction
Source: https://docs.llmlayer.ai/introduction
Search, scrape, crawl, map, extract, and answer with one API for web-aware AI applications.
# LLMLayer
LLMLayer is a web API for AI applications. Use one account and one API key to search the web, scrape pages, crawl websites, map URLs, extract structured data, process PDFs and YouTube transcripts, and generate web-grounded answers.
Create an account and start with free credits.
Inspect every endpoint and schema.
## Choose the Right API
| Job | API | Use it when |
| ------------------------------------------ | ----------------------------------------- | ------------------------------------------------------------------ |
| Answer a question with current web context | [Answer](/answer) | You want an LLM response with optional sources/images |
| Get raw search results | [Web Search](/web-search) | You want to rank, filter, or scrape results yourself |
| Scrape one page | [Scraper](/scrape) | You need markdown, HTML, or a screenshot from a URL |
| Extract data from one page | [Extract](/extract) | You need schema-shaped data, a summary, Q\&A, links, or brand data |
| Crawl multiple pages | [Crawl](/crawl) | You need streamed markdown from a site |
| Discover URLs | [Map](/map) | You need a URL inventory before scraping or crawling |
| Read a PDF | [PDF Content](/scrape-pdf) | You need text from a public PDF URL |
| Read a YouTube video | [YouTube Transcript](/youtube-transcript) | You need transcript text and video metadata |
## Install an SDK
```bash Python theme={null}
pip install llmlayer
```
```bash TypeScript theme={null}
npm install llmlayer
```
## First Request
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
response = client.answer(
query="What changed in AI regulation this week?",
model="llmlayer-web",
search_type="news",
return_sources=True,
)
print(response.answer)
print(response.sources)
```
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.answer({
query: 'What changed in AI regulation this week?',
model: 'llmlayer-web',
searchType: 'news',
returnSources: true,
});
console.log(response.answer);
console.log(response.sources);
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/answer \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "What changed in AI regulation this week?",
"model": "llmlayer-web",
"search_type": "news",
"return_sources": true
}'
```
## Core Patterns
### Structured extraction
```python theme={null}
program = client.extract(
"https://www.ycombinator.com/about",
modes=["json"],
json_schema={
"program": "string",
"duration": "string",
"funding": "string",
"benefits": ["string"],
},
)
print(program.structured_data)
```
### Search then scrape
```typescript theme={null}
const search = await client.searchWeb({
query: 'latest LLM evaluation research',
domainFilter: ['arxiv.org', 'openai.com', '-reddit.com'],
});
const firstUrl = search.results[0]?.link;
if (typeof firstUrl === 'string') {
const page = await client.scrape(firstUrl, {
formats: ['markdown'],
mainContentOnly: true,
});
console.log(page.markdown);
}
```
### Stream a crawl
```python theme={null}
for event in client.crawl_stream("https://www.ycombinator.com", max_pages=10):
if event.get("type") == "page":
print(event["page"].get("final_url"))
```
## Models
The currently documented model IDs are:
| Model | Pricing style | Best for |
| -------------------- | --------------------------- | ---------------------------- |
| `llmlayer-web` | Flat per query | Default web-grounded answers |
| `llmlayer-fast` | Flat per query | Lower latency answers |
| `openai/gpt-4o-mini` | Token priced + LLMLayer fee | Budget token-priced answers |
| `openai/gpt-5.1` | Token priced + LLMLayer fee | Premium reasoning |
`provider_key` is accepted for backward compatibility but currently ignored; provider usage is not billed to the user's provider account.
## Pricing Snapshot
| API | Cost |
| ------------------ | --------------------------------------------------------------------- |
| Answer | `$0.004 × max_queries` + model tokens, or flat LLMLayer model pricing |
| Web Search | `$0.002` per request |
| Scraper | `$0.001` per supported format |
| Extract | `$0.005` per AI mode, `$0.001` links, `$0.002` brand |
| Crawl | `$0.001` per successful page, reported in the usage event |
| Map | `$0.002` per request |
| YouTube Transcript | `$0.003` per request |
| PDF Content | `$0.002` per request |
Use response cost fields and the dashboard ledger as billing source of truth.
## Next Steps
Generate web-grounded answers.
Turn pages into structured data.
Copy production-shaped recipes.
# Map API
Source: https://docs.llmlayer.ai/map
Discover URLs on a website without scraping page content.
## Overview
Use Map when you need a list of URLs from a website before deciding what to scrape, crawl, or analyze. Map discovers links; it does not return page body content.
For multi-page content extraction, use [Crawl](/crawl). For a single page, use [Scraper](/scrape) or [Extract](/extract).
## Endpoint
`POST /api/v2/map`
## Quickstart
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const site = await client.map('https://www.ycombinator.com', {
limit: 100,
search: 'blog',
});
for (const link of site.links.slice(0, 10)) {
console.log(link.url, link.title);
}
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
site = client.map(
"https://www.ycombinator.com",
limit=100,
search="blog",
)
for link in site.links[:10]:
print(str(link.url), link.title)
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/map \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.ycombinator.com",
"limit": 100,
"search": "blog"
}'
```
## Request Parameters
| Parameter | Type | Required | Default | Description |
| ------------------- | ----------------- | -------- | ------- | ------------------------------------- |
| `url` | `string` | Yes | - | Website URL to map |
| `ignoreSitemap` | `boolean` | No | `false` | Ignore sitemap.xml discovery |
| `includeSubdomains` | `boolean` | No | `false` | Include subdomains |
| `search` | `string \| null` | No | `null` | Filter discovered URLs/titles by text |
| `limit` | `integer` | No | `5000` | Maximum links to return |
| `timeout` | `integer \| null` | No | `45000` | Timeout in milliseconds |
The raw API uses camelCase for map request fields: `ignoreSitemap` and `includeSubdomains`. The Python SDK maps these to `ignore_sitemap` and `include_subdomains`.
## Response
```json theme={null}
{
"links": [
{
"url": "https://www.ycombinator.com/blog",
"title": "YC Blog"
}
],
"statusCode": 200,
"cost": 0.002
}
```
| Field | Type | Description |
| ------------ | ---------------- | --------------------------------------- |
| `links` | `array` | Discovered links with `url` and `title` |
| `statusCode` | `integer` | `200` on success |
| `cost` | `number \| null` | Cost in USD |
## Common Patterns
### Map then crawl
Use Map to inspect the URL surface before crawling.
```typescript theme={null}
const site = await client.map('https://www.ycombinator.com', {
limit: 50,
});
const blogPages = site.links.filter((link) => link.url.includes('/blog'));
console.log(blogPages.map((link) => link.url));
```
### Map then scrape selected pages
```python theme={null}
site = client.map("https://www.ycombinator.com", search="blog", limit=20)
for link in site.links[:3]:
page = client.scrape(str(link.url), formats=["markdown"], main_content_only=True)
print(page.title, page.markdown[:200] if page.markdown else "")
```
## Pricing
Map costs `$0.002` per request.
## Errors
| Status | Meaning |
| ------ | ----------------------------------- |
| `400` | Invalid URL or request parameters |
| `401` | Missing or invalid LLMLayer API key |
| `403` | Blocked private/unsafe target |
| `500` | Mapping provider or internal error |
See [Errors & Refunds](/errors) for the shared error format.
## More Examples
Find and process selected pages.
Stream markdown from multiple pages.
# Models and Pricing
Source: https://docs.llmlayer.ai/models-pricing
Supported model IDs and endpoint pricing for LLMLayer.
## Supported Answer Models
| Model | Pricing style | Best for |
| -------------------- | --------------------------- | ---------------------------- |
| `llmlayer-web` | Flat `$0.007 × max_queries` | Default web-grounded answers |
| `llmlayer-fast` | Flat `$0.009 × max_queries` | Fast answers |
| `openai/gpt-4o-mini` | Token priced + LLMLayer fee | Budget token-priced answers |
| `openai/gpt-5.1` | Token priced + LLMLayer fee | Premium reasoning |
The `openai/gpt-5.1` backend mapping is intentionally left for your later backend update.
## Provider Keys
`provider_key` is accepted for backward compatibility but currently ignored; provider usage is not billed to the user's provider account.
Use LLMLayer response fields and the dashboard ledger as the billing source of truth.
## Answer Pricing
### LLMLayer flat-price models
```text theme={null}
llmlayer-web = $0.007 × max_queries
llmlayer-fast = $0.009 × max_queries
```
### Token-priced models
```text theme={null}
total = ($0.004 × max_queries)
+ model input token cost
+ model output token cost
+ optional image search cost
```
Current token prices:
| Model | Input | Output |
| -------------------- | ------------------: | -------------------: |
| `openai/gpt-4o-mini` | `$0.15 / 1M tokens` | `$0.60 / 1M tokens` |
| `openai/gpt-5.1` | `$1.25 / 1M tokens` | `$10.00 / 1M tokens` |
## Utility Endpoint Pricing
| Endpoint | Cost |
| ------------------------------- | --------------------------------------------------------: |
| Web Search | `$0.002` per request |
| Scraper | `$0.001` per supported format |
| Extract `json`, `summary`, `qa` | `$0.005` per selected AI mode |
| Extract `links` | `$0.001` |
| Extract `brand` | `$0.002` |
| Crawl | `$0.001` per successful page, reported in the usage event |
| Map | `$0.002` per request |
| YouTube Transcript | `$0.003` per request |
| PDF Content | `$0.002` per request |
## Cost Control
* Start Answer requests with `llmlayer-web` and `max_queries=1`.
* Use Web Search + Scraper when you need to select sources before asking an LLM.
* Use Extract only for fields you actually need.
* Request one scrape format unless you need multiple outputs.
* Read `cost`, `llmlayer_cost`, and `model_cost` from responses when present.
## FAQ
`provider_key` is accepted for backward compatibility but currently ignored; provider usage is not billed to the user's provider account.
The Answer backend currently falls back to `llmlayer-web` rather than failing the request.
Use the response cost fields for request-level visibility and the dashboard ledger as the account-level billing source of truth.
## Next Steps
Model selection, streaming, and response fields.
Error shapes, status codes, and refund behavior.
# Rate Limits
Source: https://docs.llmlayer.ai/rate-limits
LLMLayer uses tiered rate limiting based on your total credit purchases.
## Rate Limit Tiers
| Tier | Requests/Minute | Total Credits Purchased |
| ---------- | --------------- | ----------------------- |
| **Tier 0** | 20 rpm | default |
| **Tier 1** | 50 rpm | \$50+ |
| **Tier 2** | 100 rpm | \$250+ |
| **Tier 3** | 200 rpm | \$500+ |
| **Tier 4** | 500 rpm | \$1,000+ |
| **Tier 5** | 1,000 rpm | \$3,000+ |
Your tier automatically increases when you purchase credits. The upgrade is instant.
## How It Works
* **Sliding window**: Counts requests made in the last 60 seconds
* **429 errors**: Returned when you exceed your limit
## Upgrading Your Tier
Simply purchase more credits at [llmlayer.ai](https://app.llmlayer.ai). Your tier upgrades instantly based on your total purchase history.
**Need higher limits?** Contact us at [support@llmlayer.ai](mailto:support@llmlayer.ai) for custom enterprise tiers above 1,000 rpm.
# Scraper API
Source: https://docs.llmlayer.ai/scrape
Extract a single web page as markdown, HTML, or a screenshot.
## Overview
Use the Scraper API when you already have a URL and need page content. It supports:
* `markdown` for LLM-ready text
* `html` for raw rendered markup
* `screenshot` for a base64 PNG capture
PDF URLs are not scraped by this endpoint. Use the [PDF Content API](/scrape-pdf) for PDF text extraction.
## Endpoint
`POST /api/v2/scrape`
## Quickstart
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const page = await client.scrape('https://www.ycombinator.com/blog', {
formats: ['markdown'],
mainContentOnly: true,
});
console.log(page.title);
console.log(page.markdown);
console.log(page.statusCode);
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
page = client.scrape(
"https://www.ycombinator.com/blog",
formats=["markdown"],
main_content_only=True,
)
print(page.title)
print(page.markdown)
print(page.statusCode)
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/scrape \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.ycombinator.com/blog",
"formats": ["markdown"],
"main_content_only": true
}'
```
## Formats
| Format | Response field | Best for | Cost |
| ------------ | -------------- | ------------------------------- | -------- |
| `markdown` | `markdown` | LLM input, summaries, retrieval | `$0.001` |
| `html` | `html` | Archival or custom parsing | `$0.001` |
| `screenshot` | `screenshot` | Visual verification | `$0.001` |
You can request multiple formats in one call:
```typescript theme={null}
const page = await client.scrape({
url: 'https://www.ycombinator.com',
formats: ['markdown', 'html', 'screenshot'],
});
```
`pdf` is accepted by some clients for backward compatibility, but this endpoint does not generate PDF output. Direct PDF URLs return a validation error. Use `/api/v2/get_pdf_content`.
## Request Parameters
| Parameter | Type | Required | Default | Description |
| ------------------- | ---------- | -------- | ---------------------- | --------------------------------------- |
| `url` | `string` | Yes | - | Public `http` or `https` page URL |
| `formats` | `string[]` | Yes | `["markdown"]` in SDKs | Any of `markdown`, `html`, `screenshot` |
| `include_images` | `boolean` | No | `true` | Include image references in markdown |
| `include_links` | `boolean` | No | `true` | Include links in markdown |
| `advanced_proxy` | `boolean` | No | `false` | Use for heavily protected sites |
| `main_content_only` | `boolean` | No | `false` | Reduce navigation and boilerplate |
HTTP requests use `snake_case`. The TypeScript SDK uses `camelCase`, for example `advancedProxy` and `mainContentOnly`.
## Response
```json theme={null}
{
"markdown": "# Article title\n\nArticle body...",
"html": null,
"screenshot": null,
"pdf": null,
"url": "https://www.ycombinator.com/blog",
"title": "Article title",
"statusCode": 200,
"cost": 0.001,
"metadata": {
"description": "..."
}
}
```
| Field | Type | Description |
| ------------ | ---------------- | ----------------------------------------- |
| `markdown` | `string \| null` | Markdown content when available/requested |
| `html` | `string \| null` | HTML content when requested |
| `screenshot` | `string \| null` | Base64 PNG when requested |
| `pdf` | `string \| null` | Legacy field; normally `null` |
| `url` | `string` | Final URL after redirects |
| `title` | `string \| null` | Page title |
| `statusCode` | `integer` | Target status code |
| `cost` | `number \| null` | Billed cost |
| `metadata` | `object \| null` | Extracted metadata |
## Pricing
Base cost is `$0.001` per requested supported format. Advanced proxy adds `$0.004` when enabled.
```text theme={null}
markdown only: $0.001
markdown + screenshot: $0.002
markdown + html + screenshot: $0.003
markdown + proxy: $0.005
```
## Errors
| Status | Meaning |
| ------ | ------------------------------------------------------------------------ |
| `400` | Invalid URL, unsupported scheme, DNS failure, or PDF URL sent to Scraper |
| `401` | Missing or invalid LLMLayer API key |
| `403` | Blocked private/unsafe target |
| `500` | Upstream scrape failure |
See [Errors & Refunds](/errors) for the shared error format.
## More Examples
Search the web, scrape pages, and answer from collected context.
Use structured extraction when you need schema-shaped data.
# PDF Content API
Source: https://docs.llmlayer.ai/scrape-pdf
Extract text from a publicly accessible PDF URL.
## Overview
Use PDF Content for direct PDF URLs. Do not send PDF URLs to Scraper, Crawl, or Extract.
## Endpoint
`POST /api/v2/get_pdf_content`
## Quickstart
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
response = client.get_pdf_content("https://www.ycombinator.com/blog/content/files/2024/06/RemedyFest-Final-Report.pdf")
print(response.text)
print(response.pages)
print(response.statusCode)
```
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.getPdfContent('https://www.ycombinator.com/blog/content/files/2024/06/RemedyFest-Final-Report.pdf');
console.log(response.text);
console.log(response.pages);
console.log(response.statusCode);
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/get_pdf_content \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://www.ycombinator.com/blog/content/files/2024/06/RemedyFest-Final-Report.pdf"}'
```
## Request Parameters
| Parameter | Type | Required | Description |
| --------- | -------- | -------- | --------------------- |
| `url` | `string` | Yes | Public direct PDF URL |
## Response
```json theme={null}
{
"text": "Extracted PDF text...",
"pages": 12,
"url": "https://www.ycombinator.com/blog/content/files/2024/06/RemedyFest-Final-Report.pdf",
"statusCode": 200,
"cost": 0.002
}
```
| Field | Type | Description |
| ------------ | ----------------- | ---------------- |
| `text` | `string` | Extracted text |
| `pages` | `integer \| null` | Number of pages |
| `url` | `string` | PDF URL |
| `statusCode` | `integer \| null` | `200` on success |
| `cost` | `number \| null` | Cost in USD |
## Pricing
PDF Content costs `$0.002` per request.
## Limitations
* The PDF must be publicly accessible.
* Password-protected or private PDFs are not supported.
* The endpoint has a short extraction timeout.
* Use direct PDF URLs, not HTML pages that embed a PDF viewer.
## Errors
| Status | Meaning |
| ------ | ----------------------------------- |
| `400` | Invalid or unsupported URL |
| `401` | Missing or invalid LLMLayer API key |
| `504` | PDF extraction timed out |
| `500` | PDF extraction failed |
See [Errors & Refunds](/errors) for the shared error format.
# Web Search API
Source: https://docs.llmlayer.ai/web-search
Get raw web, news, image, video, shopping, and scholar search results without LLM generation.
## Overview
Use Web Search when you need ranked search results and want to decide what to do with them yourself. It does not call an LLM.
For generated answers with citations, use the [Answer API](/answer). For full page content after search, combine Web Search with the [Scraper API](/scrape).
## Endpoint
`POST /api/v2/web_search`
## Quickstart
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.searchWeb({
query: 'AI regulation updates',
searchType: 'news',
recency: 'week',
location: 'us',
});
for (const result of response.results.slice(0, 3)) {
console.log(result.title, result.link);
}
```
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
response = client.search_web(
query="AI regulation updates",
search_type="news",
recency="week",
location="us",
)
for result in response.results[:3]:
print(result.get("title"), result.get("link"))
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/web_search \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "AI regulation updates",
"search_type": "news",
"recency": "week",
"location": "us"
}'
```
## Search Types
| Type | Use it for | Notes |
| ---------- | -------------------- | ----------------------------------- |
| `general` | Standard web results | Default |
| `news` | Recent news articles | Supports `recency` |
| `shopping` | Product listings | Result fields vary by source |
| `videos` | Video search | Useful before transcript extraction |
| `images` | Image results | Returns image-oriented metadata |
| `scholar` | Academic papers | Best effort availability by query |
## Request Parameters
| Parameter | Type | Required | Default | Description |
| --------------- | ---------- | -------- | --------- | --------------------------------------------------------------- |
| `query` | `string` | Yes | - | Search query |
| `search_type` | `string` | No | `general` | `general`, `news`, `shopping`, `videos`, `images`, or `scholar` |
| `location` | `string` | No | `us` | Country/location hint |
| `recency` | `string` | No | `null` | `hour`, `day`, `week`, `month`, or `year` |
| `domain_filter` | `string[]` | No | `null` | Include domains, or exclude with a leading `-` |
HTTP requests use `snake_case`. The TypeScript SDK uses `camelCase`, for example `searchType` and `domainFilter`.
## Response
```json theme={null}
{
"results": [
{
"title": "Example result",
"link": "https://www.ycombinator.com/blog",
"snippet": "Short description..."
}
],
"cost": 0.002
}
```
| Field | Type | Description |
| --------- | ---------------- | ----------------------------------------------- |
| `results` | `array` | Search results. Shape depends on `search_type`. |
| `cost` | `number \| null` | Cost in USD for the request. |
## Common Patterns
### Domain-constrained search
```python theme={null}
response = client.search_web(
query="retrieval augmented generation",
domain_filter=["arxiv.org", "openai.com", "-reddit.com"],
)
```
### Search then scrape
```typescript theme={null}
const search = await client.searchWeb({
query: 'best practices for LLM evaluation',
searchType: 'general',
});
const firstUrl = search.results[0]?.link;
if (typeof firstUrl === 'string') {
const page = await client.scrape(firstUrl, {
formats: ['markdown'],
mainContentOnly: true,
});
console.log(page.markdown);
}
```
## Pricing
Web Search costs `$0.002` per request.
## Errors
| Status | Meaning |
| ------ | ------------------------------------------ |
| `400` | Missing or invalid query/search parameters |
| `401` | Missing or invalid LLMLayer API key |
| `429` | Rate limit exceeded |
| `500` | Search provider or internal error |
See [Errors & Refunds](/errors) for the shared error format.
## More Examples
Find pages, scrape full content, then answer with sources.
Use search results directly with an LLM.
# YouTube Transcript API
Source: https://docs.llmlayer.ai/youtube-transcript
Extract transcripts and available metadata from public YouTube videos.
## Overview
Use YouTube Transcript when you need transcript text from a public video. The endpoint returns the transcript plus available video metadata.
## Endpoint
`POST /api/v2/youtube_transcript`
## Quickstart
```python Python theme={null}
from llmlayer import LLMLayerClient
client = LLMLayerClient(api_key="YOUR_LLMLAYER_API_KEY")
response = client.get_youtube_transcript(
"https://www.youtube.com/watch?v=dQw4w9WgXcQ",
language="en",
)
print(response.title)
print(response.transcript)
```
```typescript TypeScript theme={null}
import { LLMLayerClient } from 'llmlayer';
const client = new LLMLayerClient({
apiKey: process.env.LLMLAYER_API_KEY,
});
const response = await client.getYouTubeTranscript(
'https://www.youtube.com/watch?v=dQw4w9WgXcQ',
'en',
);
console.log(response.title);
console.log(response.transcript);
```
```bash cURL theme={null}
curl -X POST https://api.llmlayer.dev/api/v2/youtube_transcript \
-H "Authorization: Bearer YOUR_LLMLAYER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"language": "en"
}'
```
## Request Parameters
| Parameter | Type | Required | Description |
| ---------- | ---------------- | -------- | -------------------------------------------------------------- |
| `url` | `string` | Yes | YouTube video URL |
| `language` | `string \| null` | No | Preferred transcript language, for example `en`, `es`, or `fr` |
Supported URL forms include:
* `https://www.youtube.com/watch?v=VIDEO_ID`
* `https://youtu.be/VIDEO_ID`
## Response
```json theme={null}
{
"transcript": "Transcript text...",
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"title": "Video title",
"description": "Video description",
"author": "Channel name",
"views": 123456,
"likes": 1234,
"date": "2026-01-01",
"language": "en",
"cost": 0.003
}
```
| Field | Type | Description |
| ------------- | ----------------- | --------------------------- |
| `transcript` | `string` | Transcript text |
| `url` | `string` | Video URL |
| `title` | `string \| null` | Video title |
| `description` | `string \| null` | Video description |
| `author` | `string \| null` | Channel or author |
| `views` | `integer \| null` | View count when available |
| `likes` | `integer \| null` | Like count when available |
| `date` | `string \| null` | Publish date when available |
| `language` | `string \| null` | Transcript language |
| `cost` | `number \| null` | Cost in USD |
## Pricing
YouTube Transcript costs `$0.003` per request.
## Limitations
* The video must be public.
* The video must have captions/transcripts available.
* Requested languages may not be available on every video.
* Live streams may not have transcripts.
## Errors
| Status | Meaning |
| ------ | ----------------------------------- |
| `400` | Invalid YouTube URL |
| `401` | Missing or invalid LLMLayer API key |
| `500` | Transcript extraction failed |
See [Errors & Refunds](/errors) for the shared error format.