Skip to main content

LLMLayer - The Web API for AI Agents

LLMLayer provides the complete web toolkit for building AI agents. Search the web, scrape content, crawl websites, map websites, extract data from PDFs and videos, and generate AI-powered answers - all through one unified API.

What is LLMLayer?

LLMLayer is a comprehensive web infrastructure API that gives AI agents the ability to:
  • Generate answers - Combine web data with multiple LLMs for intelligent responses
  • Search the web - General, news, images, videos, shopping, academic papers
  • Extract content - Scrape websites as markdown, HTML, PDFs, or screenshots
  • Process documents - Extract text from PDFs and YouTube transcripts
  • Crawl websites - Map websites and crawl multiple pages with depth control
  • Map websites - Map entire websites and crawl multiple pages
Build AI agents that can search, read, and understand the web - without managing multiple APIs or web scraping infrastructure.

Core APIs

🤖 Answer API

Combine web search with AI models for intelligent responses:
# Web-enhanced AI answers with any of multiple models
answer = client.answer(
    query="What's happening in quantum computing?",
    model="llmlayer-web",  # Recommended - or openai/gpt-5.1, openai/gpt-4o-mini, etc.
    search_type="news",
    return_sources=True
)

🔍 Web Search API

Direct access to web search across multiple content types:
# Search news
news = client.search_web(query="AI regulation", search_type="news", recency="day")

# Search academic papers
papers = client.search_web(query="transformer architecture", search_type="scholar")

# Search with domain filtering
results = client.search_web(
    query="machine learning",
    domain_filter=["arxiv.org", "nature.com", "-reddit.com"]
)

🌐 Scraper API

Extract content from any website in multiple formats:
# Get clean markdown
content = client.scrape(url="https://example.com", formats=["markdown"])

# Capture screenshots
screenshot = client.scrape(url="https://example.com", formats=["screenshot"])

# Get Html of the page
html = client.scrape(url="https://example.com", formats=["html"])

📄 Document Processing APIs

Extract text from various sources:
# PDF text extraction
pdf_content = client.get_pdf_content(url="https://example.com/document.pdf")

# YouTube transcripts
transcript = client.get_youtube_transcript(
    url="https://youtube.com/watch?v=...",
    language="en"
)

Complete Toolkit for AI Agents

Information Gathering

  • Web search (6 types)
  • Domain filtering
  • Time-based search
  • Location targeting
  • Result ranking

Content Extraction

  • Website scraping
  • PDF processing
  • Video transcripts
  • Image extraction
  • HTML parsing

AI Processing

  • multiple LLM models
  • 2 providers
  • Streaming support
  • Structured output
  • Custom prompts

Build Powerful AI Agents

Example: Research Agent

# 1. Search for recent papers
papers = client.search_web(
    query="large language models optimization",
    search_type="scholar"
)

# 2. Extract content from PDFs
for paper in papers.results[:3]:
    if paper.get('pdfUrl'):
        content = client.get_pdf_content(url=paper['pdfUrl'])

# 3. Scrape related websites
websites = client.search_web(query="LLM optimization techniques", search_type="general")
for site in websites.results[:5]:
    page_content = client.scrape(url=site['link'], formats=["markdown"])

# 4. Generate comprehensive analysis
analysis = client.answer(
    query="Summarize the latest LLM optimization techniques",
    model="openai/gpt-5.1",
    system_prompt="You are a research analyst. Synthesize the provided information."
)

Example: Content Monitoring Agent

# Monitor news across multiple sources
news = client.search_web(
    query="renewable energy",
    search_type="news",
    recency="hour",
    location="us"
)

# Extract full articles
articles = []
for item in news.results:
    content = client.scrape(url=item['link'], formats=["markdown"])
    articles.append(content.markdown)

# Generate summary with citations
summary = client.answer(
    query="Summarize today's renewable energy news",
    model="llmlayer-fast",  # Fast answers
    citations=True
)

Supported Models

Access models from leading providers with zero markup:
ProviderModelsUse Case
OpenAIGPT-5.1, GPT-4o-miniVersatile intelligence
LLMLayerllmlayer-web, llmlayer-fastFlat pricing, web search optimized
provider_key is deprecated and currently ignored by the Answer APIs. See the Answer API docs for complete model details.

Transparent Pricing

Pay only for what you use - No subscriptions, no minimums, no model markup
APICostDescription
Answer API0.004/request+modeltokens(orflatpricingwithllmlayerweb/0.004/request + model tokens (or flat pricing with `llmlayer-web`/0.007 and llmlayer-fast/$0.009)Web search + AI generation
Web Search$0.002/requestGeneral, news, images, videos, shopping, scholar
Scraper$0.001/requestMarkdown, HTML, screenshot, PDF
Crawler$0.001/pagecrawling multiple pages
Map$0.002/websitemap entire website
YouTube Transcript$0.003/requestMulti-language transcripts
PDF Content$0.002/requestText extraction from PDFs

Why Build with LLMLayer?

🏗️ Complete Infrastructure

Everything you need for web-aware AI agents in one API:
  • No web scraping setup required
  • No proxy management
  • Built-in error recovery

🔄 Maximum Flexibility

  • Switch between models instantly
  • Keep integrations stable with backward-compatible parameters
  • Choose output formats
  • Control search parameters

⚡ Production Ready

  • 15-second timeouts
  • Automatic retries
  • Comprehensive error codes
  • RESTful API with SDKs

💰 Cost Efficient

  • No model markup
  • Use cheaper models for simple tasks
  • Premium models only when needed
  • Pay-as-you-go pricing

Quick Start Examples

from llmlayer import LLMLayerClient

client = LLMLayerClient(api_key="your-api-key")

# Search the web
results = client.search_web(
query="artificial intelligence news",
search_type="news",
recency="day"
)

# Scrape a website
content = client.scrape(
url="https://example.com/article",
formats=["markdown"]
)

# Extract PDF content
pdf_text = client.get_pdf_content(
url="https://example.com/paper.pdf"
)

# Get YouTube transcript
transcript = client.get_youtube_transcript(
url="https://youtube.com/watch?v=..."
)

# Generate AI answer with web search
answer = client.answer(
query="What are the latest AI breakthroughs?",
model="llmlayer-web",  # Recommended
return_sources=True
)

Common Use Cases

Research Assistants

Search academic papers, extract PDFs, analyze findings

Content Aggregators

Monitor news, scrape articles, generate summaries

Market Intelligence

Track competitors, analyze products, monitor trends

Documentation Tools

Archive websites, process documents, extract knowledge

Media Analyzers

Search images/videos, extract transcripts, analyze content

Q&A Systems

Answer questions with current web information

Getting Started

1

Sign Up

Create account at app.llmlayer.aiGet $2 free credits to start
2

Install SDK

pip install llmlayer
# or
npm install llmlayer
3

Choose Your API

Pick the APIs you need:
  • Web Search for information gathering
  • Scraper for content extraction
  • Document APIs for PDFs/videos
  • Answer API for AI responses
4

Build Your Agent

Combine APIs to create powerful AI agents that understand the web

FAQ

LLMLayer is specifically designed as web infrastructure for AI agents. Instead of juggling multiple services (search APIs, scraping tools, LLM providers), you get everything in one unified API with consistent interfaces and transparent pricing.
Absolutely! Many users use only our Web Search, Scraper, PDF, and YouTube APIs for data collection. The Answer API is optional - use it when you need AI-powered responses.
Our scraping infrastructure handles JavaScript rendering, manages proxies, and includes automatic retries. Endpoints use operation-specific timeouts depending on the task.
The provider_key parameter is deprecated and currently ignored by the Answer APIs. It is still accepted for backward compatibility.
Rate limits depend on your subscription tier. The APIs handle rate limiting gracefully with proper error codes so you can implement retry logic.

Next Steps

Ready to build web-aware AI agents?

Start with $2 free credits. No credit card required.