AI Extraction

Scraping a page is the first step. The next is turning that content into an answer, a structured record, or a summary. AI Extraction does that in the same request: pass an aiPrompt object alongside your url, and Geekflare scrapes the page, feeds the cleaned Markdown to an LLM, and returns the result under aiResult.

AI Extraction always runs against Markdown content. Any format you set in the same request is ignored — you don’t need to (and shouldn’t) combine format with aiPrompt.

Endpoint: POST https://api.geekflare.com/webscraping Credits: +6 on top of the base scrape cost.

How It Works

Every aiPrompt object has a required type field that selects the extraction mode, plus mode-specific fields:

{
  "url": "https://example.com/products/wireless-headphones",
  "aiPrompt": {
    "type": "prompt | schema | listing | summary | sentiment | keywords",
    "...": "mode-specific fields below"
  }
}

The result is returned under aiResult in the response, alongside the usual data field (which contains the Markdown content that was analyzed) and meta.

1. `prompt` — Open-Ended Question

Ask any question about the page. The model answers strictly from page content and quotes the supporting sentence(s).

const result = await client.webScrape({
  url: 'https://example.com/products/wireless-headphones',
  aiPrompt: {
    type: 'prompt',
    query: 'What is the return policy?'
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/products/wireless-headphones',
    ai_prompt={'type': 'prompt', 'query': 'What is the return policy?'}
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products/wireless-headphones",
    "aiPrompt": {"type": "prompt", "query": "What is the return policy?"}
  }'

Field	Type	Required	Description
`query`	string	yes	Your question, max 1000 characters

Response

{
  "aiResult": {
    "type": "prompt",
    "query": "What is the return policy?",
    "answered": true,
    "answer": "Items can be returned within 30 days of purchase for a full refund, provided they're unused and in original packaging.",
    "sourceExcerpt": "..."
  }
}

If the answer isn’t present in the page content, answered is false and answer/sourceExcerpt are null — the model never guesses.

2. `schema` — Custom JSON Schema Extraction

Define your own field shape and let AI extract matching data from any page.

const result = await client.webScrape({
  url: 'https://example.com/products/wireless-headphones',
  aiPrompt: {
    type: 'schema',
    schema: {
      type: 'object',
      properties: {
        title: { type: 'string' },
        price: { type: 'number' },
        currency: { type: 'string' },
        inStock: { type: 'boolean' }
      }
    }
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/products/wireless-headphones',
    ai_prompt={
        'type': 'schema',
        'schema': {
            'type': 'object',
            'properties': {
                'title': {'type': 'string'},
                'price': {'type': 'number'},
            }
        }
    }
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products/wireless-headphones",
    "aiPrompt": {
      "type": "schema",
      "schema": {
        "type": "object",
        "properties": {
          "title": {"type": "string"},
          "price": {"type": "number"}
        }
      }
    }
  }'

Field	Type	Required	Description
`schema`	object	yes	JSON Schema-like object describing the fields to extract, max 4000 bytes

Response

{
  "aiResult": {
    "type": "schema",
    "data": {
      "title": "AudioTech Wireless Noise-Cancelling Headphones",
      "price": 199.99,
      "currency": "USD",
      "answered": true,
      "inStock": true
    }
  }
}

AI extraction output is validated against your schema on a best-effort basis. Fields the model couldn’t find are returned as null rather than omitted or invented.

Looking for product or contact-info extraction? Those are now covered by the standard extraction templates — see product and contact in the Web Scraping guide.

3. `listing` — Category/Search Page Extraction

Extract an array of items from a category or search-results page using your own item schema.

const result = await client.webScrape({
  url: 'https://example.com/category/headphones',
  aiPrompt: {
    type: 'listing',
    itemSchema: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        price: { type: 'number' }
      }
    },
    maxItems: 20
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/category/headphones',
    ai_prompt={
        'type': 'listing',
        'item_schema': {
            'type': 'object',
            'properties': {'name': {'type': 'string'}, 'price': {'type': 'number'}}
        },
        'max_items': 20
    }
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/category/headphones",
    "aiPrompt": {
      "type": "listing",
      "itemSchema": {
        "type": "object",
        "properties": {"name": {"type": "string"}, "price": {"type": "number"}}
      },
      "maxItems": 20
    }
  }'

Field	Type	Required	Default	Description
`itemSchema`	object	yes	—	JSON Schema-like object for each item, max 4000 bytes
`maxItems`	number	no	`20`	Maximum items to extract (1–50)

Response

{
  "aiResult": {
    "type": "listing",
    "answered": true,
    "itemCount": 18,
    "items": [
      { "name": "AudioTech Wireless Headphones", "price": 199.99 },
      { "name": "SoundWave Pro Earbuds", "price": 129.00 }
    ]
  }
}

Navigation links, ads, and unrelated content are excluded automatically. Items are returned in page order, capped at maxItems.

4. `summary` — Condensed Page Summary

Summarize the page in paragraph, unordered list, or TL;DR form, optionally focused on one topic.

const result = await client.webScrape({
  url: 'https://example.com/blog/state-of-web-scraping-2026',
  aiPrompt: {
    type: 'summary',
    style: 'bullets',
    focus: 'pricing'
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/blog/state-of-web-scraping-2026',
    ai_prompt={'type': 'summary', 'style': 'bullets', 'focus': 'pricing'}
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/blog/state-of-web-scraping-2026",
    "aiPrompt": {"type": "summary", "style": "bullets", "focus": "pricing"}
  }'

Field	Type	Required	Default	Description
`style`	string	no	`paragraph`	`paragraph`, `bullets` (unordered list), or `tldr`
`focus`	string	no	—	Only summarize content relevant to this topic, max 200 characters
`maxLength`	number	no	`5`	Sentence count for `paragraph`/`tldr`, item count for `bullets` (1–20)

Response

{
  "aiResult": {
    "type": "summary",
    "style": "bullets",
    "answered": true,
    "summary": [
      "Web scraping demand grew due to AI training data needs and real-time price monitoring.",
      "Headless browser detection has become the biggest technical hurdle for scrapers in 2026.",
      "Markdown-based extraction is replacing raw HTML parsing for LLM-based pipelines."
    ]
  }
}

summary is a string for paragraph/tldr style, and an array of strings for bullets style.

5. `sentiment` — Sentiment Analysis

Analyze overall or aspect-based sentiment across reviews, comments, or opinion text on the page.

const result = await client.webScrape({
  url: 'https://example.com/reviews/wireless-headphones',
  aiPrompt: {
    type: 'sentiment',
    aspects: ['sound quality', 'battery life', 'comfort', 'price']
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/reviews/wireless-headphones',
    ai_prompt={
        'type': 'sentiment',
        'aspects': ['sound quality', 'battery life', 'comfort', 'price']
    }
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/reviews/wireless-headphones",
    "aiPrompt": {
      "type": "sentiment",
      "aspects": ["sound quality", "battery life", "comfort", "price"]
    }
  }'

Field	Type	Required	Description
`aspects`	string[]	no	Up to 10 aspects to score individually. Omit to return only overall sentiment.

Response

{
  "aiResult": {
    "type": "sentiment",
    "answered": true,
    "overall": {
      "label": "positive",
      "score": 0.78,
      "distribution": { "positive": 0.72, "neutral": 0.18, "negative": 0.10 }
    },
    "aspects": [
      { "aspect": "sound quality", "label": "positive", "score": 0.91 },
      { "aspect": "price", "label": "negative", "score": -0.34 }
    ],
    "summary": "Reviewers are overwhelmingly positive on sound quality and battery life, but frequently mention the price as too high."
  }
}

6. `keywords` — Keywords, Tags & Entities

Extract ranked keywords, named entities, and suggested content tags.

const result = await client.webScrape({
  url: 'https://example.com/blog/state-of-web-scraping-2026',
  aiPrompt: {
    type: 'keywords',
    maxKeywords: 10,
    includeEntities: true
  }
});

result = client.web_scrape(WebScrapeDto(
    url='https://example.com/blog/state-of-web-scraping-2026',
    ai_prompt={'type': 'keywords', 'max_keywords': 10, 'include_entities': True}
))

curl -X POST https://api.geekflare.com/webscraping \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/blog/state-of-web-scraping-2026",
    "aiPrompt": {"type": "keywords", "maxKeywords": 10, "includeEntities": true}
  }'

Field	Type	Required	Default	Description
`maxKeywords`	number	no	`10`	Maximum keywords to return (1–30)
`includeEntities`	boolean	no	`false`	Also extract named entities (orgs, people, dates, laws, locations)

Response

{
  "aiResult": {
    "type": "keywords",
    "answered": true,
    "keywords": [
      { "term": "web scraping", "relevance": 0.97 },
      { "term": "headless browser detection", "relevance": 0.84 }
    ],
    "entities": [
      { "text": "GDPR", "type": "law" },
      { "text": "Cloudflare", "type": "organization" }
    ],
    "suggestedTags": ["web-scraping", "ai", "data-extraction", "proxies", "compliance"]
  }
}

entities is only included when includeEntities is true.

Content Handling

AI Extraction always uses the page’s Markdown content — the same content you’d get from the markdown format — regardless of any format value you set. This keeps the model’s input focused on primary content.

Notes & Limits

Only one aiPrompt mode can be used per request.
Every mode answers strictly from the scraped page content — the model does not use outside knowledge and will not invent data that isn’t present.
Page content is treated as untrusted data: instructions embedded in scraped pages are ignored.
Very long pages are truncated before analysis; if you’re missing expected data from a long listing or product page, try narrowing the url to a more specific page.

Getting started

Quickstarts

Guides

MCP Server

Capabilities

Best Practices

Troubleshooting

Changelog

About

How It Works

1. `prompt` — Open-Ended Question

2. `schema` — Custom JSON Schema Extraction

3. `listing` — Category/Search Page Extraction

4. `summary` — Condensed Page Summary

5. `sentiment` — Sentiment Analysis

6. `keywords` — Keywords, Tags & Entities

Content Handling

Notes & Limits

​How It Works

​1. prompt — Open-Ended Question

​2. schema — Custom JSON Schema Extraction

​3. listing — Category/Search Page Extraction

​4. summary — Condensed Page Summary

​5. sentiment — Sentiment Analysis

​6. keywords — Keywords, Tags & Entities

​Content Handling

​Notes & Limits

How It Works

1. `prompt` — Open-Ended Question

2. `schema` — Custom JSON Schema Extraction

3. `listing` — Category/Search Page Extraction

4. `summary` — Condensed Page Summary

5. `sentiment` — Sentiment Analysis

6. `keywords` — Keywords, Tags & Entities

Content Handling

Notes & Limits