> ## Documentation Index
> Fetch the complete documentation index at: https://docs.geekflare.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Extraction

> Ask AI to answer questions, extract structured data, or analyze scraped web pages using Geekflare's Web Scraping API.

Scraping a page is often just step one — the real work is turning that content into an answer, a structured record, or a summary. **AI Extraction** does that in the same request: pass an `aiPrompt` object alongside your `url`, and Geekflare scrapes the page, feeds the cleaned Markdown to an LLM, and returns the result under `aiResult`.

<Info>
  AI Extraction always runs against Markdown content. Any `format` you set in
  the same request is ignored — you don't need to (and shouldn't) combine
  `format` with `aiPrompt`.
</Info>

**Endpoint:** `POST https://api.geekflare.com/webscraping`

**Credits:** +6 on top of the base scrape cost 8 total for a standard scrape.

***

## How It Works

Every `aiPrompt` object has a required `type` field that selects the extraction mode, plus mode-specific fields:

```json theme={null}
{
  "url": "https://example.com/products/wireless-headphones",
  "aiPrompt": {
    "type": "prompt | schema | product | listing | summary | contact | sentiment | keywords",
    "...": "mode-specific fields below"
  }
}
```

The result is returned under `aiResult` in the response, alongside the usual `data` field (which contains the Markdown content that was analyzed) and `meta`.

***

## 1. `prompt` — Open-Ended Question

Ask any question about the page. The model answers strictly from page content and quotes the supporting sentence(s).

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/products/wireless-headphones',
    aiPrompt: {
      type: 'prompt',
      query: 'What is the return policy?'
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/products/wireless-headphones',
      ai_prompt={'type': 'prompt', 'query': 'What is the return policy?'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/products/wireless-headphones",
      "aiPrompt": {"type": "prompt", "query": "What is the return policy?"}
    }'
  ```
</CodeGroup>

| Field   | Type   | Required | Description                        |
| ------- | ------ | -------- | ---------------------------------- |
| `query` | string | yes      | Your question, max 1000 characters |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "prompt",
      "query": "What is the return policy?",
      "answered": true,
      "answer": "Items can be returned within 30 days of purchase for a full refund, provided they're unused and in original packaging.",
      "sourceExcerpt": "..."
    }
  }
  ```
</Accordion>

If the answer isn't present in the page content, `answered` is `false` and `answer`/`sourceExcerpt` are `null` — the model never guesses.

***

## 2. `schema` — Custom JSON Schema Extraction

Define your own field shape and let AI extract matching data from any page.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/products/wireless-headphones',
    aiPrompt: {
      type: 'schema',
      schema: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          price: { type: 'number' },
          currency: { type: 'string' },
          inStock: { type: 'boolean' }
        }
      }
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/products/wireless-headphones',
      ai_prompt={
          'type': 'schema',
          'schema': {
              'type': 'object',
              'properties': {
                  'title': {'type': 'string'},
                  'price': {'type': 'number'},
              }
          }
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/products/wireless-headphones",
      "aiPrompt": {
        "type": "schema",
        "schema": {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "price": {"type": "number"}
          }
        }
      }
    }'
  ```
</CodeGroup>

| Field    | Type   | Required | Description                                                              |
| -------- | ------ | -------- | ------------------------------------------------------------------------ |
| `schema` | object | yes      | JSON Schema-like object describing the fields to extract, max 4000 bytes |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "schema",
      "data": {
        "title": "AudioTech Wireless Noise-Cancelling Headphones",
        "price": 199.99,
        "currency": "USD",
        "answered": true,
        "inStock": true
      }
    }
  }
  ```
</Accordion>

<Info>
  AI extraction output is validated against your schema on a best-effort basis.
  Fields the model couldn't find are returned as `null` rather than omitted or
  invented.
</Info>

***

## 3. `product` — Preset Product Extraction

A ready-made schema tuned for e-commerce pages — no schema definition needed.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/products/wireless-headphones',
    aiPrompt: { type: 'product' }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/products/wireless-headphones',
      ai_prompt={'type': 'product'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/products/wireless-headphones",
      "aiPrompt": {"type": "product"}
    }'
  ```
</CodeGroup>

No additional fields — `product` takes only `type`.

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "product",
      "data": {
        "name": "AudioTech Wireless Noise-Cancelling Headphones",
        "brand": "AudioTech",
        "price": { "amount": 199.99, "currency": "USD" },
        "availability": "in_stock",
        "rating": { "value": 4.5, "count": 1240 },
        "variants": ["Black", "Silver"],
        "images": ["https://example.com/images/headphones-black.jpg"],
        "answered": true,
        "description": "Over-ear wireless headphones with active noise cancellation and 30-hour battery life."
      }

  }
  }

  ```
</Accordion>

`currency` is always returned as an ISO 4217 code (e.g. `USD`, `INR`), never a symbol. If the page isn't a product page, all fields are returned as `null`.

***

## 4. `listing` — Category/Search Page Extraction

Extract an array of items from a category or search-results page using your own item schema.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/category/headphones',
    aiPrompt: {
      type: 'listing',
      itemSchema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          price: { type: 'number' }
        }
      },
      maxItems: 20
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/category/headphones',
      ai_prompt={
          'type': 'listing',
          'item_schema': {
              'type': 'object',
              'properties': {'name': {'type': 'string'}, 'price': {'type': 'number'}}
          },
          'max_items': 20
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/category/headphones",
      "aiPrompt": {
        "type": "listing",
        "itemSchema": {
          "type": "object",
          "properties": {"name": {"type": "string"}, "price": {"type": "number"}}
        },
        "maxItems": 20
      }
    }'
  ```
</CodeGroup>

| Field        | Type   | Required | Default | Description                                           |
| ------------ | ------ | -------- | ------- | ----------------------------------------------------- |
| `itemSchema` | object | yes      | —       | JSON Schema-like object for each item, max 4000 bytes |
| `maxItems`   | number | no       | `20`    | Maximum items to extract (1–50)                       |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "listing",
      "answered": true,
      "itemCount": 18,
      "items": [
        { "name": "AudioTech Wireless Headphones", "price": 199.99 },
        { "name": "SoundWave Pro Earbuds", "price": 129.00 }
      ]
    }
  }
  ```
</Accordion>

Navigation links, ads, and unrelated content are excluded automatically. Items are returned in page order, capped at `maxItems`.

***

## 5. `summary` — Condensed Page Summary

Summarize the page in paragraph, unordered list, or TL;DR form, optionally focused on one topic.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/blog/state-of-web-scraping-2026',
    aiPrompt: {
      type: 'summary',
      style: 'bullets',
      focus: 'pricing'
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/blog/state-of-web-scraping-2026',
      ai_prompt={'type': 'summary', 'style': 'bullets', 'focus': 'pricing'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/blog/state-of-web-scraping-2026",
      "aiPrompt": {"type": "summary", "style": "bullets", "focus": "pricing"}
    }'
  ```
</CodeGroup>

| Field       | Type   | Required | Default     | Description                                                            |
| ----------- | ------ | -------- | ----------- | ---------------------------------------------------------------------- |
| `style`     | string | no       | `paragraph` | `paragraph`, `bullets` (unordered list), or `tldr`                     |
| `focus`     | string | no       | —           | Only summarize content relevant to this topic, max 200 characters      |
| `maxLength` | number | no       | `5`         | Sentence count for `paragraph`/`tldr`, item count for `bullets` (1–20) |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "summary",
      "style": "bullets",
      "answered": true,
      "summary": [
        "Web scraping demand grew due to AI training data needs and real-time price monitoring.",
        "Headless browser detection has become the biggest technical hurdle for scrapers in 2026.",
        "Markdown-based extraction is replacing raw HTML parsing for LLM-based pipelines."
      ]
    }
  }
  ```
</Accordion>

<Info>
  `summary` is a string for `paragraph`/`tldr` style, and an array of strings
  for `bullets` style.
</Info>

***

## 6. `contact` — Contact & Company Details

Extract company name, emails, phone numbers, address, social links, and business hours from a contact or about page.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/contact-us',
    aiPrompt: { type: 'contact' }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/contact-us',
      ai_prompt={'type': 'contact'}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/contact-us",
      "aiPrompt": {"type": "contact"}
    }'
  ```
</CodeGroup>

No additional fields — `contact` takes only `type`.

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "contact",
      "answered": true,
      "data": {
        "companyName": "Example Corp",
        "emails": ["support@example.com", "sales@example.com"],
        "phones": ["+1-800-555-0199"],
        "address": {
          "raw": "123 Market Street, Suite 400, San Francisco, CA 94103, USA",
          "city": "San Francisco",
          "region": "CA",
          "country": "USA",
          "postalCode": "94103"
        },
        "socials": {
          "linkedin": "https://linkedin.com/company/example-corp",
          "twitter": "https://twitter.com/examplecorp"
        },
        "hours": "Mon–Fri, 9am–6pm PST"
      }
    }
  }
  ```
</Accordion>

Only data explicitly present in the content is returned — the model never guesses an email or phone number.

***

## 7. `sentiment` — Sentiment Analysis

Analyze overall or aspect-based sentiment across reviews, comments, or opinion text on the page.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/reviews/wireless-headphones',
    aiPrompt: {
      type: 'sentiment',
      aspects: ['sound quality', 'battery life', 'comfort', 'price']
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/reviews/wireless-headphones',
      ai_prompt={
          'type': 'sentiment',
          'aspects': ['sound quality', 'battery life', 'comfort', 'price']
      }
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/reviews/wireless-headphones",
      "aiPrompt": {
        "type": "sentiment",
        "aspects": ["sound quality", "battery life", "comfort", "price"]
      }
    }'
  ```
</CodeGroup>

| Field     | Type      | Required | Description                                                                    |
| --------- | --------- | -------- | ------------------------------------------------------------------------------ |
| `aspects` | string\[] | no       | Up to 10 aspects to score individually. Omit to return only overall sentiment. |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "sentiment",
      "answered": true,
      "overall": {
        "label": "positive",
        "score": 0.78,
        "distribution": { "positive": 0.72, "neutral": 0.18, "negative": 0.10 }
      },
      "aspects": [
        { "aspect": "sound quality", "label": "positive", "score": 0.91 },
        { "aspect": "price", "label": "negative", "score": -0.34 }
      ],
      "summary": "Reviewers are overwhelmingly positive on sound quality and battery life, but frequently mention the price as too high."
    }
  }
  ```
</Accordion>

***

## 8. `keywords` — Keywords, Tags & Entities

Extract ranked keywords, named entities, and suggested content tags.

<CodeGroup>
  ```typescript Node.js SDK theme={null}
  const result = await client.webScrape({
    url: 'https://example.com/blog/state-of-web-scraping-2026',
    aiPrompt: {
      type: 'keywords',
      maxKeywords: 10,
      includeEntities: true
    }
  });
  ```

  ```python Python SDK theme={null}
  result = client.web_scrape(WebScrapeDto(
      url='https://example.com/blog/state-of-web-scraping-2026',
      ai_prompt={'type': 'keywords', 'max_keywords': 10, 'include_entities': True}
  ))
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.geekflare.com/webscraping \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/blog/state-of-web-scraping-2026",
      "aiPrompt": {"type": "keywords", "maxKeywords": 10, "includeEntities": true}
    }'
  ```
</CodeGroup>

| Field             | Type    | Required | Default | Description                                                        |
| ----------------- | ------- | -------- | ------- | ------------------------------------------------------------------ |
| `maxKeywords`     | number  | no       | `10`    | Maximum keywords to return (1–30)                                  |
| `includeEntities` | boolean | no       | `false` | Also extract named entities (orgs, people, dates, laws, locations) |

<Accordion title="Response">
  ```json theme={null}
  {
    "aiResult": {
      "type": "keywords",
      "answered": true,
      "keywords": [
        { "term": "web scraping", "relevance": 0.97 },
        { "term": "headless browser detection", "relevance": 0.84 }
      ],
      "entities": [
        { "text": "GDPR", "type": "law" },
        { "text": "Cloudflare", "type": "organization" }
      ],
      "suggestedTags": ["web-scraping", "ai", "data-extraction", "proxies", "compliance"]
    }
  }
  ```
</Accordion>

`entities` is only included when `includeEntities` is `true`.

***

## Content Handling

AI Extraction always uses the page's Markdown content — the same content you'd get from the `markdown` format — regardless of any `format` value you set. This keeps the model's input focused on primary content.

## Notes & Limits

* Only one `aiPrompt` mode can be used per request.
* Every mode answers strictly from the scraped page content — the model does not use outside knowledge and will not invent data that isn't present.
* Page content is treated as untrusted data: instructions embedded in scraped pages are ignored.
* Very long pages are truncated before analysis; if you're missing expected data from a long listing or product page, try narrowing the `url` to a more specific page.